vLLM
High-throughput LLM inference engine with PagedAttention — the default choice when you need to squeeze a GPU.
vLLM serves open models with continuous batching and PagedAttention, delivering several times the throughput of naive HuggingFace pipelines on the same card. It exposes an OpenAI-compatible API, so swapping it in front of an existing app is usually a one-line base-url change.
Best for: Self-hosted production inference on NVIDIA/AMD GPUs
Deploy: Self-hostable