Best Ollama alternatives

Desktop app for discovering, downloading and running local LLMs, with a built-in chat UI and an OpenAI-compatible local server.

C/C++ LLM inference engine and GGUF format that runs quantized models efficiently on CPUs, Apple Silicon and GPUs, with an OpenAI-compatible server.

Open-source, high-throughput inference engine for serving LLMs at scale, using PagedAttention to significantly cut memory waste and boost GPU utilization.

High-performance open-source serving framework for LLMs and multimodal models, with RadixAttention prefix caching and structured output support.