MLX LM is a Python package for generating text with and fine-tuning large language models on Apple silicon using Apple's MLX framework. It integrates with the Hugging Face Hub for thousands of models, can quantize and upload models, and supports low-rank and full fine-tuning, including on quantized models. Distributed inference and fine-tuning work through mx.distributed.
Install with pip install mlx-lm or conda install -c conda-forge mlx-lm. Generate with mlx_lm.generate --prompt "How tall is Mt Everest?" or start a chat with mlx_lm.chat. MIT licensed.