LLM Quantization and Conversion Tools
Shrink a model to 8 or 4 bits and convert it between formats so it fits your hardware, with bitsandbytes, LLM Compressor or Optimum.
A local LLM inference engine with multimodal models, built-in agent tools, and OpenAI-compatible and Anthropic-compatible APIs for self-hosted apps.
7.8KUpdated 6 days agoMIT
macOS · Windows · Linux · Docker · Web#Code execution#GGUF#Hugging Face integration
mistral.rs is an inference engine for developers who want to run AI models on their own computer or server. It supports browser chat and inference embedded directly in Python or Rust applications.
Agent FrameworksInference Libraries and BindingsInference Servers and ClustersLocal Model RunnersQuantization and Model Conversion+0
A local LLM Python package runs text generation and fine-tuning on Apple Silicon with MLX, Hugging Face model support and quantization.
7.7KUpdated 1 week agoMIT
macOS#Batch processing#Distributed execution#Hugging Face integration
MLX LM is a Python package for developers and researchers who want to generate text and adapt language models on Apple Silicon Macs. It uses MLX for local inference and training, with access through Python or a terminal.
Inference Libraries and BindingsLLM Fine-TuningLocal Model RunnersQuantization and Model ConversionTerminal Chat Clients+0
An LLM training framework with ready-made research scripts, distributed NVIDIA GPU training, and Hugging Face checkpoint conversion via Megatron Bridge.
18.4KUpdated 1 day ago
Docker#Distributed execution#Hugging Face integration#Quantization
Megatron-LM is a Python framework for researchers training large language models on NVIDIA GPU infrastructure. It provides reference training scripts for experimenting with models and studying distributed training.
LLM Libraries and SDKsQuantization and Model ConversionTraining Frameworks+0
A self-hosted LLM serving toolkit with text and vision models, quantization, and OpenAI-compatible APIs, licensed under Apache 2.0.
8.1KUpdated 1 day agoApache-2.0
#Batch processing#Distributed execution#Hugging Face integration
LMDeploy is an open-source toolkit for developers who want to run language and vision-language models on their own GPU hardware. It supports offline batch processing and model servers for applications.
Inference Libraries and BindingsInference Servers and ClustersQuantization and Model Conversion+0
An open-source Python model compression library supports LLM quantization in PyTorch, TensorFlow and JAX, with a focus on Intel hardware.
2.7KUpdated 3 days agoApache-2.0
Linux · Docker#Hugging Face integration#Quantization
Intel Neural Compressor is a Python library for developers who want to compress AI models for deployment on their own hardware or servers. It supports LLMs and other deep learning models, with extensive testing on Intel platforms.
Quantization and Model Conversion+0
An open-source image classification and recognition toolkit supports custom model training, image retrieval, and server or on-device deployment.
5.9KUpdated 1 month agoApache-2.0
Android#ONNX#Quantization#Semantic search
PaddleClas is a PaddlePaddle toolkit for developers and researchers building image classification and recognition applications on their own hardware. It covers fixed-category classifiers and image matching for large or frequently changing catalogs.
Computer Vision ModelsLoRA and Image Model TrainingOn-Device and In-Browser AIQuantization and Model ConversionRAG and Vector Search+0
LogotyperGot an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your project
