Tools tagged with "Quantization"

HunyuanVideo is an AI video generation model for creators and developers who want to generate clips on their own GPUs. Downloadable weights and inference code support generation from text prompts or an input image.

LLM Compressor prepares models for deployment with vLLM by reducing their storage and memory requirements through quantization. It's for developers working with local checkpoints or models from Hugging Face repositories.

TRL is a Python library for researchers and developers adapting foundation models with their own datasets. It builds on Hugging Face Transformers and supports training on a single GPU or across multiple machines.

Oumi helps ML researchers and teams build specialized AI models for their own tasks. It brings data preparation, training, evaluation, and deployment into one platform, with reusable recipes and components you can customize.

HQQ is a Python library for developers compressing language and vision models to fit the memory available on their own hardware. Its Half-Quadratic Quantization method doesn't require a calibration dataset.

Ollama JavaScript is a library for developers adding model chat, generation or retrieval features to JavaScript applications. It connects those applications to an Ollama server, including one running locally.

VibeVoice is a family of voice AI models for developers and researchers building transcription and speech generation applications. It can capture who spoke and when, or produce spoken responses from incoming text.

ChatterUI is an Android chat app for people who want to use a local LLM on their phone or connect to a remote model. It supports assistant and character conversations, with control over reply generation and chat formatting.

LiteRT Torch is a Python library for converting PyTorch models into .tflite files for the LiteRT runtime. Developers can prepare models on Linux and deploy them into Android, iOS and IoT applications that run inference on the device. Code is licensed under Apache 2.0.

TTS-Audio-Suite is a ComfyUI extension for creators producing narration, character dialogue and speech matched to subtitles on their own hardware. It brings several speech engines and audio editing tools into the same workspace.

TypeWhisper is a macOS dictation and transcription app for people who want to turn speech into text across their apps. Local speech models keep audio on your Mac; optional cloud transcription sends it to the selected provider.

TensorSharp runs local AI models for .NET developers and people who want chat, media generation and agent tools on their own hardware. You can embed it in a C# application or use its CLI, browser interface or companion app, TensorAgent.

Musubi Tuner provides training scripts for people who want to adapt image and video generation models with their own datasets. It's aimed at users comfortable with a script-based workflow.

EXAONE is a family of language models from LG AI Research for developers building multilingual AI applications on their own hardware. It combines everyday language tasks with reasoning for harder problems and supports agents that call external tools.

PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.

RamaLama runs AI models for local chat or a self-hosted REST API. It's for developers who want a container workflow for trying models on their own hardware and serving them in production.

Natively is a desktop assistant for people who want suggested answers during meetings or interviews and searchable notes afterward. It works with Zoom, Google Meet and Microsoft Teams without adding a bot to the call.

Transformers.js is a JavaScript library for developers adding on-device AI to web apps. It runs models in the browser, so processing inputs doesn't require a separate inference server.

Llama Desktop runs language models on your computer for private chat and use in other apps. It's for people who want a local ChatGPT alternative and developers connecting coding agents, editors or scripts to their own models.

Stable Diffusion Core ML is a developer toolkit for adding local AI image generation to Mac, iPhone, and iPad apps. It pairs model conversion tools with a Swift package for embedding generation in native software.

QMD searches Markdown notes, meeting transcripts, documentation and knowledge bases on your own machine. It's for people who want to find material by its wording or meaning, and make that search available to an AI agent.

vllm-mlx is a local LLM inference server for developers who want to connect AI apps and coding assistants to models running on an Apple Silicon Mac. One process exposes both OpenAI-compatible and Anthropic-compatible APIs. It's open source under Apache 2.0 and runs on macOS with at least 8 GB of unified memory.

UNO is a local AI image generation framework for researchers and creators who want to preserve subjects from reference images in generated scenes. One model handles both single-subject images and compositions with several subjects.

TensorRT Model Optimizer (ModelOpt) is a Python library for developers who need smaller models and faster inference. It accepts Hugging Face, PyTorch and ONNX models and lets developers combine optimization techniques before deployment.

SemanticFinder helps readers and researchers find passages by meaning in long documents. It runs the search model in your browser, so you can search your own text without sending it to a model server.

Augmentoolkit turns documents into datasets for training an LLM on a particular subject. It's for people building custom models around material such as research papers or fictional lore.

Llama is Meta's family of language models for developers, researchers and businesses building AI applications on their own hardware or servers. Downloadable weights let you choose a model for your application and available hardware.

Candle is a Rust machine learning framework for developers embedding local AI in applications or deploying models on their own servers. It supports inference and training, including small CPU deployments for serverless applications.

Personal LLM runs AI chat on your phone for people who want to ask questions without sending them to a cloud model. Chat, photo analysis and document questions work offline after you download a model and any required image support files.

BitNet is Microsoft's local LLM inference framework for developers and researchers working with compatible 1-bit models. Its bitnet.cpp runtime focuses on faster inference and lower energy use on your own hardware.

GPT4All is a local AI chatbot for people who want language models and document conversations on their own computer. Model inference and LocalDocs processing run on the device without a cloud connection.

MiniCPM is a family of compact language models for developers building local assistants and coding agents. It targets on-device workloads and hardware with limited computing resources.

RTranslator helps people hold spoken conversations in different languages using their Android phones. It runs translation locally and works offline.

NeMo Automodel is a Python library for researchers and developers who want to train or adapt AI models on their own GPU workstations or clusters. Its PyTorch-based training works with Hugging Face models without converting checkpoints.