Tools tagged with "MLX"

Slotstream runs Qwen3.8-Flash-Next on Apple Silicon Macs that don't have enough RAM to hold the whole model. It's aimed at people with 16 to 64 GB of memory who want local chat, image questions or a model backend for coding agents. Most model weights stay on the SSD, while frequently used expert networks stay in memory. The full model remains available.

exo is a local LLM runner that combines your devices into a cluster, letting you use models too large for one machine's memory. It's for people who want to run large models on their own hardware and developers connecting existing AI clients to local inference. It runs on macOS and Linux under the Apache 2.0 license.

Noema is a private local AI assistant for iPhone, iPad, Mac, and Vision Pro. It's for people who want to chat with models and work with their own files on Apple hardware without depending on a cloud service. Local chats can stay on your device, and file retrieval runs there too.

LM Studio is a desktop application for downloading and running language models on macOS, Windows and Linux. You can search for models, manage downloads and chat with them through the app. Downloaded models can run offline, including document chat that uses files on your computer.

MinerU parses documents locally into structured text for AI agents, RAG systems and knowledge bases. It's for people working with scanned PDFs, academic papers and Office files whose tables, formulas or page layouts need more care than plain text extraction.

LocalAI runs language models, speech, vision and image generation on hardware you control. It's for developers and teams that want a self-hosted AI server for their apps without sending model requests to a cloud service. Its OpenAI-compatible API works with existing clients, and it also accepts Anthropic, Ollama and ElevenLabs API calls.

Unsloth brings model training and everyday AI use into a desktop app for people who want to run models on their own hardware. Its no-code interface covers chat, fine-tuning and media generation on macOS, Windows and Linux. The Unsloth software is open source under Apache 2.0.

Locally AI is a native app for running language and vision models on recent iPhones, iPads, and Macs. It's for people who want a private AI assistant on their own device, with text, image processing, and voice conversations available without cloud processing. Once a model is downloaded, it works offline and doesn't require an account.

LLMKube is a free, open-source Kubernetes operator for teams and homelab owners running local LLM inference across their own hardware. It manages Linux GPU servers and Apple Silicon Macs together, so a mixed fleet can serve models through the same platform. It uses the Apache 2.0 license.

Jina Embeddings is a family of models that converts content into vectors for retrieval, similarity matching, classification and clustering. It includes multilingual text models and multimodal variants for searching across different media.

MindMac is a native macOS AI chat client for people who want to use local LLMs and cloud services in the same app. Its inline mode lets you ask questions or generate text inside Notes, Mail and browsers without switching to a chat window. It runs on Intel and Apple Silicon Macs with macOS 13 or newer.

Nexa SDK is an on-device AI inference framework for developers building applications that process text, images or audio on users' hardware. It runs models locally across CPUs, GPUs and NPUs, with a shared interface for different backends. Its scope includes language and vision models, speech recognition, speech synthesis and image generation.

Falcon-H1 is a family of language models from the Technology Innovation Institute for developers building local AI applications or serving models on their own hardware. Its hybrid design combines attention with Mamba2 state space models to reduce memory use and speed up inference. Model sizes cover smaller devices and larger server deployments.

ExecuTorch is PyTorch's runtime for developers building AI into mobile apps, desktop software and embedded devices. It runs models on the user's hardware, with support for Android, iOS, Linux, macOS and Windows, as well as microcontrollers. Developers can reuse a PyTorch model across targets, though hardware-specific deployments need their own exported model files.

GPTQModel is a Python toolkit for developers compressing LLMs and running them on their own hardware or servers. It brings model calibration, compression, quality checks and inference into one API, so teams can compare quantization methods without adopting a separate tool for each one.

Fullmoon is an open source chat app for people who want to run language models on their Apple devices and keep conversations local. It supports iOS, iPadOS, macOS and visionOS, with on-device inference optimized for Apple silicon. You can chat fully offline, and the app saves your chat history locally.

VideoLingo is a self-hosted video translation app for creators and educators who need bilingual subtitles or dubbed versions of their videos. It brings transcription, translation and subtitle timing into one browser interface, with dubbing as an optional output. The project is open source under Apache 2.0; a separate hosted service offers subtitle translation and dubbing.

Harbor is a CLI and companion app for people experimenting with AI on their own hardware. It manages a local LLM development environment, connecting model backends to chat interfaces and supporting services so you don't have to configure each connection yourself. It's open source under Apache 2.0.

Magenta RealTime 2 is a local AI music model and synthesis engine for musicians and developers who want to play or build AI musical instruments on a laptop. It generates streaming audio in real time, with open weights and code under the Apache 2.0 license.

Enconvo is a native AI assistant and agent for Mac that can use your screen and selected text as context, then work inside your apps. It's for people who want help with writing, research and everyday tasks alongside the app they're using. A sidebar keeps the agent beside the current app, while text selection tools give quick access to editing and translation.

Moshi is a voice AI model and dialogue framework that can listen while it speaks. It processes speech directly, retaining information such as emotion and non-verbal cues that a text transcription can miss. It's aimed at researchers and developers building spoken AI applications, with local inference and self-hosted server options.

RamaLama runs and serves AI models on your own hardware using OCI containers. It's aimed at developers who want local chat or a self-hosted inference API with a container workflow they can also use in production. The project uses the MIT license.

MLX LM is an open-source Python package for generating text and fine-tuning language models locally on Apple Silicon Macs. Built on MLX, it suits developers and researchers who want to work with models through Python or a terminal, including adapting models to their own tasks. The package uses the MIT license.

Outlines is an open-source Python library for developers who need LLM responses to match a defined structure. It constrains output during generation, reducing the need to repair malformed JSON or retry responses that don't fit an application's requirements. The library uses the Apache 2.0 license.

Kubeflow is a self-hosted AI platform for teams that run machine learning workloads on Kubernetes. It brings model development, training and production workflows into a modular stack that can run on a local laptop, on-premises infrastructure or a cloud Kubernetes cluster. It's open source under Apache 2.0.

Msty Studio is a private AI workspace for chatting with local or hosted models and working with your own documents. It offers a desktop app alongside web and team deployment options. Local model support includes MLX, llama.cpp and Ollama.

Vyact is an open source desktop workspace for people who want to use a local LLM with their documents, email and code. It runs on Apple Silicon Macs, Windows and Linux x64 under the AGPL-3.0 license. Intel Macs aren't supported.

Eclaire is a self-hosted AI assistant for notes, documents, photos, bookmarks and tasks. You can ask questions about saved material, inspect the sources behind an answer and ask the assistant to create notes or update tasks. Scheduled automations handle recurring requests such as a weekly task summary.