Tools tagged with "Multimodal input"

Slotstream runs Qwen3.8-Flash-Next on Apple Silicon Macs that don't have enough RAM to hold the whole model. It's aimed at people with 16 to 64 GB of memory who want local chat, image questions or a model backend for coding agents. Most model weights stay on the SSD, while frequently used expert networks stay in memory. The full model remains available.

PenEcho is an open-source workspace that puts handwritten notes, diagrams and AI-generated widgets on an infinite canvas. It runs on your computer as a macOS or Windows app, or through Node.js with a browser interface. It's for people who want to work through technical ideas visually and give AI feedback by drawing directly on the result.

Phone Harness connects an AI agent to a real iPhone or Android so it can use apps through their screens. It's for developers who want Claude Code, Codex, or another agent to carry out phone tasks without a dedicated app API. The software is open source under the MIT license, and it doesn't require a jailbreak or an app installed on the phone.

Wan2GP brings video, image, music and speech generation to your own computer, with particular attention to GPUs with limited memory. It's for creators who want several media models in one browser interface. The project builds on Wan-Video/Wan2.1.

Bifrost is a self-hosted AI gateway for developers and teams whose applications use multiple model providers. It puts Ollama, custom model deployments, and cloud services behind one OpenAI-compatible API, so applications can switch models without maintaining a separate integration for each provider.

Pipecat is a Python framework for developers building conversational AI agents that handle speech, video, text, and images. You can run it on your own machine or servers, wherever Python runs. It's open source under the BSD 2-Clause license.

DiffSynth-Studio is a Python diffusion model engine for developers and researchers who want to generate media and train models on their own hardware. It supports large models on consumer GPUs through memory offloading and quantization, with inference and training in the same framework. It's open source under Apache 2.0.

AnythingLLM is an open source AI assistant for people who want to chat with their documents and use AI agents on their own hardware. The desktop app runs on Windows, macOS and Linux, while a Docker deployment supports multiple users on a self-hosted server. No account is required for the desktop app.

vLLM is an open source engine for serving large language models on hardware you control. It suits developers and teams that need to handle many requests through an API while making efficient use of memory and compute. It's licensed under Apache 2.0 and can run with GPUs or on a CPU.

LightRAG combines knowledge graphs with vector search to answer questions across a document collection. It's a self-hosted Python framework for developers building document assistants, particularly where answers depend on relationships between facts in different files, such as legal or financial material.

llama.cpp runs language models on your own hardware and can serve them from a machine you control. It’s an MIT-licensed, open source inference engine for people building local AI apps, running a private model server, or using a model directly from the command line. It supports vision-language models too.

Gemma is Google DeepMind’s family of open-weight AI models for developers building applications that can run on their own hardware. Its range covers compact models for phones and IoT devices alongside larger Gemma 4 models for reasoning on personal computers and servers. Some applications can work offline, keeping model inference on the device. Google AI Studio and Google Cloud are also available for hosted use.

ComfyUI is a local visual AI workspace for artists and technical teams who want to control how images, video, audio, 3D models and text are made. Its node canvas shows each model and processing step, so users can build and adjust workflows without writing code. It runs on your hardware.

IndexTTS, currently IndexTTS-2.5, is a local text-to-speech system that can reproduce a speaker's voice using one reference recording. It's for people creating spoken audio and developers building speech generation into their own applications. Voice identity and emotion have separate controls, so an emotional reference can shape the delivery while a different recording supplies the voice.

Codex CLI is an open source coding assistant that runs on your computer and works with local repositories. It's for developers who want to explore code, make changes, and use their existing development tools from the terminal. It runs on macOS, Windows, and Linux under the Apache 2.0 license.

LLaMA-Factory is an open-source framework for developers and researchers who want to adapt language and multimodal models on their own hardware. It brings training and inference into one toolkit, with support for LLaMA, Qwen3, Qwen3-VL, DeepSeek, Gemma, Mistral and LLaVA. Its license is Apache 2.0.

Cherry Studio is a free, open source desktop app for people who use several AI models and want their conversations in one place. It runs on Windows, macOS and Linux. Chats, settings and knowledge base files are stored on your device.

Screenpipe records screen activity and audio as searchable local history that AI agents can use as context. It's for people who want to recall past work and teams that want agents to draft follow-ups or update work records using what actually happened. Raw history stays on your device by default.

Fish Speech, currently featuring Fish Audio S2 Pro, is a self-hosted text-to-speech system for creators producing narration and developers building voice applications. It combines voice cloning with control over emotion and delivery within a script. Code and model weights use the custom FISH AUDIO RESEARCH LICENSE.

MinerU parses documents locally into structured text for AI agents, RAG systems and knowledge bases. It's for people working with scanned PDFs, academic papers and Office files whose tables, formulas or page layouts need more care than plain text extraction.

Ollama runs language models on your own computer or server. It provides a command-line runner and a local API for people building AI applications or connecting existing tools to models they host themselves. The software is distributed under the MIT license.

SGLang is a self-hosted inference framework for teams that need to serve language and multimodal models on their own hardware. It runs on a single GPU or across distributed clusters and exposes an OpenAI-compatible API. The project is open source under the Apache 2.0 license.

Transformers is a Python library for developers and researchers who want to run pretrained AI models or train their own on hardware they control. It covers language, images, audio, video and multimodal work through a shared way of defining models. The library runs in a local Python environment; pretrained checkpoints are available from the separate Hugging Face Hub.

Off Grid AI runs language models on iOS, Android, macOS, Windows and Linux. You can chat, analyze documents and generate images on your own hardware. The mobile app uses the MIT license, while the desktop app uses AGPL.

ScribeDog is a Markdown editor for writers and note-takers who want AI assistance while keeping their documents on their own hardware. It displays formatted text, tables and images instead of Markdown syntax, but saves ordinary .md files that work with Git and other editors. It has a native desktop app and a self-hosted Server Edition available through Docker.

OpenWhispr is a free, MIT-licensed dictation and meeting transcription app for people who want voice input across their apps with control over where processing happens. It's available on macOS, Windows, Linux and iOS. Local transcription works offline and keeps audio on your device; optional cloud transcription sends audio to the selected provider, whose retention policies apply.

Locally AI is a native app for running language and vision models on recent iPhones, iPads, and Macs. It's for people who want a private AI assistant on their own device, with text, image processing, and voice conversations available without cloud processing. Once a model is downloaded, it works offline and doesn't require an account.

Agent Router is an open source AI gateway for teams whose agents use both model APIs and MCP tools. It runs on a laptop, a dedicated gateway, or Kubernetes, and gives applications one OpenAI-compatible entry point for cloud providers and self-hosted inference. The project uses the Apache 2.0 license.

Raycast is a proprietary desktop launcher that lets you use models running through Ollama within its AI assistant. It's for people who want private, offline conversations while keeping access to Raycast's chat and AI commands. Local model requests go directly to Ollama on your computer, without sending your conversations to Raycast or third parties. Local model access is paid.

CodeProject.AI Server gives developers a shared API for AI tasks that run on their own hardware. It's a self-hosted service for adding image analysis, text processing and generation to applications. Processing stays on the machine running the server, without cloud calls or sending data outside your device or network.

h2oGPT is a self-hosted ChatGPT alternative for people who want to chat with local models and ask questions about their own documents. The project is archived and no longer maintained. It's open source under Apache 2.0, with support for Linux, macOS, Windows and Docker.

Raycast Ollama brings Ollama models into Raycast on macOS and Windows, for people who want AI chat and text assistance within their desktop launcher. It connects to Ollama on your own machine or a remote server you choose. The extension is open source under the MIT license, and local inference doesn't require an Ollama API key.

Lumos is a Chrome extension for asking questions about web pages using models running on your own machine. It's for readers working through long discussions, product reviews, news articles or technical documentation who want summaries and answers tied to the material they're reading. Ollama handles inference locally, without a remote AI server.

FlagEmbedding is an open-source Python toolkit for developers building semantic search or retrieval-augmented generation (RAG) into their own applications. It runs BGE embedding and reranking models, with tools to fine-tune both and evaluate retrieval results. The library uses the MIT license.

Riffusion is a Python library for generating music and audio on your own hardware using Stable Diffusion. It's for developers and musicians who want to experiment with text-driven sound generation or build it into an app. The hobby project is no longer actively maintained.

Self-Operating Computer lets a vision-capable AI model control a desktop by reading the screen and choosing mouse and keyboard actions to carry out a goal. It's a Python framework for developers and researchers exploring AI agents that work through application interfaces. The code uses the MIT license.