Tools tagged with "Streaming inference"

chatz is a self-hosted AI chat app that can display live charts, tables and status panels inside conversations. It's for individuals and teams who want to host their own chat interface, choose their model providers and connect tools. The app is open source under the MIT license.

TranscriptionSuite is a speech-to-text app for people recording lectures, transcribing conversations or dictating documents on their own hardware. Transcription works offline after the app and model downloads, keeping audio and transcripts local.

SAHI is a Python computer vision library for developers detecting small objects in large images, including aerial imagery and surveillance footage. It divides images into smaller regions for detection, helping models find objects that occupy few pixels.

Whisper Mate is a local-first transcription app for people who need text from recordings, calls or audio playing on their Mac. It brings transcription and subtitle work into one editor, with media processing that can stay on the Mac.

Fragments is a Next.js template for developers building their own AI app generator, similar to Claude Artifacts or Vercel v0. It provides a starting point for an interface that generates and runs apps through chat.

Moonshine is an on-device AI toolkit for developers building voice agents and spoken interfaces. Voice processing runs on the device without an account or API keys.

Moshi is a voice AI model and dialogue framework for researchers and developers building spoken applications. It processes speech directly and can listen while it speaks, including information such as emotion and non-verbal cues.

WindsurfAPI, also called DevinAPI, connects coding assistants and other AI clients to models through your own Devin account. The gateway runs on your hardware, but it sends prompts and images to Devin's cloud for inference, so it requires internet access.

PageIndex is a document retrieval engine for developers building question answering over long PDFs, including financial reports, legal documents and technical manuals. It uses an LLM to select relevant sections from a document tree without a vector database or document chunking.

Whispering Tiger creates live transcripts and translations from audio on your computer for VRChat conversations and streaming captions. It can also read text in game images, so spoken dialogue and visible text can feed into the same translation workflow.

Step-Audio 2 is an audio AI model family for developers building voice assistants and applications that interpret recordings. It can reason about what people say and how they say it, as well as sounds beyond speech.

Ollama Python is a client library for developers adding model conversations to Python applications. It connects to a running Ollama service for local use or to Ollama's cloud service for larger models.

Excel AI Assistant is an MIT-licensed add-in for people who want AI to make changes directly in their spreadsheets. It runs on your machine alongside Excel for Mac, Windows or the web, and uses plain-language requests to edit the open workbook.

XTTS v2 generates speech from text on your own hardware, using a voice recording as a reference. It's for developers adding speech to applications and researchers adapting a speech model to their own data.

JamAI Base is an AI backend for developers building chatbots and applications that use their own documents and structured data. Its spreadsheet-like interface brings prompts, generated data and LLM workflows into tables.

MindFS is a self-hosted browser gateway to AI agents and project files on your workstation, for people who want to continue coding from another computer or phone. It requires an installed agent CLI and includes no AI model. With Claude Code's standard cloud connection, execution happens on your host while prompts and model context go to its remote provider. The installed agent handles model access; MindFS hosts browser access to its work.

MCP Gateway & Registry gives platform teams a shared catalog and access gateway for the tools their AI agents and coding assistants use. It addresses scattered server connections, credentials and access policies across an organization.

NeuTTS is a collection of on-device text-to-speech models for developers building voice assistants and speech-enabled apps. It generates audio on your own hardware rather than sending speech generation to a cloud API.

Pydantic AI is a Python SDK for developers building agents into their applications. It uses Pydantic validation to give application code structured results and check the arguments agents pass to tools.

CliRelay is an AI gateway for teams that want to manage coding assistants through a shared service on their own server. It builds on CLIProxyAPI and brings subscription accounts and provider API keys under central control.

MLX Audio is a Python library for developers building speech and audio applications on Apple Silicon Macs. It runs models locally using Apple's MLX framework, with support for generated speech, transcription and audio processing.

OfflineLLM is an on-device AI chat app for Android users who want to run models locally without sending conversations to a cloud service. It works fully offline and has no network permissions.

Nativ is a macOS app for people who want AI chat and coding tools to use models running on their own Mac. It combines a chat workspace, model library, and local inference server.

LiteRT is Google's framework for developers adding machine learning and generative AI to apps that run on users' devices. It succeeds TensorFlow Lite and supports local inference without a server round trip.

MOSS is a conversational language model from Fudan University for developers and researchers who want to run and adapt a local LLM. It includes model weights, dialogue datasets and fine-tuning code.

Voci is a local text-to-speech reader for Windows 10 and 11 that speaks in voices cloned from short audio recordings. It's for people who want to listen to documents or pasted text in a chosen voice without sending the content to a cloud service. After the initial downloads, it works offline and doesn't require an account or API key.

Ollama runs open models for people who want local AI applications, model chat or a backend for coding assistants. It provides a command-line interface and an API, with a choice between inference on your hardware and its cloud service.

Qwen3 is a language model family from Alibaba Cloud's Qwen team for people building chat applications or running models on their own hardware. Its public weights include dense and mixture-of-experts models in smaller and larger sizes.

Qwen3-TTS is a family of local text-to-speech models for developers building speech applications or creating audio with custom voices.

ChatGPT Subtitle Translator translates SRT subtitles for people who need translated text to stay matched to the original entries. It supports local Ollama models and OpenAI-compatible providers, so you can choose where translation runs.

gen.nvim brings LLM text generation into Neovim for developers and writers who want AI help within their editor. It can work with a selected passage or the current buffer, so you can ask about or revise the content you're editing.

Qwen-Agent is a Python framework for developers building assistants that use Qwen models to read files and carry out tasks with tools. It provides ready-made agent components and example applications for custom assistants and browser assistance.

GLiNER is a Python framework for extracting named entities from text using small models that run locally on CPUs and consumer hardware. It's for developers building document processing, personal data detection or search systems who need to choose their own entity types without training a model for each task.

Mirascope is an open-source development library for building AI applications with local models or cloud providers. Its documented local backends include Ollama, vLLM and LM Studio, plus direct inference through MLX on Apple Silicon Macs. Developers can use a shared model interface while keeping inference on their own hardware or sending requests to OpenAI, Anthropic or Google.