Tools tagged with "Multimodal input"

ExtractThinker is a Python library for developers who want to extract structured data from documents using models running locally through Ollama. It also supports OpenAI, Anthropic and Groq. With a cloud provider, extraction sends document content to that service, requires provider credentials and may incur charges.

MFLUX is a Python toolkit for creators and developers who want to generate and edit images on an Apple Silicon Mac. It provides a command-line interface for working with models locally.

EvalScope is a Python evaluation framework for teams comparing AI models or assessing a model service before deployment. It connects scores to the prompts, predictions and reviews behind them so you can examine individual results.

rclip searches folders by what their images contain, without requiring tags or a separate photo library. It's for people who keep image collections on their own computers or servers and want to search them from a terminal.

Infinity Embeddings is a self-hosted model server for developers building semantic search and retrieval-augmented generation applications. It runs retrieval models on your own hardware and can serve several models together.

Sesame CSM is a speech generation model for developers and researchers building voice applications on their own hardware. It generates spoken output from text and audio inputs, including earlier parts of a conversation.

Marqo Open Source is a self-hosted search engine for developers building document search, product discovery, or retrieval for AI applications. The project is deprecated and no longer receives updates.

Whispering Tiger creates live transcripts and translations from audio on your computer for VRChat conversations and streaming captions. It can also read text in game images, so spoken dialogue and visible text can feed into the same translation workflow.

ChatWise is a desktop AI chat app for people who want to use different model providers in one interface. It keeps chat data on your device while sending requests to the provider you choose.

RKLLM is a toolkit for developers building local AI applications on Rockchip boards. It runs language and multimodal models on the chip's neural processing unit (NPU).

Step-Audio 2 is an audio AI model family for developers building voice assistants and applications that interpret recordings. It can reason about what people say and how they say it, as well as sounds beyond speech.

Paperless-GPT adds AI document processing to an existing paperless-ngx archive. It's for people who want to reduce manual sorting and extract text from scanned paperwork.

OpenWhispr is a dictation and meeting transcription app for people who want to write by voice and keep searchable records of conversations. It lets you choose local processing or cloud providers.

rerankers is a Python library for developers building search and retrieval systems. It ranks candidate documents against a query and lets you compare reranking approaches through a shared interface.

Grounding DINO locates objects in images from category names or descriptive text. It's for developers and computer vision researchers building detection or annotation workflows that need labels beyond a fixed category list.

Airux Pocket AI runs AI conversations on Android phones for people who want offline chat and photo analysis without cloud inference. You don't need an account, and downloaded models run directly on your phone.

Data-Juicer is a Python framework for researchers and teams preparing datasets for AI training, evaluation, agents or document retrieval. It processes data on your own machine or across a Ray cluster.

CLIP is an image and text model for developers and researchers who want to classify pictures without building a labeled dataset for each task. You define the categories in natural language, and the model scores how closely each description matches an image.

OpenBrowser is a self-hosted browser automation agent for people who need visual judgment and interaction across web pages. Its server and Chrome extension run locally, but the documented Qwen setup sends model requests to DashScope's cloud API and requires an API key and internet access. Browser execution stays local; model processing uses the remote provider.

Roboflow Inference runs computer vision models on images and camera feeds for teams building inspection, monitoring and image analysis systems. You can host it on your own computer, server or edge device.

Skyvern automates browser tasks for teams handling forms, vendor portals and repeated data collection. It uses vision models to identify page elements, helping workflows adapt when website layouts change.

MemOS gives AI agents persistent memory across conversations and tasks. Developers can use it for assistants that recall preferences, support history or earlier work, with on-device, self-hosted and hosted storage options.

AI Toolkit (ostris) is a training suite for people fine-tuning image and video generation models on their own hardware or rented servers. It provides a browser interface and command-line access to training jobs.

Raycast Ollama is a desktop launcher extension for people who want to chat with models and work on text without leaving Raycast. It connects to Ollama running on your machine or a remote server you choose.

MindFS is a self-hosted browser gateway to AI agents and project files on your workstation, for people who want to continue coding from another computer or phone. It requires an installed agent CLI and includes no AI model. With Claude Code's standard cloud connection, execution happens on your host while prompts and model context go to its remote provider. The installed agent handles model access; MindFS hosts browser access to its work.

verl is a Python library for researchers and engineers training large language models on their own hardware. It implements HybridFlow for reinforcement learning after initial training, with support for supervised fine-tuning too.

ComfyUI is a visual AI workflow editor for creators who want control over generation and editing. Its node canvas lets you connect models and processing steps, inspect their settings and reuse the resulting workflows without writing code.

Alpaca is an AI chat client for people who want to use models on their own device through a graphical interface. Ollama powers its local models, which work offline once downloaded.

MMAudio generates sound for video footage and text prompts. It's for video creators adding audio to silent clips and researchers studying how generated sound matches on-screen action.

MLX VLM is a Python package for developers and researchers who want to run multimodal models on their own Mac. It supports local inference and fine-tuning through MLX.

Codex CLI is a terminal coding assistant that works in your local repository on macOS, Windows and Linux. Developers can use it with models running locally through Ollama, including gpt-oss:120b, or connect to OpenAI through a paid ChatGPT account or API key. With OpenAI access, model requests go to the remote service even though file edits and commands run on your computer.

Chatless is a desktop AI chat client for people who want local models and cloud services in one app. It keeps conversation history on your computer and lets you ask questions about your own documents.

VideoCrafter generates video on your own hardware from written prompts or still images. It's aimed at researchers and people exploring video generation for personal use; the toolbox limits use to personal, research and non-commercial purposes.

Inspect AI is a Python framework for researchers and developers evaluating language models and AI agents. It measures coding, reasoning, knowledge, behavior and multimodal understanding, including tasks where agents must take actions to succeed.