Self-Hosted Embedding and Reranking Servers
Servers that expose embedding and reranker models over an API, like Text Embeddings Inference and Infinity Embeddings.
A local LLM app and server runs MLX models on Apple Silicon, with native multi-token prediction and OpenAI and Anthropic API compatibility.
2.7KUpdated 1 week agoApache-2.0
macOS#Hugging Face integration#MLX#Multimodal input
MTPLX is a native Mac app, command-line tool and local LLM server for people running chat or coding models on Apple Silicon. It uses a model's own multi-token prediction heads to accelerate generation without a separate draft model.
Desktop Chat AppsEmbedding ServersInference Servers and ClustersLocal Model RunnersQuantization and Model Conversion+0
Self-hosted AI model serving platform with an OpenAI-compatible API for language, image and audio models, available on Linux, Windows and macOS.
9.7KUpdated 2 days agoApache-2.0
macOS · Windows · Linux · Docker · Web#Batch processing#llama.cpp backend#Multimodal input
Xinference serves AI models on your own computer or servers for developers and researchers building applications around models they control. It brings chat, document search and media generation into one platform.
Embedding ServersInference Servers and ClustersKubernetes and Cloud-Native AISpeech-to-Text and TranscriptionText-to-Speech and Audiobooks+0
A self-hosted LLM API server that runs ExLlamaV3 models with OpenAI-compatible chat endpoints and support for structured output.
1.5KUpdated 20 hours agoAGPL-3.0
Windows · Linux · Docker#Batch processing#Distributed execution#Hugging Face integration
TabbyAPI is a self-hosted LLM API server for people who want to run ExLlamaV3 models on their own hardware. It’s built for personal projects and small groups; its maintainers don’t recommend it for production servers.
Embedding ServersInference Servers and Clusters+0
A self-hosted text embedding server with a REST API, reranking, CPU and GPU support, and offline operation after downloading model weights.
5.1KUpdated 1 week agoApache-2.0
macOS · Linux · Docker#Batch processing#Hugging Face integration#LLM tracing
Text Embeddings Inference is a self-hosted server for developers building semantic search and retrieval applications. It turns text into embeddings through a REST API and can run offline once model weights are downloaded.
Embedding ServersInference Servers and Clusters+0
Local LLM inference server for Apple Silicon Macs with OpenAI and Anthropic APIs, MLX GPU acceleration, and an Apache 2.0 license.
1.7KUpdated 1 week agoApache-2.0
macOS#Batch processing#MLX#Multimodal input
vllm-mlx is a local LLM inference server for developers who want to connect AI apps and coding assistants to models running on an Apple Silicon Mac. One process exposes both OpenAI-compatible and Anthropic-compatible APIs. It's open source under Apache 2.0 and runs on macOS with at least 8 GB of unified memory.
Embedding ServersInference Servers and ClustersSpeech-to-Text and TranscriptionText-to-Speech and Audiobooks+0
A self-hosted AI application server connects Ollama and vLLM to an OpenAI-compatible API with document search and agent tool calling.
8.5KUpdated 1 day agoMIT
#Agent Skills#Batch processing#Guardrails
OGX, formerly Llama Stack, is an AI application server for developers building chat apps, document search and AI agents. It brings model access and the services those applications need into a server you can host yourself.
Agent FrameworksEmbedding ServersInference Servers and ClustersLLM Gateways and RoutersRAG Frameworks and Pipelines+0
Local AI inference server for Apple Silicon Macs. Use OpenAI and Anthropic SDKs with MLX models, keeping processing on your machine. MIT licensed.
746Updated 6 months agoMIT
macOS#Hugging Face integration#MLX#OpenAI-compatible API
MLX Omni Server lets developers use local models through both OpenAI-compatible and Anthropic-compatible APIs on an Apple Silicon Mac. It's for applications built around those SDKs that need inference on hardware the developer controls. Apple's MLX framework provides hardware acceleration, and all AI processing stays on the Mac.
Embedding ServersImage Generation and EditingInference Servers and ClustersSpeech-to-Text and TranscriptionText-to-Speech and Audiobooks+0
A self-hosted AI inference server for Linux and Windows, with OpenAI-compatible APIs, Intel hardware acceleration and an Apache 2.0 license.
957Updated 22 hours agoApache-2.0
Windows · Linux · Docker#Batch processing#GGUF#MCP
OpenVINO Model Server is a self-hosted inference server for developers and teams serving AI models on their own machines or servers. It uses OpenVINO to run workloads on x86 CPUs, including Xeon, with acceleration for Intel integrated and discrete GPUs and NPUs. It's open source under Apache 2.0.
Embedding ServersInference Servers and ClustersKubernetes and Cloud-Native AI+0
An open-source framework for running LLMs locally on mobile, desktop and browser apps, with LiteRT acceleration and Apache 2.0 licensing.
6.9KUpdated 21 hours agoApache-2.0
macOS · Windows · Linux · iOS · Android · Web#Hugging Face integration#Multimodal input#OpenAI-compatible API
LiteRT-LM is Google's open-source framework for developers building apps that run language models on the user's device. It uses LiteRT for local inference across mobile devices, desktops and browsers, with GPU and NPU acceleration. The project uses the Apache 2.0 license.
Edge and Embedded AIEmbedding ServersInference Libraries and BindingsLLM Libraries and SDKsOn-Device and In-Browser AI+0
A self-hosted embedding and reranking API serves Hugging Face models on CPUs or GPUs, with text, image and audio search support.
3KUpdated 2 days agoMIT
macOS · Docker#Batch processing#Hugging Face integration#Multimodal input
Infinity Embeddings is a self-hosted model server for developers building semantic search and retrieval-augmented generation applications. It runs retrieval models on your own hardware and can serve several models together.
Embedding ServersInference Servers and Clusters+0
A self-hosted text and image search engine with built-in embeddings and an Apache 2.0 license; the open-source project is deprecated.
5KUpdated 7 months agoApache-2.0
Docker#Hugging Face integration#Multimodal input#RAG
Marqo Open Source is a self-hosted search engine for developers building document search, product discovery, or retrieval for AI applications. The project is deprecated and no longer receives updates.
Embedding ServersVector Databases+0
An open-source Kubernetes operator serves AI models through vLLM and Ollama, with an OpenAI-compatible API and scaling from zero.
1.3KUpdated 3 days agoApache-2.0
Web#LoRA#Multimodal input#Ollama integration
KubeAI is a Kubernetes operator for teams serving AI models on their own infrastructure or cloud clusters. It manages model servers and adjusts capacity to demand, including starting models from zero running replicas.
Embedding ServersInference Servers and ClustersKubernetes and Cloud-Native AILLM Gateways and RoutersSpeech-to-Text and Transcription+0
A local embedding pipeline turns documents and media into search vectors with Candle and ONNX backends, GPU support and vector database adapters.
1.3KUpdated 3 weeks agoApache-2.0
Windows · Docker#Batch processing#Hugging Face integration#Multimodal input
EmbedAnything is a local embedding pipeline for developers building semantic search and retrieval-augmented generation (RAG) applications. It turns documents and media into vectors for indexing in a vector database.
Embedding ServersInference Libraries and BindingsRAG Frameworks and Pipelines+0
A self-hosted LLM serving platform that uses llama.cpp for CPU and GPU inference, with load balancing and a web admin panel.
1.7KUpdated 1 week agoApache-2.0
Web#Distributed execution#llama.cpp backend#Multimodal input
Paddler lets product and infrastructure teams serve AI models on their own servers and spread requests across multiple machines. It's built for applications that need text generation or embeddings while keeping inference within infrastructure the team controls.
Embedding ServersInference Servers and ClustersLLM Gateways and Routers+0
LogotyperGot an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your project
