Tools tagged with "Structured output"

100+ tools

Paddler

A self-hosted LLM serving platform that uses llama.cpp for CPU and GPU inference, with load balancing and a web admin panel.

1.7KUpdated 1 week agoApache-2.0

Web#Distributed execution#llama.cpp backend#Multimodal input

Paddler lets product and infrastructure teams serve AI models on their own servers and spread requests across multiple machines. It's built for applications that need text generation or embeddings while keeping inference within infrastructure the team controls.

Embedding ServersInference Servers and ClustersLLM Gateways and Routers

Jina Reader

1 video

A self-hosted web extraction API converts pages and documents to Markdown or JSON for LLMs, with Apache 2.0 code, Docker and a hosted service.

12.3KUpdated 6 months agoApache-2.0

Docker#Multimodal input#Structured output

Jina Reader is a web extraction API for developers building AI agents, search tools and systems that answer questions from retrieved documents. It turns page content into Markdown or JSON instead of passing raw HTML to an LLM.

Document Parsing for LLMsWeb Scraping for AI

LangExtract

2 videos

A Python library extracts structured data from text with source links and visual review, using local models through Ollama or cloud models.

39.3KUpdated 5 days agoApache-2.0

Windows · Docker#Ollama integration#OpenAI-compatible API#Source citations

LangExtract is a Python library for developers who need to turn notes, reports or other text into structured data. It links extracted details to their original passages so people can check the results against the document.

Document Parsing for LLMsLLM Libraries and SDKs

Discourse AI

1 video

An AI plugin for self-hosted Discourse forums adds chatbots, semantic search and moderation, with a choice of providers and self-hostable AI API services.

48.1KUpdated 21 hours agoGPL-2.0

Linux · Web#LLM tracing#Multi-user access#Structured output

Discourse AI adds community assistants, search and moderation to a Discourse forum you can host on your own server. It's for forum operators who want AI features inside their community site. Forum hosting and AI processing are separate choices: the plugin supports a choice of providers, and its AI API services have a self-hosting option.

AI Bots for Chat AppsLLM Evaluation and TestingWebsite and Support Chatbots

Instructor

4 videos

An LLM data extraction library that validates outputs with Pydantic, retries failed responses and supports local backends including Ollama and vLLM.

14.1KUpdated 19 hours agoMIT

#llama.cpp backend#Ollama integration#Streaming inference

Instructor is a library for developers who need to turn text into structured data using an LLM. Its focus is extraction, with typed outputs that applications can use and validation rules that check the model's responses.

LLM Libraries and SDKs

LMDeploy

A self-hosted LLM serving toolkit with text and vision models, quantization, and OpenAI-compatible APIs, licensed under Apache 2.0.

8.1KUpdated 1 day agoApache-2.0

#Batch processing#Distributed execution#Hugging Face integration

LMDeploy is an open-source toolkit for developers who want to run language and vision-language models on their own GPU hardware. It supports offline batch processing and model servers for applications.

Inference Libraries and BindingsInference Servers and ClustersQuantization and Model Conversion

Hiring Agent

1 video

A local AI resume scoring tool ranks PDF applications with editable role rubrics and evidence for each score, using Ollama or cloud models.

7.4KUpdated 1 month agoMIT

macOS · Windows · Linux#Ollama integration#OpenAI-compatible API#Structured output

Hiring Agent helps hiring teams rank resume PDFs to decide which applications to review first. HackerRank built it for intern applications, with human reviewers making the hiring decisions.

Document Parsing for LLMsDocuments, Writing and Research

Kition

A desktop AI workspace for macOS and Windows with reviewable document and whiteboard edits, structured tables, and OpenAI or Anthropic connections.

645Updated 3 days agoApache-2.0

macOS · Windows · Linux#Batch processing#Code execution#Human approval

Kition is a desktop AI workspace for writing, research, and project planning. Its agent works directly with editable project files, so you can inspect proposed changes in the documents and boards they affect.

AI Notes and Knowledge BasesAI Workflow AutomationChat With Spreadsheets and DatabasesImage Generator AppsWriting and Email Assistants

Surya

1 video

Local OCR software extracts multilingual text, page layouts and tables from documents on CPU, Apple Silicon or NVIDIA GPUs.

21.6KUpdated 2 days agoApache-2.0

macOS · Web#Batch processing#llama.cpp backend#Multilingual

Surya is a local OCR toolkit for developers who need text and page structure from document images. It handles multilingual documents and preserves reading order for use in document-processing applications.

Document Parsing for LLMsOCR and Document ScanningVision-Language Models

Genkit

1 video

An open-source AI app framework connects to Ollama, Gemini and OpenAI, with self-hosted application logic, tool calling and structured output.

6.6KUpdated 1 day agoApache-2.0

Web#LLM tracing#Multimodal input#Ollama integration

Genkit is Google's framework for developers building AI agents and chatbots into web and mobile apps. It brings model connections, application workflows and testing tools into one framework.

Agent FrameworksLLM Evaluation and TestingLLM Libraries and SDKsPrompt Management and PlaygroundsRAG Frameworks and Pipelines

Outlines

5 videos

An open-source Python library constrains LLM outputs to schemas and grammars, with local model support through Transformers and llama.cpp.

16KUpdated 3 months agoApache-2.0

#llama.cpp backend#MLX#Ollama integration

Outlines is a Python library for developers who need LLM responses in a defined format. It constrains generation to produce structured data for tasks such as extracting service tickets from emails or limiting an answer to allowed choices.

LLM Libraries and SDKs

Text Generation Inference

1 video

A self-hosted LLM inference server with an OpenAI-compatible chat API, multi-GPU support and an Apache 2.0 license; its repository is archived.

10.9KUpdated 8 months agoApache-2.0

Linux · Docker#Batch processing#Distributed execution#Hugging Face integration

Text Generation Inference (TGI) serves LLMs on your own hardware for developers building chat apps and text generation services. The project is archived and no longer receives updates.

Inference Servers and Clusters

TensorZero

1 video

A self-hosted LLM gateway with Ollama support, response evaluation and adaptive A/B testing, licensed under Apache 2.0 and no longer maintained.

11.7KUpdated 5 months agoApache-2.0

Docker · Web#Batch processing#LLM tracing#Multimodal input

TensorZero is a self-hosted platform for developers who need to compare and improve models in LLM applications. The project is archived and no longer maintained.

LLM Evaluation and TestingLLM Fine-TuningLLM Gateways and RoutersLLM Observability and TracingPrompt Management and Playgrounds

Open Deep Research

1 video

A self-hosted AI research agent with Ollama support, web search and MCP tools, licensed under MIT; the project is archived and no longer maintained.

12.7KUpdated 3 months agoMIT

Windows · Web#Human approval#MCP#Multi-agent workflows

Open Deep Research is a self-hosted AI agent that searches for information and writes research reports for teams that want to choose their models and tools. The project is archived and no longer maintained.

AI Search and Deep Research

Logotyper

Got an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.

Start your project