Tools tagged with "Distributed execution"

MLX is a machine learning array framework for researchers and developers building and testing models on their own hardware. It supports model training and inference, with APIs for numerical work and neural networks.

SGLang is a self-hosted inference framework for teams serving AI models to applications. It supports production workloads on a single GPU or across distributed clusters, including AI agents and reinforcement learning rollouts.

vLLM serves language models on hardware you control. It's for developers and teams building model APIs that need to handle concurrent requests while using memory and compute efficiently.

LightLLM is a self-hosted LLM inference and serving framework for developers and teams running models on their own GPU servers. Its focus is serving requests efficiently: it schedules work around predicted memory use and handles text encoding, model inference and output decoding asynchronously to improve GPU utilization.

DeepXDE is an open-source Python library for researchers and engineers who use neural networks to solve scientific problems. It combines physics-informed neural networks (PINNs) with operator learning, so you can work with equations and physical constraints alongside training data. Its license is LGPL-2.1.

Acme is a Python reinforcement learning library for researchers who need reference agents and reusable components for their own experiments. Its agents provide baselines for comparing algorithm performance, while the underlying building blocks let researchers develop new approaches without starting from scratch.

Optio coordinates coding agents and automated work for teams using their own infrastructure. A shared feed tracks pull requests, recurring jobs, interactive sessions and agents that stay available between tasks.

Kaito manages LLM inference, fine-tuning and document retrieval services in Kubernetes. It's for teams that want to run models in their own cluster with less manual work sizing GPU capacity and managing deployments.

AgentFlow is a Python framework for developers coordinating coding agents across tasks that depend on one another. It runs Codex, Claude, Kimi, and Pi agents in workflows that can split work and bring the results back together.

YoMo is a self-hosted framework for developers building AI agents that call application tools. It supports distributing inference and tool execution across locations closer to an application's users.

Gensim is a free, open-source Python library for developers and researchers working with large text collections on their own machines or servers. It trains models that capture topics and semantic relationships, then uses those representations to find related documents. It runs on Linux, Windows and macOS under the GNU Lesser General Public License v2.1 (LGPL-2.1).

Nanotron is a Python library for researchers and developers training language models on their own datasets and GPU infrastructure. It supports training on one machine or across multiple servers.

LMCache helps teams serving LLMs reuse context that a model has already processed. It reduces repeated work on long prompts, conversation history and retrieved documents, shortening the wait for the first token and improving throughput.

AIBrix provides infrastructure components for teams serving large language models on Kubernetes. It's aimed at organizations that need to manage inference capacity, request routing and models across a cluster.

DeePMD-kit is open-source software for molecular and materials researchers who want to train and run AI interatomic potentials on their own hardware. It learns atomic interactions from quantum-mechanical reference data, then uses those models in molecular dynamics simulations of molecules, solids and metals. It runs on CPUs and GPUs, with support for CUDA and ROCm source builds, and carries the LGPL-3.0-or-later license.

BentoML is a Python framework for developers who want to turn AI models into services on their own hardware or servers. It handles model serving and application logic for LLMs, image generation, audio, and embeddings.

TabbyAPI is a self-hosted LLM API server for people who want to run ExLlamaV3 models on their own hardware. It’s built for personal projects and small groups; its maintainers don’t recommend it for production servers.

NVIDIA NIXL is a data transfer library for developers building AI inference systems such as NVIDIA Dynamo. It provides a common interface for moving data between inference components.

Liger Kernel is a Python library for developers training or fine-tuning LLMs on their own hardware. Fused kernels reduce memory use during model operations and post-training.

TRL is a Python library for researchers and developers adapting foundation models with their own datasets. It builds on Hugging Face Transformers and supports training on a single GPU or across multiple machines.

Oumi helps ML researchers and teams build specialized AI models for their own tasks. It brings data preparation, training, evaluation, and deployment into one platform, with reusable recipes and components you can customize.

OpenResearch is an MIT-licensed, open-source workspace for people using AI coding agents to conduct research. It gives each research direction a separate agent session and isolated Git worktree, so experiments can proceed in parallel without sharing the same working files. Desktop builds support macOS and Linux; the Windows beta requires Git for Windows. A local browser dashboard is also available.

MONAI is a framework for researchers and developers building AI models for medical images. Its Core library provides medical-specific tools for training and evaluating models within existing PyTorch workflows.

ESPnet is a speech AI toolkit for researchers and developers who want to run pretrained models on their own hardware or train speech systems. Its recipes cover data preparation through evaluation, with published results for comparison.

DeepSpeed is an open-source PyTorch library for developers and researchers training or running large AI models on their own hardware or compute clusters. It helps address memory limits and distribute workloads across devices.

Director's Console is a self-hosted workspace for filmmakers and AI artists who want to develop shot prompts, generate media through ComfyUI, and organize the results by project.

SecretFlow is an open-source Python framework for teams that need to analyze data or train machine learning models across separately held datasets. It brings several privacy-preserving computing methods into one framework, so applications can combine them according to their security assumptions and performance needs.

PaddleNLP is a Python toolkit for developers training and deploying language models on their own infrastructure. Built on PaddlePaddle, it covers pretraining, fine-tuning, model compression and inference across several accelerator families.

UFO³ is a self-hosted AI agent framework for developers and researchers automating tasks across Windows, Linux and Android devices. Its Galaxy coordinator splits a request into connected tasks, assigns them to suitable devices and tracks work that depends on results from another machine. The Python project is open source under the MIT license.

NVIDIA Dynamo is a self-hosted inference framework for teams serving generative AI models across multiple GPUs or server nodes. It coordinates inference engines so teams can manage capacity, response latency and cached context across a cluster.

SpeechBrain is a toolkit for researchers and developers building speech assistants, transcription systems and conversational AI. It provides pretrained models and training recipes for adapting speech, audio and text models to your own data.

The AI Scientist is a self-hosted AI agent for researchers studying questions they can test through code. It generates research ideas, runs experiments, produces plots and writes scientific papers. Its scope is computational research.

DeepEP handles communication between GPUs for teams training or serving mixture-of-experts (MoE) models on their own servers. It moves tokens to the GPUs that hold the selected experts and combines their outputs.

ServiceStack AI Server is a self-hosted gateway for developers and teams whose applications use several AI providers. It runs in a Docker container and gives apps one API integration for local models, cloud LLMs and media processing. Teams can change providers without changing their client integrations.