Lucebox

A local AI workstation runs large models offline, with an Apache 2.0 inference engine and APIs compatible with OpenAI and Anthropic clients.

Lucebox is a local AI workstation for developers and teams who want to run models on their own hardware. Its inference engine is also available separately for supported AMD and NVIDIA machines.

Large model inference: Run DeepSeek V4 Flash and Qwen3.8-27B with hardware-tuned models, or load your own GGUF weights; the workstation can split a model across dedicated GPU memory and unified memory.
Coding assistant connections: OpenAI-compatible and Anthropic-compatible endpoints let clients such as Claude Code, Codex, OpenCode and Open WebUI use the local server.
Shared model serving: Serve concurrent users or several agents, and run Qwen and DeepSeek together through one endpoint with Lucebox Mix; available memory limits concurrent long-context workloads.
Images: The engine accepts image input with supported Qwen and DeepSeek models and generates images with MOSS-Image on the Radeon AI PRO R9700.

The Zero 495 runs Ubuntu Server with AMD ROCm, browser-based management and full root access. It pairs an AMD Ryzen AI MAX+ PRO 495 with 128 GB or 192 GB of unified memory and a Radeon AI PRO R9700 with 32 GB of dedicated memory. The separate engine is free under Apache 2.0, with tested hardware including RTX 3090, RTX 4090, RTX 5090 and AMD Radeon cards; Linux and WSL2 runs are supported.

Inference works fully offline once a model is loaded, keeping local workloads on the machine. Updates and optional cloud fallback need internet access. Optional Tailscale access uses a Tailscale account to reach the private API remotely.

GPUs, Clusters and Edge Inference Servers and Clusters Local Model Runners Windows Linux Web Context compression GGUF Multimodal input OpenAI-compatible API Quantization Speculative decoding Works offline

Nobody has claimed Lucebox yet

Work on Lucebox? Claim this page with an email at lucebox.com. Lucebox gets the verified checkmark, and you can upgrade the listing.

Want more people to find it? Promote it

Something wrong or outdated on this page?

Logotyper

Got an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.

Start your project

Similar to Lucebox

View all tools

exo

6 videos

A local LLM runner splits models across macOS and Linux devices and provides OpenAI-, Claude- and Ollama-compatible APIs.

48.4KUpdated 1 week agoApache-2.0

macOS · Linux · Web#Distributed execution#Hugging Face integration#MLX

exo is a local LLM runner for people who want to run models too large for a single machine's memory. It joins devices into a cluster and runs inference on your own hardware.

GPUs, Clusters and EdgeInference Servers and ClustersLocal Model Runners

Distributed Llama

A local LLM runner that shares computation and model memory across networked computers, with Linux, macOS and Windows support under the MIT license.

3.1KUpdated 4 days agoMIT

macOS · Windows · Linux#Distributed execution#Hugging Face integration#Quantization

Distributed Llama runs language models across your own networked computers. It's for people who want to share inference work and model memory across several devices instead of fitting the whole workload on one machine.

GPUs, Clusters and EdgeInference Servers and ClustersLocal Model RunnersTerminal Chat Clients

LocalAI

2 videos

A self-hosted AI runtime that serves local models through an OpenAI-compatible API, with CPU-only support, voice tools and built-in agents.

50.4KUpdated 21 hours agoMIT

macOS · Linux · Docker · Web#Code execution#Human approval#llama.cpp backend

LocalAI is a self-hosted AI runtime for developers and teams that want to run models on their own hardware. It brings language, speech, vision and media generation into one server that existing AI applications can connect to.

Inference Servers and ClustersKubernetes and Cloud-Native AILLM Gateways and RoutersLocal Model Runners

FastLLM

Open-source local LLM inference engine for Linux and Windows, with CPU and GPU support and APIs compatible with OpenAI and Anthropic.

5.1KUpdated 1 week agoApache-2.0

Windows · Linux · Web#Batch processing#Code execution#GGUF

FastLLM is an open-source inference engine for people running large language models on their own machines or self-hosted servers. Its main draw is hardware flexibility: large mixture-of-experts (MoE) models can spread their weights across GPU memory, system RAM and disk when they don't fit entirely on a graphics card.

Inference Libraries and BindingsInference Servers and ClustersLocal Model RunnersSelf-Hosted Chat Web UIs

koboldcpp

4 videos

A local LLM runner for GGUF models on Windows, macOS and Linux, with CPU or GPU support and a browser interface for chat and roleplay.

12.3KUpdated 2 weeks agoAGPL-3.0

macOS · Windows · Linux · Android · Docker · Web#GGUF#Hugging Face integration#llama.cpp backend

KoboldCpp runs language models on your own hardware for chat, creative writing and roleplay. It's a fork of llama.cpp with a bundled KoboldAI Lite browser interface.

AI Characters and RoleplayInference Servers and ClustersLocal Model RunnersSelf-Hosted Chat Web UIsWriting and Email Assistants

mistral.rs

A local LLM inference engine with multimodal models, built-in agent tools, and OpenAI-compatible and Anthropic-compatible APIs for self-hosted apps.

7.8KUpdated 6 days agoMIT

macOS · Windows · Linux · Docker · Web#Code execution#GGUF#Hugging Face integration

mistral.rs is an inference engine for developers who want to run AI models on their own computer or server. It supports browser chat and inference embedded directly in Python or Rust applications.

Agent FrameworksInference Libraries and BindingsInference Servers and ClustersLocal Model RunnersQuantization and Model Conversion