Kaggle AI server: Ollama on T4 GPUs vs TPU

This comparison examines a Kaggle notebook running Ollama through a Cloudflare tunnel and asks whether it suits a daily coding workflow. The setup uses two Tesla T4 GPUs with 16 GB each. The speaker explains that Ollama splits the quantized 27B model across GPU layers to fit it in memory, then cites a measured output speed of 16.23 tokens per second.

Prompt processing adds another delay. The speaker estimates roughly 92 seconds to process a 30,000-token agent prompt and describes requests approaching a reported Cloudflare timeout. Notebook session and idle limits also affect availability. Restarting requires a new tunnel URL.

The alternative is Kaggle TPU Lab on a TPU v5e-8. The video cites project benchmarks of about 130 output tokens per second and 10,300 prefill tokens per second for Qwen3.8 27B at full 16-bit precision. It also discusses API-key protection and vLLM TPU backend support.

These setups run on Google's remote hardware. The speaker flags Kaggle's acceptable-use rules and public Ollama endpoints without authentication. He recommends Kaggle for experiments, hosted inference for routine coding, and owned hardware when code must stay private.