Portkey setup for an HR policy RAG assistant

This tutorial adds Portkey to an existing HR policy RAG assistant. The speaker explains why a gateway can help manage provider keys, route requests and provide a backup when a model call fails. The demonstrated setup uses Portkey Cloud and hosted Groq models; it does not show how to run models locally.

The walkthrough covers account creation, a Portkey API key in the environment file, Python dependencies and provider integrations. Named slugs identify the primary and backup credentials. Both targets use Groq in this example, though the speaker recommends different providers for a real deployment. Several model test requests fail before he selects a working GPT OSS 20 billion model.

A new gateway.py module uses LangChain's ChatOpenAI interface with Portkey's gateway URL and headers. The speaker defines a fallback configuration, then replaces the assistant's direct ChatGroq initialization with a gateway factory. The existing pipeline continues to call the same get-LLM function. Caching, load balancing, retries and timeouts receive discussion, but the implemented example stays focused on fallback routing.

The final test runs the Streamlit app and inspects requests, tool use, responses and costs in Portkey's dashboard. The guardrail model still calls Groq directly. Moving that call behind the gateway and trying LangSmith's gateway are follow-up assignments.