Article·portkey.ai

gatewayroutingmulti-providerai-gatewayllm-opsapi-managementload-balancingobservability

Portkey AI Gateway

Implement Portkey AI Gateway to centralize and manage your LLM API calls. Route requests, load balance across providers, and monitor performance, enhancing reliability and observability for your AI applications.

beginner15 min4 steps

The play

Sign Up and Get Portkey API Key
Create an account on Portkey.ai. Navigate to your dashboard and obtain your unique Portkey API Key.
Configure LLM Providers
Add your API keys for desired LLM providers (e.g., OpenAI, Anthropic) within the Portkey dashboard. This allows Portkey to manage access to these services.
Make a Request via Portkey Gateway
Instead of calling LLM providers directly, send your requests to the Portkey gateway endpoint. Include your Portkey API Key in the `x-portkey-api-key` header and specify the target provider using `x-portkey-provider`.
Monitor and Optimize
Access the Portkey dashboard to view logs, monitor performance, and analyze usage across your LLM providers. Use these insights to optimize routing and improve application reliability.

Starter code

curl -X POST "https://api.portkey.ai/v1/chat/completions" \
     -H "x-portkey-api-key: YOUR_PORTKEY_API_KEY" \
     -H "x-portkey-provider: openai" \
     -H "x-portkey-api-key-openai: YOUR_OPENAI_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
           "model": "gpt-3.5-turbo",
           "messages": [
             {"role": "user", "content": "Explain the concept of an AI gateway."
           ]
         }'

Source

Articleportkey.ai