Skip to main content
Available on all Portkey plans.
The Chat Completions API is the most widely adopted format for LLM interaction. Portkey makes it work with every provider — send the same POST /v1/chat/completions request to OpenAI, Anthropic, Gemini, Bedrock, or any of the 3000+ supported models.

​
Quick Start

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)
Switch model to use any provider — @anthropic-provider/claude-sonnet-4-5-20250514, @google-provider/gemini-2.0-flash, or any of the 3000+ supported models.

​
Using the OpenAI SDK

The Portkey SDK is a superset of the OpenAI SDK, so all Chat Completions methods work identically. The OpenAI SDK also works directly with Portkey’s base URL: 
from openai import OpenAI

client = OpenAI(
    api_key="PORTKEY_API_KEY",
    base_url="https://api.portkey.ai/v1"
)

response = client.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms"}]
)

print(response.choices[0].message.content)

​
System Messages

Set a system prompt using the system role in the messages array: 
response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a pirate. Always respond in pirate speak."},
        {"role": "user", "content": "Say hello."}
    ]
)

​
Streaming

Stream responses token-by-token with stream: true. 
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

stream = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Write a haiku about AI"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

​
Function Calling

Define tools with the tools parameter. Works across all providers that support function calling. 
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in San Francisco?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"]
            }
        }
    }]
)

tool_call = response.choices[0].message.tool_calls[0]
print(f"Function: {tool_call.function.name}")
print(f"Arguments: {tool_call.function.arguments}")

​
Function Call Results

Pass tool results back to continue the conversation: 
response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[
        {"role": "user", "content": "What's the weather in Paris?"},
        {"role": "assistant", "tool_calls": [{"id": "call_123", "type": "function", "function": {"name": "get_weather", "arguments": '{"location": "Paris"}'}}]},
        {"role": "tool", "tool_call_id": "call_123", "content": '{"temp": "22°C", "condition": "sunny"}'}
    ]
)

print(response.choices[0].message.content)

​
Vision

Send images in the content array using the image_url type. Works with all vision-capable models. 
response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image"},
            {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}
        ]
    }]
)

print(response.choices[0].message.content)
Base64-encoded images are also supported — pass a data URL as the url value:
Python
{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ..."}}

​
Structured Output

​
JSON Schema

Force the model to return structured JSON matching a specific schema: 
response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "Extract: John is 30 years old"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"}
                },
                "required": ["name", "age"],
                "additionalProperties": False
            }
        }
    }
)

print(response.choices[0].message.content)  # {"name": "John", "age": 30}

​
JSON Mode

For free-form JSON output without a strict schema: 
response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[{"role": "user", "content": "List 3 programming languages and their main use cases as JSON"}],
    response_format={"type": "json_object"}
)

​
Multi-turn Conversations

Pass the full conversation history in the messages array: 
response = portkey.chat.completions.create(
    model="@openai-provider/gpt-4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "My name is Alice."},
        {"role": "assistant", "content": "Hello Alice! How can I help you?"},
        {"role": "user", "content": "What is my name?"}
    ]
)

print(response.choices[0].message.content)  # "Your name is Alice."

​
Using with Portkey Features

Chat Completions works with all Portkey gateway features:
  • Configs — Route, load balance, and set fallbacks
  • Caching — Cache responses for faster, cheaper calls
  • Guardrails — Add input/output guardrails
  • Observability — Full logging and tracing
portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    config="pp-config-xxx"  # Config with fallbacks, load balancing, etc.
)

response = portkey.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

​
API Reference

OpenAI Chat API Docs

OpenAI specification

API Reference

Portkey Chat Completions reference

Universal API

All three API formats

Function Calling Guide

Detailed function calling guide
Last modified on March 9, 2026