Groq

Use Groq’s ultra-fast inference for chat completions, tool calling, and audio processing through Portkey.

Quick Start

Get started with Groq in under 2 minutes:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @groq provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@groq/llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

Add Provider in Model Catalog

Before making requests, add Groq to your Model Catalog:

1. Go to Model Catalog → Add Provider
2. Select Groq
3. Enter your Groq API key
4. Name your provider (e.g., groq)

Complete Setup Guide

See all setup options and detailed configuration instructions

---

Groq Capabilities

Tool Calling

Use Groq’s tool calling feature to trigger external functions:

Supported Models

View Groq models that support tool calling

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

tools = [{
    "type": "function",
    "function": {
        "name": "getWeather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

response = portkey.chat.completions.create(
    model="@groq/llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather like in Delhi?"}
    ],
    tools=tools,
    tool_choice="auto"
)

print(response.choices[0].finish_reason)

Speech to Text (Whisper)

Transcribe or translate audio using Groq’s Whisper model:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@groq")

audio_file = open("/path/to/file.mp3", "rb")

# Transcription
transcription = portkey.audio.transcriptions.create(
  model="whisper-large-v3",
  file=audio_file
)
print(transcription.text)

# Translation
translation = portkey.audio.translations.create(
  model="whisper-large-v3",
  file=audio_file
)
print(translation.text)

Text to Speech

Convert text to natural-sounding audio:

Python

from portkey_ai import Portkey
from pathlib import Path

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@groq")

speech_file_path = Path(__file__).parent / "speech.mp3"
response = portkey.audio.speech.create(
  model="playai-tts",
  voice="Fritz-PlayAI",
  input="Today is a wonderful day to build something people love!"
)

with open(speech_file_path, "wb") as f:
    f.write(response.content)

---

Supported Models

Groq Models

View the complete list of models available on Groq

---

Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your Groq requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation