Overview

This guide demonstrates how to use Pay-i's custom headers with OpenAI's Python SDK to annotate your API requests with important metadata.

Basic Pattern

OpenAI's Python SDK makes it straightforward to add Pay-i annotations through the extra_headers parameter:

import os
from openai import OpenAI
from payi.lib.helpers import create_headers

# Initialize OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Create headers with your annotations
request_headers = create_headers(
    use_case_name="document_summarization",
    user_id="user_123",
    limit_ids=["monthly_budget"],
    request_tags=["summary", "document"]
)

# Apply the headers to your API call
response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Summarize this document for me..."}
    ],
    extra_headers=request_headers  # Apply the annotations here
)

Proxy Routing Example

When using Proxy Routing, the pattern is similar but requires configuring the client to route through Pay-i:

import os
from openai import OpenAI
from payi.lib.helpers import payi_openai_url, create_headers

# API keys (use environment variables in production)
openai_key = os.getenv("OPENAI_API_KEY")
payi_key = os.getenv("PAYI_API_KEY")

# Configure OpenAI client to use Pay-i as a proxy
client = OpenAI(
    api_key=openai_key,
    base_url=payi_openai_url(),  # Use Pay-i's URL as the base
    default_headers={"xProxy-api-key": payi_key}  # Authenticate with Pay-i
)

# Create headers for this specific API call
request_headers = create_headers(
    use_case_name="document_summary",
    user_id="user123",
    limit_ids=["department_budget"],
    request_tags=["summary", "document"]
)

# Make the API call with custom headers
response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": "Summarize the following document."},
        {"role": "user", "content": "Long document text here..."}
    ],
    extra_headers=request_headers  # Apply the annotations here
)

# Access Pay-i tracking information from the response
print(f"Request ID: {response.xproxy_result['request_id']}")
print(f"Cost: {response.xproxy_result['cost']}")

Streaming Responses

When working with streaming responses, the pattern is the same, but be sure to fully consume the stream:

import os
from openai import OpenAI
from payi.lib.helpers import create_headers

client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

headers = create_headers(
    use_case_name="chat_assistant",
    user_id="customer_456",
    limit_ids=["monthly_usage"],
    request_tags=["chat", "streaming"]
)

# Stream with annotations
stream = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Write a short story about a robot."}],
    stream=True,
    extra_headers=headers  # Add annotations here
)

# Important: Fully consume the stream
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

For more information about custom headers and how to use them across different providers, see the Custom Headers documentation.