Overview

Pay-i's custom headers provide a straightforward way to annotate your GenAI API calls with important metadata such as use cases, limits, user IDs, and tags. This approach gives you fine-grained control over tracking on a per-request basis.

Basic Concept

Custom headers are the fundamental mechanism for annotating your API calls with important metadata such as use cases, limits, user IDs, and tags. These headers allow Pay-i to properly track, categorize, and control your GenAI usage.

While the concept of annotating requests is consistent across all providers, the implementation details vary slightly depending on the provider SDK's architecture:

OpenAI, Anthropic, Azure OpenAI: Use the extra_headers parameter directly in their SDKs
AWS Bedrock: Requires registering callbacks first for proper header handling due to its unique SDK architecture
LangChain: Uses a callback-based approach instead of headers, as it's an abstraction layer rather than a direct provider SDK

Key advantages of custom headers:

Direct per-call control: Apply different annotations to individual API calls
Fine-grained tracking: Customize tracking based on the specific context of each request
Simplicity: No need for additional function structures or decorators (for most providers)
Flexibility: Works with all major GenAI provider SDKs (with provider-specific implementation details)

The `create_headers()` Helper Function

Pay-i provides a convenient helper function called create_headers() that constructs the correct header format for you:

from payi.lib.helpers import create_headers

# Create headers with various annotations
headers = create_headers(
    use_case_name="summarization",   # The use case this request belongs to
    user_id="jane_doe",              # Who initiated this request
    limit_ids=["budget_a", "budget_b"],  # Budgets to track against (always a list)
    request_tags=["important", "customer"]  # Tags for filtering/analytics
)

This function handles the proper formatting of headers that Pay-i requires. The resulting dictionary can be passed directly to the extra_headers parameter of your API calls.

Basic Example

Here's a simple example showing how to use custom headers with OpenAI:

from openai import OpenAI
from payi.lib.helpers import create_headers

# Initialize client
client = OpenAI(api_key="your-openai-api-key")

# Create headers with annotations
headers = create_headers(
    use_case_name="document_summary",  # Business purpose
    user_id="user123",                 # User attribution
    limit_ids=["department_budget"],   # Budgets (always a list)
    request_tags=["summary", "document"] # Tags for filtering
)

# Make API call with annotations
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Summarize this document..."}],
    extra_headers=headers  # Apply annotations here
)

For complete examples and provider-specific implementation details, see our provider-specific guides:

Provider-Specific Implementation Summary

Pay-i works with all major GenAI providers, but the implementation details vary:

Provider	Annotation Method	Special Considerations
OpenAI	`extra_headers` parameter	Uses `extra_headers` directly
Azure OpenAI	`extra_headers` parameter	Uses `extra_headers` directly
Anthropic	`extra_headers` parameter	Uses `extra_headers` directly
AWS Bedrock	`extra_headers` parameter	Requires callback registration first
LangChain	`PayiCallbackHandler`	Uses callbacks instead of headers

Using Custom Headers with Direct Provider Calls

When using Direct Provider Calls (the Direct Provider Call with Telemetry approach), your application communicates directly with the GenAI provider, and Pay-i captures metrics separately. The pattern remains the same - you use the create_headers() function and pass the result to extra_headers:

from openai import OpenAI
from payi.lib.instrument import payi_instrument
from payi.lib.helpers import create_headers

# Initialize Pay-i instrumentation (required once)
payi_instrument()

# Configure client as normal (direct to provider)
client = OpenAI(api_key="your-openai-api-key")

# Create and apply annotation headers
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Your prompt here"}],
    extra_headers=create_headers(
        use_case_name="customer_support",
        user_id="support_agent_42",
        limit_ids=["support_budget"],
        request_tags=["support", "urgent"]
    )
)

Important: When using Direct Provider Calls with streaming responses, be sure to read the stream completely. Pay-i needs the complete token information to accurately track usage and calculate costs.

Available Annotation Parameters

The create_headers() function accepts the following parameters:

Parameter	Type	Description
`limit_ids`	List[str]	List of limit IDs to track against (always provide as a list)
`request_tags`	List[str]	List of tags to associate with the request
`use_case_name`	str	Name of the use case for this request
`use_case_id`	str	ID of an existing use case
`user_id`	str	User ID to associate with the request

For Sequences of Related Calls

While custom headers provide direct control for individual API calls, you might benefit from Pay-i's inheritable decorators when dealing with sequences of related GenAI calls that share the same business context:

Decorators - For function-level annotations with inheritance capabilities across multiple related API calls

Related Resources

Pay-i Instrumentation - Overview of all instrumentation approaches
Decorators - Function-level annotations with inheritance capabilities
OpenAI Custom Headers - OpenAI-specific implementation
Azure OpenAI Custom Headers - Azure OpenAI-specific implementation
Anthropic Custom Headers - Anthropic-specific implementation
AWS Bedrock Custom Headers - AWS Bedrock-specific implementation
LangChain Callback Handler - LangChain callback approach