Overview

While auto-instrumentation gives you immediate tracking of your GenAI usage with minimal code changes, custom instrumentation takes your insights to the next level by adding business context to your API calls.

Pay-i tracks dimensions at three levels:

1. Auto-collected dimensions: Automatically captured technical data like model names, token counts, and costs (see Auto-Instrumentation)
2. Optional standard dimensions: Built-in Pay-i parameters you can set via annotations (like user_id, use_case_name, limit_ids, etc.) - see the complete list
3. Custom dimensions: Fully custom name-value pairs for business-specific context (see Property Bags)

This guide explains how to enhance your GenAI application with custom Annotations that connect technical usage to business outcomes. It builds on Auto-Instrumentation and allows you to:

• Associate API calls with specific business use cases
• Track usage by individual users or customer segments
• Apply spending limits and controls
• Organize requests with meaningful tags for analysis
• Add custom business dimensions beyond standard tracking
• Measure business ROI and value metrics

For an understanding of the business value these capabilities enable, see Value of Pay-i Instrumentation in the Pay-i Concepts section.

Optional Standard Dimensions

Pay-i provides built-in support for common business dimensions that you can set through annotations. These optional parameters let you add structured business context:

Business Context Dimensions

• Use Cases: use_case_name, use_case_id, use_case_version, use_case_step - Group and categorize requests by business function
• User Attribution: user_id - Track usage by individual users or entities
• Budget Management: limit_ids - Apply spending controls and usage quotas

Setting Optional Standard Dimensions

These dimensions can be set through any annotation method:

• Global defaults: Set via payi_instrument() configuration
• Function level: Applied using the @track decorator
• Block level: Set with track_context() context manager
• Request level: Added via custom headers

For a complete list of all available standard dimensions, see the get_context() documentation.

Custom Dimensions

Beyond standard dimensions, Pay-i supports fully custom dimensions for business-specific context:

• Property Bags: use_case_properties and request_properties - Define your own name-value pairs for dimensions like document types, customer tiers, processing stages, etc.

Annotation Methods

Once you have auto-instrumentation configured, you can add business context through Annotations. Pay-i offers two main approaches for adding these Annotations, each suited to different scenarios:

The key difference is scope: custom headers apply to individual requests, while the @track decorator automatically applies to everything in a function's call tree. For detailed information about how the decorator propagates annotations through call trees, see the Track Decorator documentation.

Implementation Examples

Here's how to add custom annotations to your API calls:

Using Custom Headers (Per-Request)

import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument
from payi.lib.helpers import create_headers

# Initialize Pay-i instrumentation
payi_instrument()

# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Make a request with annotations - only applies to this specific request
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello, how are you?"}],
    extra_headers=create_headers(
        use_case_name="customer_service",
        user_id="jane_doe",
        limit_ids=["department_budget"],
        request_tags=["greeting"]
    )
)

For provider-specific implementation details, see:

• OpenAI Custom Headers
• Azure OpenAI Custom Headers
• Anthropic Custom Headers
• AWS Bedrock Custom Headers
• LangChain Integration

Using the Track Decorator (Call Tree)

import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument, track

# Initialize Pay-i instrumentation
payi_instrument()

# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Use decorator to add annotations - applies to all API calls made within this function
# and any other functions it calls (the entire call tree)
@track(
    use_case_name="customer_service",
    user_id="jane_doe",
    limit_ids=["department_budget"],
    request_tags=["greeting"]
)
def handle_greeting(user_message):
    # These annotations will apply to ALL API calls made within this function
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": user_message}]
    )
    return response.choices[0].message.content

Note: If you're using Pay-i Proxy Configuration for Block limits, make sure you've initialized with config={"proxy": True} as described in the proxy configuration documentation.

Implementation Approaches

Review the detailed documentation for each approach to decide which best fits your needs:

• Custom Headers - For direct, per-request annotation of API calls
• Track Decorator - For function-level annotations that propagate through call trees
  Both approaches can be combined in the same application if needed, giving you maximum flexibility.

Provider-Specific Implementation

The approach to adding annotations varies depending on the provider SDK's capabilities:

Direct Header Support

These providers support annotations via the extra_headers parameter directly in their inference calls:

Alternative Approaches

These providers require different approaches since they don't natively support extra_headers:

See our provider-specific guides for complete examples and implementation details for each provider:

• OpenAI Implementation
• Azure OpenAI Implementation
• Anthropic Implementation
• AWS Bedrock Implementation
• LangChain Implementation

Related Resources

• Auto-Instrumentation - Basic setup and configuration
• Pay-i Proxy Configuration - For when you need Block limits
• Pay-i Concepts - Core concepts explanation
• Use Cases - Creating and managing use cases
• Budget Limits - Setting and monitoring budget limits