Overview

Pay-i Python SDK offers powerful inheritable decorators that make it easy to organize and track your GenAI consumption. These decorators can be nested and inherit parameters from each other, making it easy to create hierarchical tracking structures in your application.

Purpose of Inheritable Decorators

The primary purpose of these decorators is to annotate your functions with metadata such as:

• Use case names and IDs (formerly experience names and IDs)
• Limit IDs for budget tracking
• Request tags for organization
• User IDs for attribution

When used with Pay-i's instrumentation system (via payi_instrument()), these decorators help maintain consistent tracking across multiple API calls while reducing boilerplate code.

Operational Modes and Requirements

Pay-i supports two operational modes as described in the Pay-i Concepts documentation. The tables below show what's required for basic API tracking versus adding function annotations:

Pay-i as a Proxy

Ingesting Metrics into Pay-i

Key points:

• Pay-i as a Proxy can work with just GenAI provider client configuration for basic API tracking
• Ingesting Metrics into Pay-i requires payi_instrument() for basic tracking
• Adding function annotations (use cases, limits, users, tags) in either mode requires both payi_instrument() initialization and the appropriate decorator
• Decorators provide additional capabilities like parameter inheritance and custom dimension tracking in both modes

IMPORTANT: Choose either Pay-i as a Proxy or Ingesting Metrics into Pay-i for your application - do not mix them. Using @ingest when Pay-i is configured as a Proxy will cause double-counting of GenAI calls. Using @proxy when Pay-i is configured to ingest metrics can cause your GenAI provider to fail requests due to unexpected custom headers.

Setup & Installation

Prerequisites

• Pay-i Python SDK installed (pip install payi)
• A valid Pay-i API key
• One or more supported GenAI providers (OpenAI, Azure OpenAI, Anthropic, AWS Bedrock)

Initializing Pay-i Instrumentation

Before you can use either decorator, you must initialize payi_instrument():

import os
from payi import Payi
from payi.lib.instrument import payi_instrument

# Read API key from environment variables (best practice)
payi_key = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")

# Create Pay-i client
payi = Payi(api_key=payi_key)

# Initialize instrumentation - required before using decorators
payi_instrument(payi)

Once you've initialized the instrumentation, import the appropriate decorator for your chosen mode:

For using Pay-i as a proxy:

from payi.lib.instrument import proxy

For Ingesting Metrics into Pay-i:

from payi.lib.instrument import ingest

GenAI Provider Client Configuration

For Ingesting Metrics into Pay-i

In Ingest mode, configure your GenAI provider client normally (direct to provider):

import os
from openai import OpenAI

# Configure a standard provider client with direct access
openai_key = os.getenv("OPENAI_API_KEY", "YOUR_OPENAI_API_KEY")
client = OpenAI(api_key=openai_key)

For Pay-i as a Proxy

When using Pay-i as a Proxy, configure your GenAI provider client (OpenAI, Azure OpenAI, etc.) to use Pay-i as a proxy:

import os
from openai import OpenAI  # Can also be AzureOpenAI or other providers
from payi.lib.helpers import payi_openai_url

# Read API keys from environment variables
payi_key = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")
openai_key = os.getenv("OPENAI_API_KEY", "YOUR_OPENAI_API_KEY")

# Configure provider client to use Pay-i as a proxy
client = OpenAI(
    api_key=openai_key,
    base_url=payi_openai_url(),  # Use Pay-i's URL as the base
    default_headers={"xProxy-api-key": payi_key}  # Authenticate with Pay-i
)

Note:

• This is a basic example for OpenAI. For detailed configuration examples with other providers (Azure OpenAI, Anthropic, AWS Bedrock), refer to the Pay-i GenAI Provider Configuration guide.
• With proxy setup alone, Pay-i will track all API calls but won't have function annotations. To add annotations, you must also initialize payi_instrument() and use the @proxy decorator in your code.

Using the Decorators

After initializing payi_instrument() and configuring your provider client, you can use the decorators to annotate your functions.

Using @ingest Decorator (Ingest Mode)

Use this decorator when your provider client is configured for direct access (not through Pay-i):

Important: When using @ingest with streaming responses, be sure to read the stream completely. Pay-i needs the complete token information to accurately track usage and calculate costs. If you don't read the entire stream, you'll have incomplete data for ingestion.

from payi.lib.instrument import ingest
from payi.lib.helpers import create_headers

@ingest(request_tags=['summarization'], experience_name='document_summary')
def summarize_document(client, document_text, limit_ids, user_id):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": f"Summarize this: {document_text}"}],
        # Pass metadata to Pay-i using extra_headers
        # Note: limit_ids is always passed as a list, even with a single item
        extra_headers=create_headers(user_id=user_id, limit_ids=limit_ids)  # limit_ids=['budget_1', 'team_limit']
    )
    
    return response.choices[0].message.content

When using the @ingest decorator:

1. The API call is made directly to the provider (e.g., OpenAI)
2. Pay-i instruments the call to capture usage data
3. After the call completes, data is automatically sent to Pay-i with the annotations from the decorator

Using @proxy Decorator (Pay-i as a Proxy)

Use this decorator when your provider client is configured to use Pay-i as a proxy:

from payi.lib.instrument import proxy
from payi.lib.helpers import create_headers

@proxy(request_tags=['chat'], experience_name='customer_support')
def answer_customer_question(client, question, user_id):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": question}],
        # You can track against multiple limits simultaneously
        extra_headers=create_headers(
            user_id=user_id,
            limit_ids=['personal_limit', 'department_budget', 'project_x']
        )
    )
    
    return response.choices[0].message.content

When using the @proxy decorator:

1. The decorator's metadata (e.g., experience_name, request_tags) is added to the context
2. The API call is routed through Pay-i's proxy service
3. Pay-i captures usage data with the annotations and returns the response with additional metadata

Parameter Behavior

Both decorators accept the same parameters, which control how usage data is reported to Pay-i:

Available Parameters

Note: The experience_name and experience_id parameters are deprecated in favor of use_case_name and use_case_id parameters, which provide the same functionality with updated terminology.

Parameter Inheritance Rules

When decorators are nested, parameters are combined or inherited according to these rules:

Combining Parameters

• limit_ids: Values from all nested decorators are combined
• request_tags: Values from all nested decorators are combined

Inheriting Parameters

• use_case_name/experience_name:

  • If not specified in inner decorator, inherits from outer decorator
  • If specified in inner decorator, overrides outer decorator's value

• use_case_id/experience_id:

  • If not specified, but name is the same as outer decorator, inherits ID from outer
  • If not specified, but name is different from outer, generates a new UUID
  • If specified, uses the provided value

• user_id:

  • Inner decorator's value takes precedence over outer decorator

Here's how parameters are inherited in the execution context when decorators are nested:

# Using ingest mode example (same pattern works for proxy mode)
@ingest(limit_ids=['limit1'], request_tags=['outer'], use_case_name='outer_usecase')
def outer_function():
    # Context: limit_ids=['limit1'], request_tags=['outer'], use_case_name='outer_usecase'
    
    @ingest(limit_ids=['limit2'], request_tags=['inner'])
    def inner_function():
        # Context: limit_ids=['limit1', 'limit2'], request_tags=['outer', 'inner'], use_case_name='outer_usecase'
        pass
        
    inner_function()

IMPORTANT: Be consistent and use the same decorator throughout your application. Do not mix @ingest and @proxy decorators in the same application.

Advanced Examples

Nested Decorators with Inheritance

This example demonstrates how parameters are inherited when decorators are nested:

from payi.lib.instrument import ingest  # Use proxy instead for proxy mode

@ingest(request_tags=['app'], use_case_name='document_processing')
def process_document(document):
    # First process the document
    parsed_content = parse_document(document)
    
    # Then summarize it
    summary = summarize_content(parsed_content)
    
    return summary

@ingest(request_tags=['parsing'])
def parse_document(document):
    # This function inherits use_case_name='document_processing' from the parent
    # The combined request_tags are ['app', 'parsing']
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": f"Parse this document and extract key information: {document}"}]
    )
    return response.choices[0].message.content

@ingest(request_tags=['summarization'], use_case_name='document_summary')
def summarize_content(content):
    # This function uses use_case_name='document_summary' (overriding the parent)
    # The combined request_tags are ['app', 'summarization']
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": f"Summarize this content: {content}"}]
    )
    return response.choices[0].message.content

User ID Precedence Example

This example demonstrates how the innermost user_id takes precedence:

from payi.lib.instrument import ingest
from payi.lib.helpers import create_headers

# This example demonstrates how user_id precedence works
@ingest(user_id='default_user', request_tags=['app'])
def process_user_request(actual_user_id, query):
    # This function's user_id is 'default_user'
    
    response = query_llm(actual_user_id, query)
    return response

@ingest(request_tags=['query'])
def query_llm(user_id, query):
    # This function's decorator doesn't specify user_id
    # So it inherits 'default_user' from the parent
    
    # But we override with the actual user ID in the API call
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": query}],
        extra_headers=create_headers(
            user_id=user_id,  # This takes precedence over the decorator's user_id
            limit_ids=['limit_a', 'limit_b']  # Note: limit_ids always requires an array
        )
    )
    return response.choices[0].message.content

Related Resources

• Pay-i Concepts
• Ingesting Events
• Pay-i Examples & Quickstarts

Advanced Configuration

The payi_instrument() function supports additional parameters for advanced use cases:

Important Note: While the Pay-i SDK collects prompt and completion data by default (log_prompt_and_response=True), this data is not actually stored on the Pay-i service side until you explicitly opt in. Each application has its own "Logging Enabled" checkbox in its Settings page that is off by default.

This design enables you to toggle prompt/completion logging on the service side without needing to redeploy your application. When you're ready to analyze prompt data, simply enable the checkbox in the Settings page of the respective application, and the data will start flowing immediately. You can also completely disable collection at the SDK level by setting log_prompt_and_response=False if you have strict compliance requirements.

Additional Options:

1. Standard Deployment: Pay-i is SOC2 compliant, suitable for most enterprise use cases.

2. Private Deployment: For enterprises with strict data residency requirements, Pay-i offers private deployments within your own Azure VNET or AWS VPC with full network isolation. This ensures sensitive data never leaves your network boundary.

3. Custom Logging: For organizations with HIPAA, PCI-DSS, or other regulatory requirements:

   • Use a custom logger to store both the Pay-i request ID and the actual prompts/completions in your existing regulation-approved observability system
   • When analyzing metrics in Pay-i (cost, performance, failures, KPIs), manually correlate with the stored data using the request ID

Example with advanced configuration:

import os
from payi import Payi
from payi.lib.instrument import payi_instrument
from payi.lib.helpers import PayiCategories

# Read the API key from environment variables with a fallback value
payi_key = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")

# Custom logger function that integrates with OpenTelemetry
def custom_logger(request_id, log_data):
    # Correlate Pay-i request_id with your existing observability system
    import opentelemetry.trace as trace
    
    # Get current span from OpenTelemetry context
    current_span = trace.get_current_span()
    
    # Add Pay-i request_id as an attribute to your span
    current_span.set_attribute("payi.request_id", request_id)
    
    # Log prompt/completion data to your existing logging infrastructure
    for key, value in log_data.items():
        current_span.set_attribute(f"payi.{key}", value)

# Initialize with advanced options for high-security environment
payi_instrument(
    payi=Payi(api_key=payi_key),
    instruments={PayiCategories.openai, PayiCategories.anthropic},  # Only instrument these providers (default: all)
    log_prompt_and_response=False,  # Disable prompt collection for handling sensitive data (HIPAA, PII, etc.)
    prompt_and_response_logger=custom_logger  # Custom logger if you need to collect data elsewhere
)

Using Synchronous and Asynchronous Clients

Pay-i provides both Payi (synchronous) and AsyncPayi (asynchronous) clients to match your application's programming model. The instrumentor works with both client types automatically:

• Use Payi with regular synchronous functions
• Use AsyncPayi with async/await coroutines

For applications that use both programming models, you can initialize instrumentation with both clients simultaneously:

import os
from payi import Payi, AsyncPayi
from payi.lib.instrument import payi_instrument

# Read the API key from environment variables with a fallback value
payi_key = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")

# Create both client types
payi = Payi(api_key=payi_key)     # For synchronous code
apayi = AsyncPayi(api_key=payi_key)  # For asynchronous code

# Initialize instrumentation with both clients
payi_instrument([payi, apayi])

With this configuration, the appropriate client is automatically used based on whether the decorated function is synchronous or asynchronous. This allows you to use the same decorators across your entire codebase regardless of the function type.