Overview

The payi_instrument() function is the foundation of SDK initialization in Pay-i applications. This document explains its purpose, usage patterns, and configuration options.

Purpose

The payi_instrument() function initializes the Pay-i SDK in your Python application, enabling:

• Automatic instrumentation of supported GenAI providers (OpenAI, Azure OpenAI, Anthropic, AWS Bedrock)
• Global context for parameter inheritance across your application
• Prompt and response logging for debugging and analysis
• Integration with @track decorator and track_context function

Without calling this function, most SDK features (including automatic provider instrumentation and decorators) won't work properly. If you're using proxy mode, you can alternatively instrument your GenAI calls manually with custom headers. For use cases where SDK auto-instrumentation isn't suitable, you can call the Ingest API directly.

Note: payi_instrument() is required for automatic instrumentation. After initializing with payi_instrument(), you can add custom context to your GenAI calls using the @track decorator, track_context() function, or custom headers to enrich your telemetry data.

When to Call

You should call payi_instrument() once during your application startup, typically:

• In your application's main module

• Before any instrumented GenAI API calls

• Before using the @track decorator or track_context function

• Before creating provider SDK client instances (e.g., OpenAI(), Bedrock()). This is crucial because payi_instrument() needs to hook into the provider libraries during their initialization. While it might sometimes work if called after client creation (e.g., for OpenAI where objects are created per request), it can fail for others (e.g., Bedrock). Best Practice: Always call payi_instrument() first.

# Example: Initializing in your main.py or app.py
from payi.lib.instrument import payi_instrument

def initialize_app():
    # Initialize Pay-i as early as possible in your application startup
    payi_instrument()
    # Continue with application initialization...

if __name__ == "__main__":
    initialize_app()
    # Rest of your application...

Key Functionalities

The payi_instrument() function enables these core features:

1. Provider Instrumentation

Automatically hooks into supported GenAI provider libraries to capture API calls without requiring code changes to your existing integration.

Currently supported providers:

• OpenAI and Azure OpenAI
• Anthropic
• AWS Bedrock

For detailed information on configuring each of these providers, see our dedicated Provider Configuration documentation.

2. Global Context

Establishes a base context for parameter inheritance throughout your application, which:

• Acts as the base layer from which decorators and context managers inherit parameters.
• Provides default values for all instrumented calls
• Persists for the lifetime of the application

3. Annotation Integration

Enables the use of the @track decorator and track_context function in your application:

• Direct Provider Call with Telemetry mode (calls go directly to providers, telemetry sent to Pay-i)
• Proxy Routing mode (calls routed through Pay-i with additional features)

The operational mode is controlled by the proxy parameter:

• proxy=False for Direct Provider Call with Telemetry (default)
• proxy=True for Proxy Routing

See the track decorator documentation and track_context documentation for more details.

4. Default Behaviors

Sets up reasonable defaults that work for most applications:

• Direct Provider Call mode by default (proxy=False)
• Client Behavior: payi_instrument will communicate with Pay-i using default settings, requiring the PAYI_API_KEY environment variable to be set and optionally PAYI_BASE_URL. If you need to alter the Pay-i client used by payi_instrument, you can provide your own via the payi parameter.
• Use Case Defaults:
  • use_case_name: Automatically uses the filename of the caller of payi_instrument() (e.g., "main" for main.py)
  • use_case_id: Automatically generates a UUID on the caller's behalf
• Prompt/response content logging enabled by default (though storage of this content must also be enabled in the ingesting app settings)
• Automatic callstack capture for debugging

Configuration Options

The payi_instrument() function accepts several parameters to customize its behavior:

def payi_instrument(
    payi: Optional[Union[Payi, AsyncPayi, 'list[Union[Payi, AsyncPayi]]']] = None,
    instruments: Optional[Set[str]] = None,
    log_prompt_and_response: bool = True,
    prompt_and_response_logger: Optional[Callable[[str, "dict[str, str]"], None]] = None,
    config: Optional[PayiInstrumentConfig] = None,
    logger: Optional[logging.Logger] = None,
) -> None:

Common Parameters

Advanced Note on Client Configuration:
payi_instrument requires the PAYI_API_KEY environment variable to be set. The PAYI_BASE_URL environment variable is optional and will use the default Pay-i URL if not specified.

If you do not provide a client instance via the payi parameter, payi_instrument will use these environment variables to communicate with the Pay-i service.

You only need to use the payi parameter if you want to customize the settings (e.g., timeouts, headers) of the client used by payi_instrument.

If you intend to call the Pay-i client SDK APIs directly in your application code (separate from the automatic instrumentation or decorators), you should typically create a separate Payi or AsyncPayi instance for those calls.

The config Parameter

The config parameter accepts a dictionary that defines global settings applied to all instrumented calls made through the SDK. These settings serve as the base layer for all requests, which can be overridden by more specific contexts.

Note: The experience_name and experience_id parameters are deprecated and will be removed in a future version. Use use_case_name and use_case_id instead.

For more information on user-level attribution, see the user-level attribution documentation.

Usage Examples

Basic Initialization

For most applications, the simplest initialization with default settings is sufficient, as long as you have set the required environment variables:

from payi.lib.instrument import payi_instrument

# Environment variables must be set:
# - PAYI_API_KEY (required)
# - PAYI_BASE_URL (optional, will use default if not specified)

# Simple initialization with default settings
payi_instrument()

This will:

• Initialize in Direct Provider Call mode
• Use the Pay-i API key from the environment variables
• Enable prompt and response logging
• Instrument all supported providers

Selecting Specific Providers

If you only want to instrument specific providers, use the instruments parameter with a set of provider names:

from payi.lib.instrument import payi_instrument
from payi.lib.helpers import PayiCategories

# Only instrument OpenAI and Anthropic providers
payi_instrument(instruments={PayiCategories.openai, PayiCategories.anthropic})

This is useful when your application only uses certain providers, or when you want to reduce the overhead of instrumenting unused providers.

Setting Application Use Case

While automatic defaults are provided for development and testing, we recommend explicitly setting the use case name and ID for production applications:

Important: The use case must exist before the first provider instrumented call is made. Make sure to set the use case name and version for all requests in your application at initialization time.

from payi.lib.instrument import payi_instrument

# Best practice for production: explicitly set use case name and ID
payi_instrument(config={
    "use_case_name": "MyApplication",  # Descriptive name for your application
    "use_case_id": "app-123",          # Stable identifier for your application
    "use_case_version": 2              # If not specified, version 1 is used by default
})

This ensures consistent tracking and reporting across deployments and makes it easier to identify your application in Pay-i dashboards.

Setting Global Request Tags

You can define global request tags that will be applied to all requests:

from payi.lib.instrument import payi_instrument

# Set global request tags
payi_instrument(config={
    "request_tags": {
        "environment": "production",
        "region": "us-west",
        "application": "customer-service"
    }
})

These tags will be inherited by all requests in your application. Like other list parameters in Pay-i, request tags follow these inheritance rules:

• Adding Tags: When you specify request_tags in track_context or @track decorators, they combine with global tags
• Clearing Tags: To override all global tags, use an empty list/dictionary ([] or {})
• Inheritance: When request_tags is not specified, tags from the parent scope are inherited

For example, if you have global tags set in payi_instrument and then use track_context with additional tags:

# Both global tags and context-specific tags will be applied
with track_context(request_tags={"operation": "summarize"}):
    client.chat.completions.create(...)  # Has all combined tags

For complete details on request tags inheritance and usage, see the Request Tags documentation and Parameter Precedence in Pay-i Instrumentation.

With Proxy Mode

Enabling Proxy Routing mode:

from payi.lib.instrument import payi_instrument

# Enable proxy routing mode
payi_instrument(config={"proxy": True})

With Custom Client

When you need to customize client settings (like timeout), you can provide your own Payi instance. Note that the environment variables (PAYI_API_KEY and optionally PAYI_BASE_URL) are still required:

from payi import Payi
from payi.lib.instrument import payi_instrument

# Environment variables must be set:
# - PAYI_API_KEY (required)
# - PAYI_BASE_URL (optional, will use default if not specified)

# Create a client with custom timeout setting
payi = Payi(timeout=60)  # 60-second timeout
payi_instrument(payi=payi)

With User ID Attribution

Setting a default user ID for attribution:

from payi.lib.instrument import payi_instrument

# Set default user ID for all requests
payi_instrument(config={"user_id": "user-123"})

Configuring Prompt Logging

For prompts and responses to be stored and accessible in Pay-i, both of the following conditions must be met:

1. Client-side: The log_prompt_and_response parameter in payi_instrument() must be set to True (this is the default)
2. Server-side: Logging must be enabled for the application in the Pay-i portal under Application settings

If either of these conditions is not met, prompt and response content will not be stored in Pay-i.

Client-side configuration

For sensitive applications, you can disable prompt collection at the client side:

from payi.lib.instrument import payi_instrument

# Disable prompt and response collection
payi_instrument(log_prompt_and_response=False)

When disabled this way, no prompt or response content will be sent to Pay-i, regardless of server-side settings.

Server-side configuration

Even when log_prompt_and_response=True in your code, prompt and response content is only stored if also enabled in the Pay-i portal's Application settings. This dual-control design allows:

• Application administrators to enable/disable logging without requiring code changes
• Developers to implement safeguards for highly sensitive prompts at the code level
• Selective enabling of prompt logging for debugging and then disabling it in production

Custom Prompt and Response Logging

You can register a custom callback function to receive prompt and response data, which is useful for correlating Pay-i's data with your own observability systems:

from typing import Dict
from payi.lib.instrument import payi_instrument

def my_prompt_logger(request_id: str, log_data: Dict[str, str]) -> None:
    """Log request IDs and data to your own observability system.
    
    Args:
        request_id: Pay-i's unique request identifier
        log_data: Dictionary with prompt and response data
    """
    # Log the Pay-i request ID to your observability system
    # This allows correlation between Pay-i's data and your own logs
    my_logging_system.log(
        correlation_id=request_id,
        event_type="llm_request",
        timestamp=datetime.now(),
        prompt_length=len(log_data.get("provider_request_json", "")),
        response_length=len(log_data.get("provider_response_json", ""))
    )

# Initialize with the callback
payi_instrument(
    log_prompt_and_response=True,  # Must be True for the callback to be invoked
    prompt_and_response_logger=my_prompt_logger
)

The callback receives:

• request_id: Pay-i's unique identifier for the request
• log_data: A dictionary containing the complete prompt and response content

This feature enables you to:

• Correlate data between Pay-i's systems and your own observability tools
• Record Pay-i request IDs in your existing logging infrastructure
• Capture additional metrics about your LLM usage
• Create custom dashboards with data from both systems

Note: This callback is invoked in addition to normal Pay-i telemetry (not instead of it). The callback is only called if log_prompt_and_response=True.

Configuring Instrumentation Logging

You can control where Pay-i SDK instrumentation logs are sent by providing a custom logger:

import logging
from payi.lib.instrument import payi_instrument

# Create and configure a custom logger
app_logger = logging.getLogger("my_app.payi")
app_logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
app_logger.addHandler(handler)

# Use your custom logger for Pay-i instrumentation
payi_instrument(logger=app_logger)

If you don't provide a logger:

• The instrumentation will log to a logger named "payi.instrument" by default
• The "payi.instrument" logger is still used for early bootstrap before your custom logger is available
• If instrumentation wasn't properly initialized but decorators are still being called, logs will go to "payi.instrument"

This allows you to integrate Pay-i's diagnostic logs with your application's logging system for better visibility and management of log messages.

Best Practices

1. Initialize Early: Call payi_instrument() early in your application startup sequence.
2. Initialize Once: Only call the function once in your application lifecycle.
3. Set Environment Variables: Ensure PAYI_API_KEY is set in your environment before calling payi_instrument(). Optionally set PAYI_BASE_URL if using a non-default Pay-i endpoint.
4. Use Global Defaults: Set reasonable defaults for parameters that apply to your entire application.
5. Explicit Use Case in Production: While automatic use case name/ID defaults are convenient for development, explicitly set these values in production for consistency and clarity.
6. Consider Privacy: Disable prompt logging (log_prompt_and_response=False) if handling sensitive data that should never be stored.
7. Consider Proxy Mode: Use proxy=True when you need Block limits functionality, which is only available in proxy mode.

Related Topics

• Pay-i Client Classes - Details on the Payi and AsyncPayi client classes.
• Track Decorator - Function-level annotations enabled by payi_instrument().
• Track Context - Block-level annotations enabled by payi_instrument().
• Custom Headers - Learn how to use custom headers with payi_instrument() for fine-grained per-request annotations.
• User-Level Attribution - Track user activity across multiple requests.
• Request Tags - Understanding how to use request tags for categorizing and filtering requests.
• Parameter Precedence in Pay-i Instrumentation - Understand how parameters from payi_instrument, decorators, context managers, and headers interact.