Guides
Guides

Day 2: Tracking Users with track_context()

Suggest Edits

📋 Overview

Welcome back to our 5-day learning path! In Day 1, you learned how to use payi_instrument() for basic auto-instrumentation of your GenAI calls. Today, we'll build on that foundation by adding user-specific context to your tracked calls.

Today's Goal: Learn how to associate individual users with your GenAI calls using the track_context() function.

Why Track Users?

Tracking which users are making GenAI calls provides several important benefits:

  • Cost allocation: Understand which users or customers are driving your AI costs
  • Usage patterns: Analyze how different users interact with your AI features
  • Per-user limits: Apply budget constraints or rate limits for specific users
  • Troubleshooting: Quickly identify issues related to specific users
  • Compliance: Meet regulatory requirements for user-level tracking and accountability

Recap: Auto-Instrumentation

Yesterday, we set up basic instrumentation with a single function call:

from payi.lib.instrument import payi_instrument

# Initialize Pay-i auto-instrumentation
payi_instrument()

# All supported GenAI calls are now automatically tracked!

This approach is great for getting started quickly, but it doesn't tell you who is making each call. That's where custom instrumentation with the track_context() function comes in!

🔑 Core Concept: The track_context() Function

While payi_instrument() handles automatic instrumentation of all supported GenAI calls, the track_context() function allows you to add custom context to specific code blocks in your application, particularly when dealing with dynamic information that changes with each request.

Important distinctions:

  • payi_instrument() is called once at the start of your application
  • track_context() creates a context manager for specific code blocks where you want to add runtime information
  • Both work together: payi_instrument() captures the calls, while track_context() enriches them with context

Note: The track_context() function creates a scope that applies to all GenAI calls made within its block. This behavior gives you precise control over context propagation. Later in the learning path, we'll cover the @track decorator for function-level static annotations.

The track_context() function works alongside auto-instrumentation, adding valuable runtime context like user IDs to your tracked events.

Two Patterns for Tracking Users

Pay-i provides two primary approaches for tracking user IDs in your GenAI calls:

1. User ID from Runtime Context

When your user ID is available only at runtime (during function execution):

from payi.lib.instrument import track_context

def my_function(prompt):
    # Get current user ID at runtime - this changes per request
    user_id = get_current_user_id()
    
    # Use track_context for runtime values
    with track_context(user_id=user_id):
        response = client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}]
        )
    return response.choices[0].message.content

This approach is ideal when the user ID is determined at runtime and changes with each request.

2. User ID from Function Parameters

When your user ID is passed as a parameter to the function:

from payi.lib.instrument import track_context

def my_function(prompt, user_id):
    # Use track_context to capture the user_id parameter
    with track_context(user_id=user_id):
        response = client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}]
        )
    return response.choices[0].message.content

This approach is perfect for when the user ID is passed to your function as a parameter.

Note: You can also use custom headers with the extra_headers parameter, but track_context() provides a cleaner, more readable solution for Python applications.

Practical Examples

Let's implement both approaches with complete examples:

Example 1: User ID from Flask Context

When using Flask, the proper way to access the user ID from the request context:

# Flask example
from flask import Flask, request, g
import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument, track_context

# Initialize Pay-i instrumentation
payi_instrument()

# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

app = Flask(__name__)

@app.route('/ai-chat')
def ai_chat():
    # User ID is available in the request context
    prompt = request.args.get('prompt')
    return get_ai_response(prompt)

def get_ai_response(prompt):
    # Use track_context to capture g.user.id at runtime (during the request)
    with track_context(user_id=g.user.id):
        response = client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}]
        )
    return response.choices[0].message.content

Here, we access the user ID from Flask's g object at runtime during each request. This ensures each request gets the correct user ID from the current request context.

Example 2: User ID from Function Parameters

When the user ID is passed as a parameter to the function call:

import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument, track_context

# Initialize Pay-i instrumentation
payi_instrument()

# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Function that takes user ID as a parameter
def get_ai_response_for_user(prompt, user_id):
    # Use track_context to capture the user_id parameter
    with track_context(user_id=user_id):
        response = client.chat.completions.create(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": prompt}]
        )
    return response.choices[0].message.content

# Call with different user IDs
result1 = get_ai_response_for_user("Tell me a joke", "user_123")
result2 = get_ai_response_for_user("What's the weather?", "user_456")

Here, we pass the user ID to each function call and use track_context() to properly associate it with the GenAI request. This allows us to specify different user IDs for each call.

Best Practices for User IDs

Your user_id should be a string that uniquely identifies the end-user responsible for each call:

  • Always use track_context() for user IDs: Since user IDs typically change with each request
  • Capture user IDs at runtime: Get the ID when processing the request, not when defining functions
  • Use at the closest scope: Place track_context() as close as possible to the GenAI call
  • Use consistent identifiers that match your application's user system
  • Consider privacy implications when selecting identifiers
  • Avoid using personally identifiable information (PII) when possible
  • Use anonymous but consistent IDs for unauthenticated users

For more details on user-level attribution in Pay-i, see the User-level Attribution.

✅ Verification: Checking User Tracking

To verify that your user tracking is working correctly:

  1. Run your application and make a few AI calls through your decorated functions using different user IDs
  2. Log in to developer.pay-i.com
  3. Navigate to your application dashboard
  4. Click on Cost Drivers in the left sidebar
  5. Select the Users tab at the top of the page
  6. You'll see a list of all user IDs that have been tracked, along with their request counts and spend

You should see different user IDs listed with their corresponding request counts and costs, allowing you to analyze usage patterns on a per-user basis.

➡️ Next Steps

Congratulations! You've learned how to add user context to your GenAI calls using track_context(). Tomorrow in Day 3, we'll build on this knowledge by introducing the @track decorator for assigning specific Use Cases with @track(use_case_name="...").

The @track decorator is well-suited for use case names since they are typically static values known when your functions are defined, unlike user IDs which change with each request.

💡 Additional Resources

Updated 5 days ago