Day 2: Tracking Users with track_context()
📋 Overview
Welcome back to our 5-day learning path! In Day 1, you learned how to use payi_instrument()
for basic auto-instrumentation of your GenAI calls. Today, we'll build on that foundation by adding user-specific context to your tracked calls.
Today's Goal: Learn how to associate individual users with your GenAI calls using the track_context()
function.
Why Track Users?
Tracking which users are making GenAI calls provides several important benefits:
- Cost allocation: Understand which users or customers are driving your AI costs
- Usage patterns: Analyze how different users interact with your AI features
- Per-user limits: Apply budget constraints or rate limits for specific users
- Troubleshooting: Quickly identify issues related to specific users
- Compliance: Meet regulatory requirements for user-level tracking and accountability
Recap: Auto-Instrumentation
Yesterday, we set up basic instrumentation with a single function call:
from payi.lib.instrument import payi_instrument
# Initialize Pay-i auto-instrumentation
payi_instrument()
# All supported GenAI calls are now automatically tracked!
This approach is great for getting started quickly, but it doesn't tell you who is making each call. That's where custom instrumentation with the track_context()
function comes in!
🔑 Core Concept: The track_context()
Function
track_context()
FunctionWhile payi_instrument()
handles automatic instrumentation of all supported GenAI calls, the track_context()
function allows you to add custom context to specific code blocks in your application, particularly when dealing with dynamic information that changes with each request.
Important distinctions:
payi_instrument()
is called once at the start of your applicationtrack_context()
creates a context manager for specific code blocks where you want to add runtime information- Both work together:
payi_instrument()
captures the calls, whiletrack_context()
enriches them with context
Note: The
track_context()
function creates a scope that applies to all GenAI calls made within its block. This behavior gives you precise control over context propagation. Later in the learning path, we'll cover the@track
decorator for function-level static annotations.
The track_context()
function works alongside auto-instrumentation, adding valuable runtime context like user IDs to your tracked events.
Two Patterns for Tracking Users
Pay-i provides two primary approaches for tracking user IDs in your GenAI calls:
1. User ID from Runtime Context
When your user ID is available only at runtime (during function execution):
from payi.lib.instrument import track_context
def my_function(prompt):
# Get current user ID at runtime - this changes per request
user_id = get_current_user_id()
# Use track_context for runtime values
with track_context(user_id=user_id):
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
This approach is ideal when the user ID is determined at runtime and changes with each request.
2. User ID from Function Parameters
When your user ID is passed as a parameter to the function:
from payi.lib.instrument import track_context
def my_function(prompt, user_id):
# Use track_context to capture the user_id parameter
with track_context(user_id=user_id):
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
This approach is perfect for when the user ID is passed to your function as a parameter.
Note: You can also use custom headers with the
extra_headers
parameter, buttrack_context()
provides a cleaner, more readable solution for Python applications.
Practical Examples
Let's implement both approaches with complete examples:
Example 1: User ID from Flask Context
When using Flask, the proper way to access the user ID from the request context:
# Flask example
from flask import Flask, request, g
import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument, track_context
# Initialize Pay-i instrumentation
payi_instrument()
# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
app = Flask(__name__)
@app.route('/ai-chat')
def ai_chat():
# User ID is available in the request context
prompt = request.args.get('prompt')
return get_ai_response(prompt)
def get_ai_response(prompt):
# Use track_context to capture g.user.id at runtime (during the request)
with track_context(user_id=g.user.id):
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
Here, we access the user ID from Flask's g
object at runtime during each request. This ensures each request gets the correct user ID from the current request context.
Example 2: User ID from Function Parameters
When the user ID is passed as a parameter to the function call:
import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument, track_context
# Initialize Pay-i instrumentation
payi_instrument()
# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
# Function that takes user ID as a parameter
def get_ai_response_for_user(prompt, user_id):
# Use track_context to capture the user_id parameter
with track_context(user_id=user_id):
response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
# Call with different user IDs
result1 = get_ai_response_for_user("Tell me a joke", "user_123")
result2 = get_ai_response_for_user("What's the weather?", "user_456")
Here, we pass the user ID to each function call and use track_context()
to properly associate it with the GenAI request. This allows us to specify different user IDs for each call.
Best Practices for User IDs
Your user_id
should be a string that uniquely identifies the end-user responsible for each call:
- Always use
track_context()
for user IDs: Since user IDs typically change with each request - Capture user IDs at runtime: Get the ID when processing the request, not when defining functions
- Use at the closest scope: Place
track_context()
as close as possible to the GenAI call - Use consistent identifiers that match your application's user system
- Consider privacy implications when selecting identifiers
- Avoid using personally identifiable information (PII) when possible
- Use anonymous but consistent IDs for unauthenticated users
For more details on user-level attribution in Pay-i, see the User-level Attribution.
✅ Verification: Checking User Tracking
To verify that your user tracking is working correctly:
- Run your application and make a few AI calls through your decorated functions using different user IDs
- Log in to developer.pay-i.com
- Navigate to your application dashboard
- Click on Cost Drivers in the left sidebar
- Select the Users tab at the top of the page
- You'll see a list of all user IDs that have been tracked, along with their request counts and spend
You should see different user IDs listed with their corresponding request counts and costs, allowing you to analyze usage patterns on a per-user basis.
➡️ Next Steps
Congratulations! You've learned how to add user context to your GenAI calls using track_context()
. Tomorrow in Day 3, we'll build on this knowledge by introducing the @track
decorator for assigning specific Use Cases with @track(use_case_name="...")
.
The @track
decorator is well-suited for use case names since they are typically static values known when your functions are defined, unlike user IDs which change with each request.
💡 Additional Resources
- Custom Instrumentation guide - Comprehensive coverage of instrumentation options
- track_context() reference - Details on the
track_context()
function and its parameters - User-level Attribution - Advanced strategies for user attribution
Updated 3 days ago