Guides
Guides

Day 2: Tracking Users with @ingest(user_id)

Suggest Edits

📋 Overview

Welcome back to our 5-day learning path! Yesterday, you learned how to use payi_instrument() for basic auto-instrumentation of your GenAI calls. Today, we'll build on that foundation by adding user-specific context to your tracked calls.

Today's Goal: Learn how to associate individual users with your GenAI calls using the @ingest decorator.

Why Track Users?

Tracking which users are making GenAI calls provides several important benefits:

  • Cost allocation: Understand which users or customers are driving your AI costs
  • Usage patterns: Analyze how different users interact with your AI features
  • Per-user limits: Apply budget constraints or rate limits for specific users
  • Troubleshooting: Quickly identify issues related to specific users
  • Compliance: Meet regulatory requirements for user-level tracking and accountability

Recap: Auto-Instrumentation

Yesterday, we set up basic instrumentation with a single function call:

from payi.lib.instrument import payi_instrument

# Initialize Pay-i auto-instrumentation
payi_instrument()

# All supported GenAI calls are now automatically tracked!

This approach is great for getting started quickly, but it doesn't tell you who is making each call. That's where custom instrumentation with the @ingest decorator comes in!

🔑 Core Concept: The @ingest Decorator

While payi_instrument() handles automatic instrumentation of all supported GenAI calls, the @ingest decorator allows you to add custom context to specific functions in your codebase.

Important distinctions:

  • payi_instrument() is called once at the start of your application
  • @ingest is applied to individual functions that you want to instrument with additional context
  • Both work together: payi_instrument() captures the calls, while @ingest enriches them with context

Note: The @ingest decorator not only applies to the decorated function but also to any GenAI calls made within subfunctions called by the decorated function. This inheritance behavior gives you powerful control over context propagation. We'll cover this in more detail in a later day of the learning path.

The @ingest decorator works alongside auto-instrumentation, adding valuable context like user IDs to your tracked events.

Two Approaches for Tracking Users

Pay-i provides two primary approaches for tracking user IDs in your GenAI calls:

1. User ID from Context

When your user ID is available from application context (global state, request context, etc.):

from payi.lib.instrument import ingest

# The user ID is accessed from application context
# Could be from your auth system, a session, or Flask's g object
@ingest(user_id=get_current_user_id())  # Function is evaluated at decoration time
def my_function(prompt):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

This approach works when the user ID is available in the scope where the function is defined.

2. User ID from Function Parameters

When your user ID is passed as a parameter to the function, use extra_headers:

from payi.lib.helpers import create_headers

def my_function(prompt, user_id):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        # Pass user_id via extra_headers
        extra_headers=create_headers(user_id=user_id)
    )
    return response.choices[0].message.content

This approach works when the user ID is only available at function call time.

Note: If you specify user_id in both the @ingest decorator and in extra_headers, the extra_headers value takes precedence.

Practical Examples

Let's implement both approaches with complete examples:

Example 1: User ID from Context

When the user ID is available from application context:

# Flask example
from flask import Flask, request, g
import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument, ingest

# Initialize Pay-i instrumentation
payi_instrument()

# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

app = Flask(__name__)

@app.route('/ai-chat')
def ai_chat():
    # User ID is available in the request context
    prompt = request.args.get('prompt')
    return get_ai_response(prompt)

# Using the user ID directly from Flask's context
@ingest(user_id=g.user.id)  # Access application context directly
def get_ai_response(prompt):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Here, we access the user ID from Flask's g object, which is available at the time the function is defined. The user ID is attached to all calls made by this function.

Example 2: User ID from Function Parameters

When the user ID is passed as a parameter to the function call:

import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument
from payi.lib.helpers import create_headers

# Initialize Pay-i instrumentation
payi_instrument()

# Configure OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Function that takes user ID as a parameter
def get_ai_response_for_user(prompt, user_id):
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        # Pass user_id directly to the API call
        extra_headers=create_headers(user_id=user_id)
    )
    return response.choices[0].message.content

# Call with different user IDs
result1 = get_ai_response_for_user("Tell me a joke", "user_123")
result2 = get_ai_response_for_user("What's the weather?", "user_456")

Here, we pass the user ID to each API call via extra_headers, which allows us to specify different user IDs for each call.

Best Practices for User IDs

Your user_id should be a string that uniquely identifies the end-user responsible for each call:

  • Use consistent identifiers that match your application's user system
  • Consider privacy implications when selecting identifiers
  • Avoid using personally identifiable information (PII) when possible
  • Use anonymous but consistent IDs for unauthenticated users

For more details on user-level attribution in Pay-i, see the User-level Attribution.

✅ Verification: Checking User Tracking

To verify that your user tracking is working correctly:

  1. Run your application and make a few AI calls through your decorated functions using different user IDs
  2. Log in to developer.pay-i.com
  3. Navigate to your application dashboard
  4. Click on Cost Drivers in the left sidebar
  5. Select the Users tab at the top of the page
  6. You'll see a list of all user IDs that have been tracked, along with their request counts and spend

You should see different user IDs listed with their corresponding request counts and costs, allowing you to analyze usage patterns on a per-user basis.

➡️ Next Steps

Congratulations! You've learned how to add user context to your GenAI calls. Tomorrow in Day 3, we'll build on this knowledge by introducing another powerful capability of the @ingest decorator: assigning specific Use Cases using @ingest(use_case_name="...").

Use Cases allow you to categorize different AI functionalities in your application, providing even more granular insights into your GenAI usage.

💡 Additional Resources

Updated 5 days ago