Guides
Guides

Requests

Any attempted transaction, such as making an API call to a GenAI service, is considered a “request”.

Suggest Edits

Overview

In Pay-i, a Request refers to an API call to a Provider, such as getting a chat completion from OpenAI or generating an image with DALL-E. Requests are the fundamental unit of tracking in the Pay-i system and serve as your statement of record for all GenAI interactions.

The Core of Pay-i Tracking

Requests form the backbone of Pay-i's tracking and analytics system. Each request represents a discrete API call that Pay-i captures, enriches, and analyzes. When Pay-i instruments a request, it records much more than just cost:

  • Usage metrics: Tokens, images, or other resource consumption
  • Performance data: Response times, latency statistics
  • Cost information: Calculated based on the latest provider pricing
  • Success/failure status: Whether the API call completed successfully
  • Semantic information: Use cases, user IDs, custom tags
  • Limit tracking: How the request affects your defined budgets and quotas

This comprehensive data collection makes requests the central element in Pay-i's monitoring, analytics, and control capabilities.

How Pay-i Works with Requests

Pay-i provides two primary operational approaches for handling API requests:

Direct Provider Call with Telemetry (Default)

With this approach, your application communicates directly with the GenAI provider, while Pay-i captures metrics asynchronously:

Your App → GenAI Provider
       ↓
     Pay-i

This approach provides:

  • Minimal latency impact
  • No direct dependency on Pay-i for request success
  • Comprehensive usage tracking
  • Support for all Pay-i analytics features

The Pay-i SDK in this approach uses the Ingest API behind the scenes to report telemetry data after each provider call completes.

For detailed setup instructions, see the Auto-Instrumentation and Custom Instrumentation guides.

Proxy Routing (Advanced)

For advanced scenarios requiring real-time control or blocking limits, Pay-i offers Proxy Routing where API calls flow through Pay-i:

Your App → Pay-i → GenAI Provider
          ↑
     Request validation
     & limit enforcement

This approach provides:

  • Real-time cost calculation and limit enforcement
  • Ability to block requests that would exceed limits
  • Enhanced response objects with embedded Pay-i data
  • Support for advanced control features

For more details, see Operational Modes and Proxy Configuration.

Annotating Requests

Pay-i allows you to annotate API calls with additional metadata to enhance tracking and categorization. These annotations enable powerful features like:

The specific method for adding these annotations depends on whether you're using decorated functions or sending custom headers. For complete instructions, refer to the Custom Headers documentation.

Next Steps

To understand how to implement request tracking in your system:

Updated 10 days ago