Guides
Guides

OpenAI Provider Configuration

Suggest Edits

Overview

This guide explains how to configure OpenAI to work with Pay-i Instrumentation. OpenAI is a supported Provider with several Resources that Pay-i can track.

Prerequisites

To use Pay-i with OpenAI, you'll need:

  • OpenAI Service:

  • Pay-i:

  • Client Library:

    • OpenAI Python SDK (pip install openai)
    • Optional: httpx for advanced features (pip install httpx)

SDK Support

The examples in this guide use the Pay-i Python SDK, which provides comprehensive support for OpenAI integration. If you're using a different programming language, you can utilize the Pay-i OpenAPI specification to generate a client SDK for your language of choice. The core concepts remain the same, though the exact implementation details may vary depending on the language and client library used.

Direct Provider Call with Telemetry (Default)

The standard approach for using Pay-i with OpenAI is to initialize Pay-i instrumentation while making API calls directly to OpenAI:

import os
from openai import OpenAI
from payi.lib.instrument import payi_instrument

# Initialize Pay-i instrumentation
payi_instrument()

# Configure standard OpenAI client
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Use the client normally
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

With this configuration, Pay-i automatically tracks API calls and calculates costs without adding any latency to your requests.

Optional Proxy Routing (For Block Limits)

If you need to implement Block limits that prevent requests from being sent to the provider when a budget is exceeded, you'll need to use Pay-i's proxy configuration:

import os
from openai import OpenAI
from payi.lib.helpers import payi_openai_url
from payi.lib.instrument import payi_instrument

# Initialize Pay-i instrumentation with proxy mode
payi_instrument(config={"proxy": True})

# Configure OpenAI client to use Pay-i as a proxy
client = OpenAI(
    api_key=os.getenv("OPENAI_API_KEY"),
    base_url=payi_openai_url(),  # Use Pay-i's URL as the base
    default_headers={"xProxy-api-key": os.getenv("PAYI_API_KEY")}  # Authenticate with Pay-i
)

For detailed information about proxy configuration, including when to use it and how it works, see the Pay-i Proxy Configuration guide.

Advanced Configuration

Note: The following advanced configurations can be applied to both standard and proxy configurations. If you're using proxy configuration, remember to include the base_url and default_headers parameters as shown in the proxy configuration example above.

Connection Pooling

For high-throughput applications, you may want to configure connection pooling:

from openai import OpenAI
import httpx
from payi.lib.instrument import payi_instrument

# Initialize Pay-i instrumentation
payi_instrument()

# Configure with httpx transport for connection pooling
transport = httpx.HTTPTransport(limits=httpx.Limits(max_connections=100))
client = OpenAI(
    api_key="your-key",
    http_client=httpx.Client(transport=transport)
)

Timeouts

For operations that might take longer, consider configuring appropriate timeouts:

from openai import OpenAI
import httpx
from payi.lib.instrument import payi_instrument

# Initialize Pay-i instrumentation
payi_instrument()

# Configure with longer timeouts for large requests
timeout = httpx.Timeout(timeout=60.0)  # 60 seconds
client = OpenAI(
    api_key="your-key",
    http_client=httpx.Client(timeout=timeout)
)

Related Resources

Quickstart Guides

Conceptual Guides

Example Repositories

Updated 10 days ago