Overview

Pay-i supports various GenAI Providers and offers specialized configuration helpers to simplify integration. These providers offer not just language models (LLMs) but also image generation, audio processing, video analysis, vision capabilities, and Retrieval-Augmented Generation (RAG) technologies. This guide provides an overview of configuring supported providers for both Proxy and Ingest modes, with links to detailed provider-specific documentation.

SDK Support

Pay-i provides a Python SDK for seamless integration with various GenAI providers. This includes not just Large Language Models (LLMs), but also:

Image Generation & Vision: Process and create images with vision-capable models
Audio Processing: Speech-to-text, text-to-speech, and audio analysis
Video Analysis: Process and analyze video content
Multimodal Models: Work with models that can handle multiple input and output types
RAG Systems: Implement Retrieval-Augmented Generation for knowledge-intensive applications

For other programming languages, you can use the OpenAPI specification to generate client SDKs. The examples in this documentation use the Python SDK, which offers the most comprehensive support and helper functions.

Configuration Modes

Pay-i offers two operational modes as described in the Pay-i Concepts documentation:

Pay-i as a Proxy: API calls go through Pay-i, which forwards them to the provider
Ingesting Metrics into Pay-i: API calls go directly to the provider with metrics sent to Pay-i

Your provider configuration will vary based on which mode you choose.

Important Note for Streaming in Ingest Mode: When working with streaming responses in Ingest mode, you must read the stream to the end before ingesting the response. Pay-i needs the complete token information to accurately track usage and calculate costs.

Provider-Specific Configuration

Pay-i supports various GenAI Providers that offer capabilities beyond just language models. The table below provides an overview of the supported providers and their configuration helpers:

Provider	Helper Function	Detailed Documentation
OpenAI	`payi_openai_url()`	OpenAI Configuration
Azure OpenAI	`payi_azure_openai_url()`	Azure OpenAI Configuration
Anthropic	`payi_anthropic_url()`	Anthropic Configuration
AWS Bedrock	`payi_aws_bedrock_url()`	AWS Bedrock Configuration
Google Vertex	`payi_google_vertex_url()`	Google Vertex Configuration
LangChain	`PayiHandler` (custom callback)	LangChain Configuration

These providers support a range of GenAI capabilities including:

Text Generation & Chat: Generate coherent text and hold interactive conversations
Embeddings: Create vector representations of text for semantic similarity and search
Knowledge Retrieval: Retrieve and use information from external sources (including RAG implementations)
Image Generation: Create images from text descriptions
Vision & Image Analysis: Process and analyze image content
Speech Processing: Convert between text and speech
Multimodal Processing: Work with multiple types of data (text, images, audio) simultaneously

For a complete and up-to-date list of all supported models and Resources for each Provider, refer to the Pay-i Managed Categories and Resources documentation.

Additional Configuration Options

Custom Base URLs

If you need to specify a custom base URL (for private deployments or enterprise configurations), you can pass it to the helper functions:

from payi.lib.helpers import payi_openai_url

# Use a custom base URL (e.g., for private deployments)
custom_url = payi_openai_url(payi_base_url="https://custom.payi.domain.com")

Connection Pooling

For high-throughput applications, you may want to configure connection pooling:

from openai import OpenAI
import httpx
from payi.lib.helpers import payi_openai_url

# Configure with httpx transport for connection pooling
transport = httpx.HTTPTransport(limits=httpx.Limits(max_connections=100))
client = OpenAI(
    api_key="your-key",
    base_url=payi_openai_url(),
    default_headers={"xProxy-api-key": "your-payi-key"},
    http_client=httpx.Client(transport=transport)
)

Timeouts

For operations that might take longer, consider configuring appropriate timeouts:

from openai import OpenAI
import httpx
from payi.lib.helpers import payi_openai_url

# Configure with longer timeouts for large requests
timeout = httpx.Timeout(timeout=60.0)  # 60 seconds
client = OpenAI(
    api_key="your-key",
    base_url=payi_openai_url(),
    default_headers={"xProxy-api-key": "your-payi-key"},
    http_client=httpx.Client(timeout=timeout)
)

GenAI Provider Configuration