Overview

This guide explains how to configure AWS Bedrock to work with Pay-i in both Proxy and Ingest modes. AWS Bedrock is a supported Provider with several Resources that Pay-i can track.

Note: AWS Bedrock requires a different approach when using Pay-i as a Proxy compared to other providers. Instead of simply changing the base URL, it uses event handlers to intercept and modify API calls before they reach AWS. This is because of how the AWS SDK authenticates and signs requests.

SDK Support

The examples in this guide use the Pay-i Python SDK, which provides comprehensive support for AWS Bedrock integration. If you're using a different programming language, you can utilize the Pay-i OpenAPI specification to generate a client SDK for your language of choice. The core concepts remain the same, though the exact implementation details may vary depending on the language and client library used.

Configuration Helper

Pay-i provides an AWS Bedrock URL helper in payi.lib.helpers to make AWS Bedrock setup easier:

Helper Function	Description
`payi_aws_bedrock_url()`	Generates the correct proxy URL for AWS Bedrock

Using Pay-i as a Proxy

AWS Bedrock integration requires a slightly different approach using event handlers to redirect API calls through Pay-i:

import boto3
import json
import os
import urllib3

from payi.lib.helpers import payi_aws_bedrock_url

# Read API key from environment variables
payi_api_key = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")

payi_aws_bedrock_url_path = payi_aws_bedrock_url()

def handle_payi_parameters(params, context, **kwargs):
    context["extra_headers"] = params.pop("extra_headers", {})

def redirect_to_payi(request, event_name, **kwargs):
    if not event_name.startswith('request-created.bedrock-runtime'):
        return
    
    parsed_url = urllib3.util.parse_url(request.url)
    route_path = parsed_url.path
    request.url = f"{payi_aws_bedrock_url_path}{route_path}"

    request.headers['xProxy-api-key'] = payi_api_key
    request.headers['xProxy-Provider-BaseUri'] = parsed_url.scheme + "://" + parsed_url.host
    extra_headers = request.context.get('extra_headers', {})
    for key, value in extra_headers.items():
        request.headers[key] = value


def register_bedrock_client_callbacks(client, model):
    # Process the extra_headers parameter passed to the bedrock runtime call
    client.meta.events.register(f'provide-client-params.bedrock-runtime.{model}',
                                handle_payi_parameters,
                                unique_id=handle_payi_parameters)

    # Redirect the request to the Pay-i endpoint after the request has been signed
    client.meta.events.register_last('request-created',
                                     redirect_to_payi,
                                     unique_id=redirect_to_payi)
    
# Create the bedrock client
region_name = "us-west-2"  # Use your preferred region
bedrock = boto3.client(
    'bedrock-runtime',
    region_name=region_name,
)

# Register client callbacks for all supported Bedrock operations
register_bedrock_client_callbacks(bedrock, 'InvokeModel')
register_bedrock_client_callbacks(bedrock, 'InvokeModelWithResponseStream')
register_bedrock_client_callbacks(bedrock, 'Converse')
register_bedrock_client_callbacks(bedrock, 'ConverseStream')

# Use the client with boto3 for AWS Bedrock
request_dict = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 512,
    "temperature": 0.5,
    "messages": [
        {
            "role": "user",
            "content": [{"type": "text", "text": "Hello, how are you?"}],
        }
    ],
}
request_body = json.dumps(request_dict)
model_id = 'anthropic.claude-3-haiku-20240307-v1:0'

response = bedrock.invoke_model(
    modelId=model_id,
    body=request_body,
)

Ingesting Metrics into Pay-i

In Ingest mode, configure your AWS Bedrock client normally:

import os
import json
import boto3
from payi.lib.instrument import ingest
from payi.lib.helpers import create_headers

# Configure with direct AWS Bedrock access
bedrock_runtime = boto3.client(
    service_name='bedrock-runtime',
    region_name='us-west-2'  # Use your preferred region
)

# Use the decorator to track usage
@ingest(request_tags=['bedrock'], use_case_name='bedrock_example')
def bedrock_example():
    # Call AWS Bedrock directly
    response = bedrock_runtime.invoke_model(
        modelId='anthropic.claude-v2',
        body=json.dumps({
            "prompt": "\n\nHuman: Hello, how are you?\n\nAssistant:",
            "max_tokens_to_sample": 300
        }),
        headers=create_headers(
            user_id="user123",
            limit_ids=['bedrock_budget']
        )
    )
    
    # Process the response
    response_body = json.loads(response['body'].read())
    return response_body['completion']

Advanced Configuration

Custom Base URLs

If you need to specify a custom base URL for private deployments or enterprise configurations:

from payi.lib.helpers import payi_aws_bedrock_url

# Use a custom base URL (e.g., for private deployments)
custom_url = payi_aws_bedrock_url(payi_base_url="https://custom.payi.domain.com")

AWS Session Configuration

For more complex AWS configurations:

import boto3
from payi.lib.helpers import payi_aws_bedrock_url

# Create a custom session with specific credentials
session = boto3.Session(
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY',
    region_name='us-west-2'
)

# Use the session to create a client with Pay-i proxy
bedrock_runtime = session.client(
    service_name='bedrock-runtime',
    endpoint_url=payi_aws_bedrock_url()
)

Request Timeouts

For operations that might take longer, configure appropriate timeouts:

import boto3
from botocore.config import Config
from payi.lib.helpers import payi_aws_bedrock_url

# Configure with longer timeouts for large requests
bedrock_config = Config(
    connect_timeout=10,  # 10 seconds
    read_timeout=60,     # 60 seconds
    retries={'max_attempts': 3}
)

# Use the config when creating the client
bedrock_runtime = boto3.client(
    service_name='bedrock-runtime',
    region_name='us-west-2',
    endpoint_url=payi_aws_bedrock_url(),
    config=bedrock_config
)

AWS Bedrock Provider Configuration