Guides
Guides

AWS Bedrock Provider Configuration

Suggest Edits

Overview

This guide explains how to configure AWS Bedrock to work with Pay-i Instrumentation. AWS Bedrock is a supported Provider with several Resources that Pay-i can track.

Note: AWS Bedrock requires a different approach when using Pay-i's proxy configuration compared to other providers. Instead of simply changing the base URL, it uses event handlers to intercept and modify API calls before they reach AWS. This is because of how the AWS SDK authenticates and signs requests.

Prerequisites

To use Pay-i with AWS Bedrock, you'll need:

  • AWS Bedrock Service:

  • Pay-i:

  • Client Libraries:

    • Boto3 (pip install boto3)
    • urllib3 (pip install urllib3)

SDK Support

The examples in this guide use the Pay-i Python SDK, which provides comprehensive support for AWS Bedrock integration. If you're using a different programming language, you can utilize the Pay-i OpenAPI specification to generate a client SDK for your language of choice. The core concepts remain the same, though the exact implementation details may vary depending on the language and client library used.

Basic Setup

Let's start with the common configuration variables needed for both approaches:

import os
import json
import boto3
from payi.lib.instrument import payi_instrument
from payi.lib.helpers import create_headers

# API keys and configuration
PAYI_API_KEY = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")
AWS_REGION = "us-west-2"  # Use your preferred region

Direct Provider Call with Telemetry (Default)

The default approach makes API calls directly to AWS Bedrock while Pay-i tracks usage:

# Initialize Pay-i instrumentation
payi_instrument()

# Configure direct AWS Bedrock client
bedrock_runtime = boto3.client(
    service_name='bedrock-runtime',
    region_name=AWS_REGION
)

# Make a request with annotations
model_id = 'anthropic.claude-3-haiku-20240307-v1:0'
request_body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 512,
    "temperature": 0.5,
    "messages": [
        {
            "role": "user",
            "content": [{"type": "text", "text": "Hello, how are you?"}],
        }
    ],
})

response = bedrock_runtime.invoke_model(
    modelId=model_id,
    body=request_body,
    headers=create_headers(
        use_case_name="bedrock_example",
        user_id="example_user",
        limit_ids=["bedrock_limit"],
        request_tags=["bedrock-example"]
    )
)

# Process the response
response_body = json.loads(response['body'].read())
print(response_body['content'][0]['text'])

With this configuration, Pay-i automatically tracks API calls and calculates costs without adding any latency to your requests. You can also use decorators for tracking multiple related API calls - see Custom Instrumentation for more details.

Optional Proxy Routing (For Block Limits)

If you need to implement Block limits that prevent requests from being sent to the provider when a budget is exceeded, use Pay-i's proxy configuration. For AWS Bedrock, this requires using event handlers to redirect requests:

import boto3
import json
import os
import urllib3
from payi.lib.helpers import payi_aws_bedrock_url
from payi.lib.instrument import payi_instrument

# 1. Initialize with proxy mode enabled
payi_instrument(config={"proxy": True})

# 2. Get the Pay-i proxy URL
payi_aws_bedrock_url_path = payi_aws_bedrock_url()

# 3. Set up event handlers to redirect through Pay-i
def handle_payi_parameters(params, context, **kwargs):
    context["extra_headers"] = params.pop("extra_headers", {})

def redirect_to_payi(request, event_name, **kwargs):
    if not event_name.startswith('request-created.bedrock-runtime'):
        return
    
    parsed_url = urllib3.util.parse_url(request.url)
    route_path = parsed_url.path
    request.url = f"{payi_aws_bedrock_url_path}{route_path}"

    request.headers['xProxy-api-key'] = PAYI_API_KEY
    request.headers['xProxy-Provider-BaseUri'] = parsed_url.scheme + "://" + parsed_url.host
    extra_headers = request.context.get('extra_headers', {})
    for key, value in extra_headers.items():
        request.headers[key] = value


def register_bedrock_client_callbacks(client, model):
    # Process the extra_headers parameter passed to the bedrock runtime call
    client.meta.events.register(f'provide-client-params.bedrock-runtime.{model}',
                                handle_payi_parameters,
                                unique_id=handle_payi_parameters)

    # Redirect the request to the Pay-i endpoint after the request has been signed
    client.meta.events.register_last('request-created',
                                     redirect_to_payi,
                                     unique_id=redirect_to_payi)
    
# 4. Create the bedrock client
bedrock = boto3.client(
    'bedrock-runtime',
    region_name=AWS_REGION,
)

# 5. Register client callbacks for all supported Bedrock operations
register_bedrock_client_callbacks(bedrock, 'InvokeModel')
register_bedrock_client_callbacks(bedrock, 'InvokeModelWithResponseStream')
register_bedrock_client_callbacks(bedrock, 'Converse')
register_bedrock_client_callbacks(bedrock, 'ConverseStream')

# Use the client with boto3 for AWS Bedrock
request_dict = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 512,
    "temperature": 0.5,
    "messages": [
        {
            "role": "user",
            "content": [{"type": "text", "text": "Hello, how are you?"}],
        }
    ],
}
request_body = json.dumps(request_dict)
model_id = 'anthropic.claude-3-haiku-20240307-v1:0'

response = bedrock.invoke_model(
    modelId=model_id,
    body=request_body,
)

For detailed information about proxy configuration, including when to use it and how it works, see the Pay-i Proxy Configuration guide.

Advanced Configuration

Note: The following advanced configurations can be applied to both default instrumentation and proxy configurations, though the proxy configuration requires additional event handlers as shown above.

AWS Session Configuration

For more complex AWS configurations:

import boto3

# Create a custom session with specific credentials
session = boto3.Session(
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY',
    region_name='us-west-2'
)

# Use the session to create a client
bedrock_runtime = session.client(
    service_name='bedrock-runtime'
)

# For proxy configuration, you would:
# 1. Create the client using the session
# 2. Register the event handlers as shown in the proxy configuration example

Request Timeouts

For operations that might take longer, configure appropriate timeouts:

import boto3
from botocore.config import Config

# Configure with longer timeouts for large requests
bedrock_config = Config(
    connect_timeout=10,  # 10 seconds
    read_timeout=60,     # 60 seconds
    retries={'max_attempts': 3}
)

# Use the config when creating the client
bedrock_runtime = boto3.client(
    service_name='bedrock-runtime',
    region_name='us-west-2',
    config=bedrock_config
)

# For proxy configuration, you would:
# 1. Create the client with the config
# 2. Register the event handlers as shown in the proxy configuration example

Related Resources

Quickstart Guides

Conceptual Guides

Example Repositories

Updated 10 days ago