Guides
Guides

AWS Bedrock Provider Configuration

Suggest Edits

Overview

This guide explains how to configure AWS Bedrock to work with Pay-i Instrumentation. AWS Bedrock is a supported Provider with several Resources that Pay-i can track.

Note: AWS Bedrock integration with Pay-i uses the track_context() function to capture request state and apply business context. This works with both direct provider calls and proxy mode configurations.

Prerequisites

To use Pay-i with AWS Bedrock, you'll need:

SDK Support

The examples in this guide use the Pay-i Python SDK, which provides comprehensive support for AWS Bedrock integration. If you're using a different programming language, you can utilize the Pay-i OpenAPI specification to generate a client SDK for your language of choice. The core concepts remain the same, though the exact implementation details may vary depending on the language and client library used.

Basic Setup

First, install the required packages:

# Install the required packages
pip install payi boto3 python-dotenv

Let's start with the common configuration variables needed for both approaches:

import os
import json
import boto3
from payi.lib.instrument import payi_instrument, track_context

# API keys and configuration
PAYI_API_KEY = os.getenv("PAYI_API_KEY", "YOUR_PAYI_API_KEY")
AWS_REGION = "us-west-2"  # Use your preferred region

Direct Provider Call with Telemetry (Default)

The default approach makes API calls directly to AWS Bedrock while Pay-i tracks usage. AWS Bedrock requires using the track_context() context manager:

# Initialize Pay-i instrumentation
payi_instrument()

# Configure direct AWS Bedrock client
bedrock_runtime = boto3.client(
    service_name='bedrock-runtime',
    region_name=AWS_REGION
)

# Create the request body
model_id = 'anthropic.claude-3-haiku-20240307-v1:0'
request_body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 512,
    "temperature": 0.5,
    "messages": [
        {
            "role": "user",
            "content": [{"type": "text", "text": "Hello, how are you?"}],
        }
    ],
})

# Use track_context to associate business context with your request
with track_context(
    use_case_name="bedrock_example",
    user_id="example_user",
    limit_ids=["bedrock_limit"],
    request_tags=["bedrock-example"]
):
    response = bedrock_runtime.invoke_model(
        modelId=model_id,
        body=request_body
    )

# Process the response
response_body = json.loads(response['body'].read())
print(response_body['content'][0]['text'])

With this configuration, Pay-i automatically tracks API calls and calculates costs without adding any latency to your requests.

Multiple Related Calls

For more complex scenarios where you need to track multiple related API calls, you can use track_context() to group them together:

# Use track_context for grouping related calls
with track_context(
    use_case_name="conversation_flow",
    user_id="data_scientist_123",
    limit_ids=["aws_budget"],
    request_tags=["bedrock", "conversation"]
):
    # First request
    response1 = bedrock_runtime.invoke_model(
        modelId=model_id,
        body=request_body1
    )
    
    # Process first response
    response_body1 = json.loads(response1["body"].read())
    first_response = response_body1["content"][0]["text"]
    
    # Second request (follow-up)
    response2 = bedrock_runtime.invoke_model(
        modelId=model_id,
        body=request_body2
    )
    
    # Both requests will be associated with the same tracking context

Optional Proxy Routing (For Block Limits)

If you need to implement Block limits that prevent requests from being sent to the provider when a budget is exceeded, use Pay-i's proxy configuration:

import boto3
import json
from payi.lib.instrument import payi_instrument, track_context

# Initialize with proxy mode enabled
payi_instrument(config={"proxy": True, "use_case_name": "bedrock_example"})

# Create the bedrock client
bedrock = boto3.client(
    'bedrock-runtime',
    region_name=AWS_REGION
)

# Create request body
request_body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 512,
    "temperature": 0.5,
    "messages": [
        {
            "role": "user",
            "content": [{"type": "text", "text": "Hello, how are you?"}],
        }
    ],
})
model_id = 'anthropic.claude-3-haiku-20240307-v1:0'

# Make API call with tracking context
with track_context(
    user_id="example_user",
    limit_ids=["bedrock_limit"],
    request_tags=["bedrock-example"]
):
    response = bedrock.invoke_model(
        modelId=model_id,
        body=request_body
    )

# Process the response
response_body = json.loads(response["body"].read())
print(response_body["content"][0]["text"])

# With proxy mode, you get access to real-time cost information
xproxy_result = response_body['xproxy_result']
print(json.dumps(xproxy_result, indent=4))

To enable proxy mode, simply set proxy=True in the config:

payi_instrument(config={"proxy": True})

Important Requirements

  1. For both direct calls and proxy mode, you MUST call payi_instrument() before creating any Bedrock client.
  2. Always use track_context() to associate business context with your requests.
  3. For proxy mode, no callback registration is needed; simply set proxy=True in the config.

For detailed information about proxy configuration, including when to use it and how it works, see the Pay-i Proxy Configuration guide.

Advanced Configuration

AWS Client Configuration Options

The AWS Bedrock client supports various configuration options for authentication, session management, and performance tuning. For details on these options, refer to the AWS Boto3 Documentation.

When using these options with Pay-i, remember that Pay-i instrumentation must be initialized before creating any Bedrock clients:

import boto3
from botocore.config import Config
from payi.lib.instrument import payi_instrument, track_context

# Initialize Pay-i instrumentation first
# For standard mode
payi_instrument()
# Or for proxy mode
# payi_instrument(config={"proxy": True})

# Then configure and create the AWS Bedrock client with your desired options
# Example: Custom session with specific credentials and timeouts
session = boto3.Session(
    region_name='us-west-2'
    # Other session options as needed
)

# Configure client options
bedrock_config = Config(
    connect_timeout=10,  # Connection timeout in seconds
    read_timeout=60,     # Read timeout in seconds
    retries={'max_attempts': 3}
    # Other client configuration options as needed
)

# Create the client with your configuration
bedrock_runtime = session.client(
    service_name='bedrock-runtime',
    config=bedrock_config
)

# Always use track_context with your AWS Bedrock requests
with track_context(
    use_case_name="bedrock_example",
    user_id="example_user"
):
    # Your AWS Bedrock API calls here
    # ...

For detailed information about AWS authentication methods, session configuration, and client options, refer to the AWS SDK for Python (Boto3) documentation.

Related Resources

Quickstart Guides

Conceptual Guides

Example Repositories

Updated 6 days ago