Overview

A Category is simply a grouping of resources.

A Resource has four properties for each Unit Type.

Unit Types

A Resource can have any number of Unit Types. Unit Types represent the individually-priced modalities that the resource supports. For example, the most common modality for LLMs is text. Text is commonly charged by the token or the character, and this varies significantly between models. However, the modality across all of them is "text", and so the Unit Type is "text".

Different LLMs support many additional Unit Types, such as vision, various types of caching, batch pricing, and so on. An image generation model might have the "image" Unit Type, where each image is a unit. A video generation model might have the "video" Unit Type where each unit represents 1 second of generated video.

Simply put, a resource has a Unit Type for each type of billable output. For each Unit Type, a resource specifies the following properties:

Cost per input unit in USD (e.g., the cost for a prompt token for an LLM).
The max number of input units (e.g., the largest number of input tokens supported by the LLM).
Cost per output unit in USD (e.g., the cost of 1 completion token for an LLM, or the cost of 1 second of generated video).
The max number of output units (e.g., the largest number of output tokens supported by the LLM, or the most number of images that can be returned in one call to a diffusion model).

Note that not all Unit Types are supported when Pay-i is configured as a proxy, such as real-time audio.

Managed and Custom Categories and Resources

For popular GenAI Providers (OpenAI, Anthropic, Gemini, etc.), Pay-i maintains the categories, resources, and their unit prices automatically. You can see the list that Pay-i maintains in the Managed Categories and Resources section.

You can create your own categories and resources, either for models not known to Pay-i or to support cost tracking from non-GenAI sources. Learn more in the Custom Categories and Resources section.

Categories whose resources are managed by Pay-i have a reserved prefix: "system.", to differentiate them from custom categories. If you are part of an enterprise with a negotiated price for a Provider that Pay-i manages, you can create your own custom category with the negotiated prices.

For example, Pay-i automatically keeps all of the latest AWS Bedrock pricing up to date under the system.bedrock Category. If you are part of an enterprise that has negotiated custom pricing with Bedrock, then you can create a custom bedrock Category, and define the custom resource pricing within it. Pay-i can still proxy API calls when defining custom pricing in this way. To use this feature, refer to Using Custom Resources with Proxy Routing.

For all resources, system or custom, Pay-i keeps a historic record as resource prices change, allowing you to see the impact on your business over time.

Example Categories:

system.openai
system.anthropic
MyCustomCategory

Example Resources:

gpt-4-0613
claude-3-sonnet-20240229
MyCustomResource

Fixed Cost Resources

Pay-i also supports Fixed Cost Resources, such as Provisioned Throughput Units, Model Units of Provisioned Throughput, and other forms of resources whose spend is fixed per unit time instead of being consumption based. If you need to leverage Fixed Cost Resources, please contact [email protected].