Overview

A Category is simply a grouping of resources.

A Resource has five properties:

Cost per input unit in USD (e.g., the cost for a prompt token for an LLM).
The max number of input units (e.g., the largest number of input tokens supported by the LLM).
Cost per output unit in USD (e.g., the cost of 1 second of generated video).
The max number of output units (e.g., the most number of images that can be returned in one call to a diffusion model).
Some resources are incompatible with Proxy Routing. Usage data for these must be submitted after the provider interaction using the methods associated with the Direct Provider Call with Telemetry mode (explicitly via the Ingest API or implicitly via decorators).
- This limitation typically applies to very latency-sensitive use cases (like the OpenAI Realtime API) or custom categories and resources.

Managed and Custom Categories and Resources

For popular GenAI Providers (OpenAI, Anthropic, Gemini, etc.), Pay-i maintains the categories, resources, and their unit prices automatically. You can see the list that Pay-i maintains in the Managed Categories and Resources section.

You can create your own categories and resources, either for models not known to Pay-i or to support cost tracking from non-GenAI sources. Learn more in the Custom Categories and Resources section.

Categories whose resources are managed by Pay-i have a reserved prefix: "system.", to differentiate them from custom categories. If you are part of an enterprise with a negotiated price for a Provider that Pay-i manages, you can create your own custom category with the negotiated prices.

For example, Pay-i automatically keeps all of the latest AWS Bedrock pricing up to date under the system.bedrock Category. If you are part of an enterprise that has negotiated custom pricing with Bedrock, then you can create a custom bedrock Category, and define the custom resource pricing within it. Pay-i can still proxy API calls when defining custom pricing in this way. To use this feature, refer to Using Custom Resources with Proxy Routing.

For all resources, system or custom, Pay-i keeps a historic record as resource prices change, allowing you to see the impact on your business over time.

Example Categories:

system.openai
system.anthropic
MyCustomCategory

Example Resources:

gpt-4-0613
claude-3-sonnet-20240229
MyCustomResource

Fixed Cost Resources

Pay-i also supports Fixed Cost Resources, such as Provisioned Throughput Units, Model Units of Provisioned Throughput, and other forms of resources whose spend is fixed per unit time instead of being consumption based. If you need to leverage Fixed Cost Resources, please contact [email protected].