Overview

A Category is simply a grouping of resources.

A Resource has five properties:

Cost per input unit in USD (e.g., the cost for a prompt token for an LLM).
The max number of input units (e.g., the largest number of input tokens supported by the LLM).
Cost per output unit in USD (e.g., the cost of 1 second of generated video).
The max number of output units (e.g., the most number of images that can be returned in one call to a diffusion model).
Some resources are "Ingest Only", meaning they can only be used as part of an Ingest workflow and are not supported with Pay-i as a proxy.
- This is typically for very latency-sensitive use cases (such as the OpenAI Realtime API), or for custom categories and resources, described below.

Managed and Custom Categories and Resources

For popular GenAI Providers (OpenAI, Anthropic, Gemini, etc.), Pay-i maintains the categories, resources, and their unit prices automatically. You can see the list that Pay-i maintains in the Managed Categories and Resources section.

You can create your own categories and resources, either for models not known to Pay-i or to support cost tracking from non-GenAI sources. Learn more in the Custom Categories and Resources section.

Categories whose resources are managed by Pay-i have a reserved prefix: "system.", to differentiate them from custom categories. If you are part of an enterprise with a negotiated price for a Provider that Pay-i manages, you can create your own custom category with the negotiated prices.

For example, Pay-i automatically keeps all of the latest AWS Bedrock pricing up to date under the system.bedrock Category. If you are part of an enterprise that has negotiated custom pricing with Bedrock, then you can create a custom bedrock Category, and define the custom resource pricing within it. Pay-i can still proxy API calls when defining custom pricing in this way. To use this feature, please see the Using Custom Resources with Proxy Routing section.

For all resources, system or custom, Pay-i keeps a historic record as resource prices change, allowing you to see the impact on your business over time.

Example Categories:

system.openai
system.anthropic
MyCustomCategory

Example Resources:

gpt-4-0613
claude-3-sonnet-20240229
MyCustomResource

Fixed Cost Resources

Pay-i also supports Fixed Cost Resources, such as Provisioned Throughput Units, Model Units of Provisioned Throughput, and other forms of resources whose spend is fixed per unit time instead of being consumption based. If you need to leverage Fixed Cost Resources, please contact [email protected].