Custom Categories and Resources

Pay-i's tools can be used to track costs from any source, including non-GenAI related sources. If you want to add a GenAI model that Pay-i doesn't yet have native support for, or incorporate costs from other sources so that you can correlate them with your GenAI costs, you can accomplish this by creating custom categories and resources.

See the Categories and Resources section to learn about the types of properties each contains.

Defining a Custom Category

1. Custom category and resource definitions are organization-wide and apply to all applications.
2. Categories cannot be created with the "system." prefix, as those are reserved for categories managed by Pay-i.
3. Categories themselves are just containers for resources, and therefore they are not created directly. Instead, when creating a resource, you specify the name of the category it should belong to.

Defining a Custom Resource

1. Resources must be uniquely named within a category but are not unique across categories. For example, you can have a llama-3.1-70b resource defined in a together.ai category, and in a lambdalabs category with different prices.
2. When creating a new resource, if the resource name already exists within the specified category, a new Resource Version will be created.

Unit Types

Most resources have multiple, differently priced units. For example, LLMs from major Providers typically have one or more of the following pricing units, in addition to their standard text token costs:

1. Batch
2. Cache
3. Vision
4. Long Context (e.g., Gemini)
5. etc.

Further, each of these unit types typically has different prices across input and output.

Pay-i therefore tracks the prices of resources as a dictionary of unit types, each with an input and output price per unit. When defining a resource, you specify the types and the price per each unit as follows.

"units": {
    "text": {
      "input_price": 0.000003,
      "output_price": 0.000015
    },
    "text_cache_write": {
      "input_price": 0.00000375,
      "output_price": 0
    },
    "text_cache_read": {
      "input_price": 0,
      "output_price": 3e-7
    }
  }

When Pay-i is acting as a proxy for Managed Categories and Resources, or when Pay-i is Proxy Routing Custom Resources, it automatically tabulates all of the unit types and amounts for each request, even in cases when that information is not returned by the Provider.

When using Ingest, you are required to specify the types and amounts for each unit you wish to ingest.

Resource Creation Fields

The following table describes the available fields for custom resources.

Resource Versions

The price of resources, or individual unit types within a resource, will likely change over time. Pay-i keeps track of the price history of all unit types in a resource for use in viewing/using historical data. Further, when you Ingest events that occurred in the past, the correct pricing data will be used for the cost of the resource at that time.

When a custom resource is first created, a resource_id is returned which uniquely identifies all aspects of the resource definition, including the category, pricing, and start_timestamp.

If you create a resource that already exists within the specified category, then you are creating a new version of that resource whose pricing is in effect from the specified start_timestamp onwards, and a new resource_id is returned.

For example, the resource my-llm in the category SelfHosted was introduced on 05/13/24 with the following pricing for the text unit type:

Create Resource: my-llm

• category = SelfHosted
• start_timestamp = 2024-05-13T00:00:00
• text
  • input_price = 0.000005
  • output_price = 0.000015

When this resource was created, Pay-i returned a resource_id of 10.

On 08/06/24, the text pricing for my_llm was changed:

Create Resource: my-llm

• category = SelfHosted
• start_timestamp = 2024-08-06T00:00:00
• text
  • input_price = 0.0000025
  • output_price = 0.00001

Pay-i returned a resource_id of 11.

Requests for my-llm with an event_timestamp on or after 05/13/24, but before 08/06/24, will use the pricing for resource_id 10.

Any events ingested on or after 08/06/24 will use the pricing for resource_id 11.

Using Custom Resources with Proxy Routing

If you need to use custom resources while still maintaining Pay-i as a proxy, please contact [email protected].

Related APIs

• Create a Resource
• Get a list of versions of a Resource
• Get all Resources for a Category