What are Experiences?

The majority of GenAI use cases are not accomplished in a single API call. They are a series of calls, either as part of a chat conversation, an agent, or a workflow that builds on a shared context. This is what Pay-i refers to as “Experiences”. They are a collection of related, cost-generating events that contribute to a single user experience. Since these events are always done together, their costs need to be understood in aggregate.

For example, imagine a simple agentic system with two LLMs talking to each other to accomplish a goal. Each time they go back and forth, they generate a cost. That cost increases as the conversation length increases. Given the number of tokens used for each LLM inference may greatly differ for each step of the agent, and the number of steps in the conversation may greatly differ between executions, and either may change over time, the price of each instance of this agentic “experience” will be different.

Therefore, the cost for these types of use cases can only be expressed probabilistically.

Why not just use Tags?

You may initially think that simply tagging all requests associated with an experience is sufficient. However, a tagging system only provides two insights:

How much was spent in total on all requests having to do with the experience, across all the times that experience was executed.
How much was spent on average per request that was a part of any experience.

Pay-i’s Experience feature enables you to understand the breakdown of costs for series of related cost-generating events. Crucially, it lets you answer the following questions:

How much did any one execution of the experience cost me, across all involved steps?
What is the distribution of costs for this experience across all the times it was executed (e.g., min/max/percentiles)?
Which experience instances costed significantly more/less than average?
...and much more.

By using Experiences in combination of tags, Pay-i can answer all questions and provide all possible pivots to help you understand your GenAI spending.

More information can be found in the Experiences (Continued) section.