Overview

Value metrics in Pay-i allow organizations to quantify the business impact and return on investment (ROI) from their GenAI applications. Pay-i offers two types of Value metrics:

1. Value Generated: Measures financial impact in monetary units (dollars, euros, etc.)
2. Time Saved: Measures operational efficiency gains in time units (minutes, hours, etc.)

These metrics build upon the foundation of KPIs but serve a different purpose: while KPIs are custom application metrics that track specific aspects of your GenAI implementation (like user satisfaction or content quality), Value metrics translate these application-specific metrics into organization-wide business outcomes.

Required Setup Sequence

Value metrics require a specific setup sequence:

1. First, create a Use Case to organize your GenAI implementation
2. Next, define KPIs for that Use Case to measure its performance
3. Finally, configure Value metrics using Business Equations that reference those KPIs

If you attempt to configure Value metrics without completing these prerequisites, you'll see a message prompting you to "Create a Use Case to manage KPIs and define Value."

Key Distinctions from KPIs

Value metrics enable organizations to bridge the gap between technical performance and financial outcomes, transforming usage data into actionable insights about the actual business value generated by GenAI investments.

Business Equations

Value metrics are configured using Business Equations - simple formulas that determine how Value is calculated for each request. These powerful formulas are the "secret sauce" that converts technical metrics into concrete business impact.

Business Equations:

• Use basic arithmetic operations (+, -, ×, ÷)
• Can incorporate KPIs defined for the specific Use Case
• Can include standard metrics that Pay-i automatically tracks (like Instance Cost and Instance E2EL)
• Can include constants (like baseline values for comparison)
• Are evaluated per-request and then aggregated automatically

A common Business Equation pattern is: -Instance Cost + (Constant × KPI)

For example, a "Time Saved" equation might subtract the actual processing time (measured by End-to-End Latency or a custom KPI) from a baseline time that represents how long the process would take without AI assistance.

Formula Components

Business Equations can use:

• KPIs: Custom metrics you've defined for your Use Case (like user satisfaction, content quality)
• Standard metrics: Metrics automatically collected by Pay-i (like Instance E2EL - End-to-End Latency, Instance Cost)
• Constants: Fixed values you define (e.g., baseline comparison times)
• Arithmetic operators: Simple mathematical operations (+, -, ×, ÷)

The formula is evaluated for each individual request, allowing Pay-i to provide:

• Time series charts showing trends over time
• Frequency histograms showing distribution of values
• Statistical measures (P50, P95, P99 percentiles)
• Aggregated totals across all requests

Common Value Metrics Implementation

Based on best practices from successful Pay-i implementations, the most effective Value metric implementations focus on consistent measurement across all use case types. Common examples include:

• Time Saved: A Value metric measuring minutes or hours saved compared to manual processes. This metric is typically applied consistently across various use case types, such as content generation, chatbots, and document processing.

• Money Saved: A Value metric measuring direct cost reduction from using the GenAI feature. This is often calculated by converting time savings into financial impact based on labor costs.

Best practice is to implement these metrics with identical names and calculation methodologies across all use case types. For example, using "time" with the same definition for all AI features enables automatic aggregation of total time saved across all use cases.

These cross-application Value metrics enable organizations to compare ROI across different AI implementations and calculate aggregate business value at the organizational level.

Business Benefits

Implementing Value metrics delivers several key benefits:

Quantifiable ROI Measurement

Value metrics provide concrete, data-driven evidence of return on investment from GenAI-infused applications. This quantification helps:

• Justify existing GenAI investments to stakeholders and leadership
• Support business cases for expanding successful GenAI programs
• Compare value generated across different AI applications
• Translate technical performance into dollars and cents for executive dashboards

Version Comparison and Trade-off Analysis

Pay-i provides powerful tools for comparing different versions of your use cases:

• Side-by-side cost comparisons between versions (e.g., v1 vs v2)
• Value-per-dollar metrics to identify the most cost-effective implementations
• Automatic trade-off recommendations based on KPI goals and cost considerations
• Visual indicators of which versions maximize value (e.g., green bars for positive ROI)

This version-aware approach helps teams understand whether increased costs in newer versions deliver proportional business value.

Strategic Decision Making

With Value metrics, organizations can make more informed decisions about:

• Which GenAI use cases to prioritize for expansion
• Where to allocate AI development resources
• When to refine or pivot specific implementations
• How to optimize spending across different models and providers
• Which version of a use case provides the best balance of cost and business outcome

Optimization Targeting

Value metrics highlight opportunities for improvement by identifying:

• High-value use cases that deserve additional investment
• Underperforming implementations that need refinement
• Cost-effectiveness ratios across different AI approaches
• Patterns that correlate with maximum business impact

Value Metrics in Pay-i

Pay-i's Value metrics system integrates several data sources to create a comprehensive view of business impact:

Relationship to Usage Data

Value metrics build upon Pay-i's robust usage tracking capabilities:

• Usage data provides the foundation for understanding costs and activity levels
• User interactions and outcomes are mapped to specific business impacts
• Patterns in usage data can reveal optimization opportunities

Connection to Costs

Pay-i automatically calculates costs for all tracked GenAI interactions, enabling:

• Direct comparison of costs vs. value generated
• ROI calculations across different time periods
• Cost-effectiveness analysis by model, use case, or user segment

Value Calculation Using Business Equations

Pay-i uses a formula-based approach for calculating Value metrics. These Business Equations:

• Are created in the Pay-i UI using a visual formula builder
• Use basic arithmetic operations (+, -, ×, ÷)
• Can incorporate KPIs from the associated Use Case
• Can include constants (like baseline values for comparison)
• Are evaluated per-request and automatically aggregated

For example, common formulas include:

1. Time Saved formula:
   (Baseline Time Constant) - (Instance E2EL)
   This calculates time saved compared to a baseline.

2. Value Generated with KPI-based logic:
   -Instance Cost + ($5 × Deflection)
   This subtracts the cost of each request but adds $5 when the "Deflection" KPI is true (meaning human intervention was avoided).

Each formula is applied to individual requests, with results visible in real-time examples in the preview section. You can also incorporate use case-specific KPIs in your formulas by clicking the "Add KPI" button.

Configuring Value Metrics

Accessing Value Configuration

Value metrics are configured exclusively through the Pay-i UI:

1. Log in to the Pay-i Developer Portal
2. Navigate to your organization dashboard
3. Select the "Configuration" tab
4. Choose a specific use case from the dropdown (e.g., "Chat-Bot", "Image-Ad")
5. Select the "Value" tab, then choose either "Time Saved" or "Value Generated"

Using the Formula Builder

The Formula Builder provides a simple visual interface for creating Value calculations:

1. In the "Define Time Saved" or "Define Value Generated" section:

   • You'll see a formula workspace with mathematical operators (+, -, =)
   • The default starting value is typically "$0.00" for Value Generated or "0 m" for Time Saved

2. Build your formula by:

   • Clicking "+" to add components
   • Adding constant values (like baseline times or costs)
   • Using operators (+, -, etc.)
   • Adding standard metrics (like Instance E2EL or Instance Cost)
   • Clicking "Add KPI" to incorporate use case-specific KPIs (like "Deflection Rate" or "Clickthrough Rate")

3. For Time Saved metrics, formulas typically:

   • Start with a baseline time (how long something would take without AI)
   • Subtract the actual time taken (measured by a KPI or Instance E2EL)
   • Result is time saved per request in minutes

4. For Value Generated metrics, formulas typically:

   • Represent monetary value in dollars/euros
   • May involve costs avoided, revenue generated, or other financial impacts

5. The Preview section shows:

   • Real examples of your formula applied to actual request data
   • Option to view different samples with "Next Sample" button

6. When satisfied with your formula, click "Save" to apply it

Understanding Value Analytics

Organization-Wide Value Dashboards

Pay-i provides executive-level dashboards that offer a complete view of your GenAI investments:

1. Total GenAI spend across all initiatives
2. Aggregate Value Generated and Time Saved metrics
3. Multi-cloud, multi-model, multimodal visibility
4. FinOps Foundation-certified accuracy

This organization-level view frames the conversation around business outcomes rather than just costs.

Accessing and Navigating Value Dashboards

Once your Value metrics are configured and data is collected:

1. Select "Value" from the main navigation menu
2. Choose the "Value Generated" or "Time Saved" tab
3. Use the date range selector at the top to adjust the analysis period
4. Drill down using the navigation path: Organization → Business Unit → Application → Use Case

Value Visualizations and Analysis

Pay-i provides comprehensive visualization and analysis tools for Value metrics:

Time Series Charts

These charts track your Value metrics over time:

• Shows how value accumulates or changes over days, weeks, or months
• Different colored lines represent different use case versions
• Vertical lines mark version changes
• Helps identify trends, improvements, or regressions

Frequency Histograms

These histograms show the distribution of Value across individual requests:

• X-axis shows value ranges (dollars for Value Generated, minutes for Time Saved)
• Y-axis shows number of instances in each range
• Hover information shows detailed statistics (like percentiles)
• Helps identify common outcomes and value distribution patterns

Detailed Results Table

Below the visualizations, a comprehensive table provides:

• Use case names and versions
• Number of instances processed
• Aggregate totals (dollars generated, time saved in weeks/months)
• Results broken down by version for easy comparison
• Color-coded indicators for positive value (green) vs. negative value (red)

Best Practices for Value Metrics

Start with Clear Business Objectives

Before configuring Value metrics:

1. Define what "value" means for your specific AI implementations
2. Identify measurable components that contribute to this value
3. Determine appropriate baselines for comparison
4. Focus on metrics that will drive decision-making

Use Consistent Measurement Approaches

For effective Value metrics:

1. Apply consistent formulas across similar use cases
2. Use the same units and calculation methods where possible
3. Establish clear baselines that represent non-AI alternatives
4. Document your formula logic for future reference
5. Set version-specific KPI goals to track improvements over time

Build on Your Metrics Foundation

Value metrics work best when built on well-defined metrics:

1. First define KPIs that measure application-specific outcomes (like quality ratings or completion rates)
2. Utilize standard metrics that Pay-i automatically collects (like Instance E2EL or Instance Cost)
3. Then create Value metrics that translate these inputs into business impact
4. Ensure your metrics reliably track what matters to your organization
5. Update both KPIs and Value metrics as your use cases evolve

Examples

Real-World Use Case Examples

Here are examples of how different use case types implement Value metrics based on their specific KPIs:

Chat-Bot Use Case

A customer support chatbot might have:

1. KPIs specific to this use case:

   • "Deflection Rate" (Boolean - tracks whether human intervention was needed)
   • "User Satisfaction" (Boolean - tracks if the user was satisfied)

2. Time Saved Value Metric:

   • Formula: 5 m - Instance E2EL
   • This subtracts the actual end-to-end latency from a 5-minute baseline
   • Optionally incorporates the "Deflection Rate" KPI for further refinement

3. Results Measurement:

   • Total minutes/hours/months saved across all chat interactions
   • Distribution of time saved per interaction
   • Comparison of time saved between different versions

Image-Ad Use Case

An AI-powered advertising use case might have:

1. KPIs specific to this use case:

   • "Clickthrough Rate" (Boolean - tracks if ads were clicked)
   • "Close Rate" (Boolean - tracks if sales were closed)
   • "Customer Satisfaction" (Likert scale - rates satisfaction level)
   • "Conversion $ Value" (Number - tracks the monetary value of conversions)

2. Value Generated Metric:

   • Formula: $0.00 + Conversion $ Value - Instance Cost
   • This calculates net profit by subtracting costs from conversion value
   • Can incorporate KPIs like "Clickthrough Rate" to refine the calculation

3. Results Measurement:

   • Total value generated across all ad interactions
   • Distribution of value per ad interaction
   • ROI comparison between different ad versions

Step-by-Step Configuration Guide

Let's walk through the exact steps to configure both types of Value metrics:

Configuring a Time Saved Metric

1. Access the Configuration:

   • Navigate to the Configuration tab
   • Select your use case from the dropdown (e.g., "Chat-Bot")
   • Select "Time Saved" tab

2. Build the Formula:

   • The interface shows "Default Time Saved" with a starting value of "0 m"
   • Click "+" to add a component
   • Enter a baseline time (e.g., "5 m" representing average time without AI)
   • Click "-" to subtract
   • Select "Instance E2EL" from available metrics
   • Optionally click "Add KPI" to incorporate a use case KPI

3. Review and Save:

   • The preview section will show real examples with calculated time saved
   • Examine several samples using the "Next Sample" button
   • When satisfied with the formula's results, click "Save"

Configuring a Value Generated Metric

1. Access the Configuration:

   • Navigate to the Configuration tab
   • Select your use case from the dropdown (e.g., "Image-Ad")
   • Select "Value Generated" tab

2. Build the Formula:

   • The interface shows "Default Value Generated" with a starting value of "$0.00"
   • Click "+" to add a component
   • Add a KPI (e.g., "Conversion $ Value") or a constant value
   • Use operators to build your formula
   • Include "Instance Cost" to account for expenses

3. Review and Save:

   • Examine the preview with real-world data samples
   • Verify the values make business sense
   • Click "Save" to activate your Value metric

Setup Process Summary

To implement Value metrics in Pay-i:

1. Create a Use Case to organize your GenAI implementation

2. Define KPIs for that Use Case to measure its performance

3. Configure Value Metrics using the formula builder:

   • For Time Saved: create a formula that calculates time efficiency gains
   • For Value Generated: create a formula that calculates financial impact

4. Analyze the Results through:

   • Time series charts tracking value over time
   • Histograms showing value distribution
   • Statistical tables with aggregated totals

This process transforms technical performance data into concrete business outcomes that demonstrate the ROI of your GenAI investments. Because each use case has its own KPIs and Value metrics, you can track both detailed performance and business impact for each distinct AI application.

Pay-i's Value metrics ultimately help answer the most critical question for AI investments: whether each version of a GenAI use case is worth its cost.

Related Resources

• For understanding the broader value of Pay-i instrumentation, see Value of Pay-i Instrumentation
• For details about user-defined KPIs, see Use Case KPIs
• For KPI implementation guidance, see Working with KPIs