> ## Documentation Index
> Fetch the complete documentation index at: https://help.loopiq.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Review Agent Usage Scorecards

## What This Does

Agent usage scorecards help admins understand which agents and packs are running, what they produced, how often they failed, and whether users accepted the recommendations.

Use scorecards to monitor:

* invocation count
* failure rate
* average latency
* stale outputs
* accepted recommendations
* rejected recommendations
* created actions
* user feedback
* tenant and workflow usage

## Who Should Use This

Use this guide if you are:

* an organization admin
* an AI governance owner
* a platform owner
* responsible for release governance, testing, security, or incident workflows

## Open Agent Scorecards

1. Open `Organization Settings` or the admin area.
2. Go to `AI` or `Agent Scorecards`.
3. Select the tenant, workflow, pack, or date range.
4. Review the scorecard summary.
5. Drill into agent or pack details when a metric needs attention.

## Key Metrics

### Invocations

Shows how often an agent or pack ran.

High invocation count can mean the workflow is valuable, noisy, or being triggered too often. Review the source of the runs if usage spikes unexpectedly.

### Failures

Shows runs that failed because of service errors, provider issues, malformed output, timeout, missing context, or validation failures.

Investigate failures when:

* the same agent fails repeatedly
* failure rate increases after deployment
* a release-critical pack fails
* source events are being quarantined

### Latency

Shows how long agent or pack runs take.

High latency may affect user experience or lifecycle automation. For release certification events, review whether durable trigger jobs are delayed or retrying.

### Stale Outputs

Shows outputs that may no longer match current evidence.

Outputs can become stale when:

* provider evidence refreshes after the agent ran
* release status changes
* approvals change
* linked records are updated

### Accepted Recommendations

Shows recommendations users accepted or acted on.

Use this to identify useful agents and workflows.

### Created Actions

Shows actions created from agent recommendations, such as remediation tasks, sprint items, test coverage work, or incident follow-ups.

Created actions should be traceable to the originating signal, pack run, and user approval where required.

### Feedback

Shows user feedback on agent quality.

Use feedback to identify agents that need better prompts, schemas, grounding, or provider data.

## How to Interpret Scorecards

Healthy scorecards usually show:

* successful runs for high-value workflows
* low failure rate
* bounded latency
* recommendations tied to accepted actions
* few stale outputs
* clear tenant and workflow scope

Unhealthy scorecards may show:

* repeated failures for the same agent
* high latency on release-critical workflows
* many stale outputs
* recommendations that users consistently reject
* actions without enough evidence or approval context

## What to Do When Something Looks Wrong

If an agent has high failure rate:

* check provider integration health
* check whether required evidence is missing
* check recent deployments
* review malformed output or schema validation messages

If outputs are stale:

* refresh the related provider evidence
* rerun the pack
* confirm scheduled jobs are active
* confirm source events are being accepted

If users reject many recommendations:

* review whether recommendations are too generic
* check whether the agent has the right grounding sources
* check whether the workflow should require stronger schemas
* collect feedback from the users rejecting the recommendations

## Recommended Practices

* Review scorecards weekly for release-critical packs.
* Watch failures after enabling a new provider installation.
* Use accepted recommendation rates to decide which workflows to expand.
* Treat stale outputs seriously when they affect release decisions.
* Keep scorecards tenant-scoped so one organization's data does not affect another.
* Use feedback to prioritize agent hardening work.