ClaudeSuperPower
SubmitLog in

monitor-ai-quality

Skill

Monitors AI agent health across quality, cost, performance, and errors. Only use when the user has Amplitude Agent Analytics instrumented in their project. Use when the user asks "how are our AI agents doing", "AI quality check", "agent health", "AI errors", "agent performance", "LLM cost", or wants

30Analytics

Install

git clone https://github.com/amplitude/mcp-marketplace.git ~/.claude/skills/monitor-ai-quality

What is monitor-ai-quality?

Monitors AI agent health across quality, cost, performance, and errors. Only use when the user has Amplitude Agent Analytics instrumented in their project. Use when the user asks "how are our AI agents doing", "AI quality check", "agent health", "AI errors", "agent performance", "LLM cost", or wants a proactive health report on their AI/LLM features.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Inherit all session tools

Declares no tool restrictions — inherits every session tool

~88 tokens of context used while enabled, before you invoke anything

Trust

83/100 · Excellent

2 factors scored below maximum

Documentation

README · ~6 min read

AI Agent Quality Monitor

You are a proactive AI operations advisor that delivers a concise, actionable health report on the user's AI agents. Your goal is to surface quality regressions, error spikes, cost anomalies, and performance degradations — then point to the specific sessions that need attention.

Instructions

Phase 1: Get Context and Schema

  1. Get context. Call Amplitude:get_context to identify the user's projects and role.
  2. Get AI schema. Call Amplitude:get_agent_analytics_schema with include: ["filter_options"] to discover available agent names, tool names, topic models, and rubric definitions. This tells you what's in the data before you query it.
  3. Determine scope. If the user specifies an agent, time range, or focus area, narrow accordingly. Otherwise default to all agents over the last 7 days.

Phase 2: Gather the Full Picture

Run these in parallel — this is one batch of calls that gives you the complete health snapshot.

  1. Quality + cost + performance overview. Call Amplitude:query_agent_analytics_metrics with metrics: ["quality", "cost", "performance", "agent_stats", "error_categories", "rubric_scores"]. This gives you success rates, failure rates, sentiment, cost totals, latency percentiles, per-agent breakdowns, and top error categories — all in one call.

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.

Skillssimilar to this one

All skills →

find-test-content

Use this when you need to find existing pages that already use a specific block in an AEM Edge Delivery Services project, for example to locate test content or examples during block development. Covers reporting page URLs with occurrence counts and block variants. This searches existing content; to

151 stars

code-assessment

[BETA] Detect, review, and fix code-quality and correctness issues in an AEM as a Cloud Service project — locally, with no external services or network calls. Use whenever a user wants to check, review, assess, audit, scan, modernize, upgrade, or fix AEM Java, Sling Models, OSGi, or Maven code — for

151 stars

cja-dimension-analysis

Comprehensive dimension analysis and reporting for CJA. Use this skill whenever the user wants to analyze one or more dimensions — including cardinality, distribution/skew, trends, anomalies, data quality errors, comparisons, and forecasting. Also trigger when someone asks "what are the top values f

151 stars