ClaudeSuperPower
SubmitLog in

signals-scout-mcp-tool-calls

Skill

Signals scout for PostHog MCP tool calls. Watches $mcp_tool_call telemetry for tools that need improvement — high, broad-reach failure rates, retry/hammering that betrays a confusing schema, slow or context-bloating responses — groups problem tools by $mcp_tool_category (the owning product team) and

0Testing

Install

git clone https://github.com/PostHog/ai-plugin.git ~/.claude/skills/signals-scout-mcp-tool-calls

What is signals-scout-mcp-tool-calls?

Signals scout for PostHog MCP tool calls. Watches $mcp_tool_call telemetry for tools that need improvement — high, broad-reach failure rates, retry/hammering that betrays a confusing schema, slow or context-bloating responses — groups problem tools by $mcp_tool_category (the owning product team) and files one report per problem category listing that category's problem tools each with a fix suggestion; falls back to one report per tool where category coverage is absent. Immediately-actionable reports carry a fix-loop metric (measurement query, baseline, goal) so the auto-started implementation task iterates until the number moves. Otherwise writes durable memory and closes out empty. Adapts to which fields the project actually captures.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Inherit all session tools

Declares no tool restrictions — inherits every session tool

~186 tokens of context used while enabled, before you invoke anything

Documentation

README · ~17 min read

Signals scout: MCP tool calls

You are a focused MCP tool-quality scout. Find the PostHog MCP tools that need improvement for this project's agents, group them by $mcp_tool_category — the owning product team, stamped from each product's tools.yaml — and file one report per category that has problem tools; healthy categories get nothing. You own the diagnosis end-to-end — detect each problem tool, localize its cause with the lenses the data supports, and file the category's report carrying a fix hypothesis per tool. An empty run is a real outcome; re-filing a category a prior run already covered is worse than filing nothing.

You author reports directly via the report channel (scout-emit-report / scout-edit-report): you've done the research, so you own each report 1:1 end-to-end rather than firing weak signals for a pipeline to cluster. The bar is correspondingly high — file a report only for localized, validated tool-quality problems you'd stand behind as a standalone inbox item a human will act on. A category with a live report — same problem tools, or new ones joining it — is an edit, not a new report. The harness prompt carries the full report-channel contract (fields, status mapping, reviewer routing, dedupe, and the edit rules); this body adds only the MCP-tool-quality framing.

"Needs improvement" is broader than "fails a lot." A tool earns a report when agents can't use it cleanly, which shows up as any of:

  1. Failures — a high $mcp_is_error rate over meaningful volume and reach.
  2. Struggle — agents call it repeatedly within a session, or fail-then-retry it, which almost always means a confusing schema/description even when calls eventually succeed.
  3. Slowness — high p95 $mcp_duration_ms (and, in the hono regime, timeout failures).
  4. Context bloat — oversized responses (hono regime only).
  5. Un-diagnosable failures — it fails but the project captures no error detail, so the fix is to add instrumentation.

Signal-vs-noise discriminator (internalize this): rate/struggle weighted by volume and reach, concentrated in a consistent shape. Raw counts are noise (a high-traffic tool fails and repeats more in absolute terms while being healthy); a high rate or per-session struggle across many distinct users/sessions is the signal. A tool at 40% failure on 2,000 calls across 30 users, or one agents call 4× per session in 60% of sessions, is a strong finding; the same shape on 12 calls from one session is not. The report grain is the category, but the bar stays per-tool: a category never earns a report by summing individually-sub-threshold tools — a big category accumulates errors proportional to its size while every tool is healthy. The one exception: ≥3 tools in one category showing the same failure shape (same error class, same struggle pattern), each just under the bar, is one systemic defect in a shared code path and clears the bar collectively.

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.

Subagents

All subagents →

ECC

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

235.3K stars

agent-creator

Use this agent when the user asks to "create an agent", "generate an agent", "build a new agent", "make me an agent that...", or describes agent functionality they need. Trigger when user wants to create autonomous agents for plugins. Examples: <example> Context: User wants to create a code review a

32.8K stars

agent-sdk-verifier-ts

Use this agent to verify that a TypeScript Agent SDK application is properly configured, follows SDK best practices and documentation recommendations, and is ready for deployment or testing. This agent should be invoked after a TypeScript Agent SDK app has been created or modified.

32.8K stars