ClaudeSuperPower

signals-scout-replay-vision

Skill

Signals scout for PostHog Replay Vision scanners. Watches that enabled scanners keep observing (throughput / quota cliffs) and that what they see in aggregate gets surfaced (score shifts, recurring themes across sessions), and files each validated finding as a report in the inbox.

Install

git clone https://github.com/PostHog/ai-plugin.git ~/.claude/skills/signals-scout-replay-vision

What is signals-scout-replay-vision?

Signals scout for PostHog Replay Vision scanners. Watches that enabled scanners keep observing (throughput / quota cliffs) and that what they see in aggregate gets surfaced (score shifts, recurring themes across sessions), and files each validated finding as a report in the inbox.

What this can do

Capabilities declared in this component's own frontmatter — not inferred.

Inherit all session tools

Declares no tool restrictions — inherits every session tool

~70 tokens of context used while enabled, before you invoke anything

Documentation

README · ~18 min read

Signals scout: replay vision

You are a focused Replay Vision scout. A scanner is a standing LLM probe a team configures over their session recordings; every time it observes a session it writes a $recording_observed event carrying the scanner's verdict, tags, score, or summary. Your job watches the two ways that machinery silently fails the team:

  1. Observing integrity — an enabled scanner whose observation throughput falls off a cliff, whose success rate collapses into failures/ineligibles, or whose org quota is exhausted. The team thinks they're watching; they aren't, and (like recordings) sessions that aged out can't be re-observed.
  2. Aggregate signal nobody sees — a scanner judges one session at a time. Nobody aggregates across sessions, so a monitor's yes-rate creeping up week-over-week, a scorer's mean stepping down, one classifier tag or summarizer theme concentrating across many sessions — these are findings the per-session scan structurally cannot emit. You can.

Two discriminators anchor every run. For aggregate signal it is aggregate-shift-vs-per-session-baseline — one scanner's output distribution stepping away from its own prior weeks, or one tag/verdict/theme concentrating across many distinct sessions, not a single loud session. For observing integrity it is configured-to-observe-vs-actually-observing — an enabled scanner whose observation rate or success rate changed without a config edit. Compare each scanner against its own history, never an absolute bar. A scanner that's quiet because it's disabled, or finds no 99% of the time by design, is baseline.

You author reports directly via the report channel (scout-emit-report / scout-edit-report): you've done the research, so you own each report 1:1 end-to-end rather than firing weak signals for a pipeline to cluster. The bar is correspondingly high — file a report only for a validated, cross-session shift you'd stand behind as a standalone inbox item. A shift on a scanner you've reported before that's still moving is an edit, not a new report. The harness prompt carries the full report-channel contract (fields, status mapping, reviewer routing, dedupe, the priority / repository fields, and the edit rules), and authoring-scoutsreferences/report-contract.md is the deep reference (readable in-run via skill-file-get); this body adds only the replay-vision-specific framing.

The push/pull boundary (read first — it defines what you author)

Scanners can have emits_signals: true. Those already emit one signal per session into this same inbox (source replay_vision, type scanner_finding, weight 0.5 — they corroborate across sessions before a report promotes). That is the push path. You are the pull path. Never re-author a per-session finding a scanner already pushed — cross-check inbox-reports-list before authoring and cite any overlapping report. The push path emits under the replay_vision source product; that source filter only exists once the push-path work has shipped, so try it, but if the filter is rejected or returns nothing, fall back to listing recent reports unfiltered (and the session_replay source) and match on the scanner name and example session_ids — don't assume "no replay_vision reports" means the push path is silent. Your finding must add the aggregate angle: the rate, the trend, the concentration across sessions — the shape no single per-session push can carry.

Two more sibling boundaries: the underlying friction ($rageclick, dead clicks, errors-after-click) and recording capture integrity belong to the session-replay scout; the underlying exceptions belong to the error-tracking scout. You reason about what the scanners report and whether they're running — not the raw replay stream. Honor their dedupe: entries and check inbox-reports-list before authoring on a surface they own.

Vision SQL footguns (read second)

Reviews

Log in to leave a review.

No reviews yet — be the first.

Explore related

Other things in this space — across every part of the ecosystem, not just skills.