Test Flakiness
Detect flaky tests by analyzing CI logs and pass rate history
What You Can Do
Analyze CI run logs and test result history to detect tests that fail non-deterministically, calculate per-test pass rates across multiple runs, and distinguish between genuinely broken tests and intermittent failures. You get actionable recommendations on whether to quarantine or fix each flaky test, plus an updated registry to prevent masking real failures.
Features
automatically detects your CI platform
Example Output
Input: CI logs from 15 recent runs
Output:
## Flaky Tests Detected (Pass Rate < 95%)
### High Priority (40-80% pass rate)
- `test_player_spawn_collision` — 65% pass rate — Likely: race condition in physics engine initialization
- **Recommendation:** Quarantine immediately, add synchronization to test setup
- **Severity:** Critical (masks real failures)
### Medium Priority (80-95% pass rate)
- `test_ui_animation_complete` — 92% pass rate — Likely: timing-dependent, platform-specific
- **Recommendation:** Increase wait timeout or add retry logic
- **Severity:** Moderate
What's Included
- `test-flakiness` SKILL.md instruction file with multi-platform CI log parsing:
- Flakiness detection algorithm with statistical thresholds:
- Quarantine registry template for regression-suite.md:
- Pass-rate calculation framework and failure pattern analyzer:
- Remediation recommendation matrix (quarantine vs. fix guidance):
Who It's For
- QA/Test Engineers — diagnose why CI is unreliable and decide quarantine vs. fix strategies
- Release Engineers — reduce false positives during release validation runs
- Software Engineers — identify which flaky tests to prioritize fixing during Polish phase
- Platform Teams — maintain test reliability and prevent test rot across CI/CD pipelines
- DevOps Engineers — monitor test suite health and flag systemic flakiness trends
Best For
- Detecting intermittent test failures across multiple CI runs without manual log review
- Quarantining known flaky tests to prevent masking real failures in CI
- Diagnosing root causes of test non-determinism (timing, race conditions, environment sensitivity)
- Building a historical registry of flaky tests and remediation status
- Polish phase regression testing when sufficient CI data has accumulated
You might also like
You can transform raw geotechnical field data into professional-grade site investigation reports that include soil classification using USCS standards, bearing capacity calculations, settlement predictions, and specific foundation design recommendations. The skill organizes stratigraphic information across multiple boring locations, cross-references laboratory test results, and produces defensible conclusions compliant with IBC, ASCE 7, and local building codes—suitable for building officials, structural engineers, and construction teams.
by az9713
You can conduct comprehensive safety impact assessments for any transportation design intervention—from intersection redesigns to speed reduction corridors. The skill guides you through hazard identification, risk quantification using established methodologies (conflict studies, crash prediction models), and evidence-based mitigation development. You'll produce defensible safety documentation suitable for audits, litigation, and project approvals.
by az9713
Quickly evaluate pavement design scenarios using AASHTO 1993 methodology to calculate required structural thickness for flexible (asphalt) and rigid (concrete) pavements. Analyze traffic loading by computing Equivalent Single Axle Loads (ESALs), assess subgrade conditions through CBR interpretation, and compare multiple design alternatives with different material combinations to optimize cost and performance. Generate preliminary design documentation with technical justification for client presentations and design reports.
by az9713
This skill cross-references your residential design specifications against jurisdiction-specific building codes (IBC, IRC, local amendments) to identify code violations, ambiguous details inspectors will challenge, and compliance documentation gaps. You can validate designs across multiple jurisdictions, evaluate design trade-offs (fire-rating vs. acoustic performance), and generate compliance narratives that support permit applications—all before formal submission.
by az9713
You can input room dimensions, functional requirements, and design constraints to receive optimized furniture layouts with documented traffic flow patterns and compliance verification. Claude analyzes competing spatial demands—usable area, sightlines, egress, zoning—and produces data-driven plans that satisfy functional, regulatory, and aesthetic objectives simultaneously. This accelerates initial space planning and helps identify conflicts before costly revisions.
by az9713
This skill deconstructs your writing samples to identify the specific patterns, personality markers, emotional range, language quirks, and formatting habits that make your voice uniquely yours. You'll receive a detailed voice DNA profile that includes your core communication style, signature phrases, emotional tone, formality level, and what you actively avoid—creating a blueprint that AI systems can use to replicate your authentic voice consistently across projects, platforms, and content types.
by az9713
Design earth retention systems by inputting soil properties, site geometry, and regulatory requirements. Claude calculates lateral earth pressures using Coulomb, Rankine, and Mononobe-Okabe methods, verifies factors of safety for internal and external stability, compares competing retention concepts (gravity walls, cantilever, anchored, sheet-pile, soil-nail), and generates cost-benefit analysis and regulatory documentation packages ready for permit submission.
by az9713
Query your local Screenpipe instance to retrieve screen recordings, audio transcriptions, UI element accessibility trees, keyboard/mouse input logs, and productivity analytics. You can search by keywords, filter by content type (audio, OCR, accessibility, input), set time ranges, and extract structured data about your applications, meetings, and work sessions without sending data to external servers.
by az9713