Test Flakiness
Detect flaky tests by analyzing CI logs and pass rate history
What You Can Do
Analyze CI run logs and test result history to detect tests that fail non-deterministically, calculate per-test pass rates across multiple runs, and distinguish between genuinely broken tests and intermittent failures. You get actionable recommendations on whether to quarantine or fix each flaky test, plus an updated registry to prevent masking real failures.
Features
automatically detects your CI platform
Example Output
Input: CI logs from 15 recent runs
Output:
## Flaky Tests Detected (Pass Rate < 95%)
### High Priority (40-80% pass rate)
- `test_player_spawn_collision` — 65% pass rate — Likely: race condition in physics engine initialization
- **Recommendation:** Quarantine immediately, add synchronization to test setup
- **Severity:** Critical (masks real failures)
### Medium Priority (80-95% pass rate)
- `test_ui_animation_complete` — 92% pass rate — Likely: timing-dependent, platform-specific
- **Recommendation:** Increase wait timeout or add retry logic
- **Severity:** Moderate
What's Included
- `test-flakiness` SKILL.md instruction file with multi-platform CI log parsing:
- Flakiness detection algorithm with statistical thresholds:
- Quarantine registry template for regression-suite.md:
- Pass-rate calculation framework and failure pattern analyzer:
- Remediation recommendation matrix (quarantine vs. fix guidance):
Who It's For
- QA/Test Engineers — diagnose why CI is unreliable and decide quarantine vs. fix strategies
- Release Engineers — reduce false positives during release validation runs
- Software Engineers — identify which flaky tests to prioritize fixing during Polish phase
- Platform Teams — maintain test reliability and prevent test rot across CI/CD pipelines
- DevOps Engineers — monitor test suite health and flag systemic flakiness trends
Best For
- Detecting intermittent test failures across multiple CI runs without manual log review
- Quarantining known flaky tests to prevent masking real failures in CI
- Diagnosing root causes of test non-determinism (timing, race conditions, environment sensitivity)
- Building a historical registry of flaky tests and remediation status
- Polish phase regression testing when sufficient CI data has accumulated
You might also like
You can rapidly synthesize disparate urban site information—demographics, infrastructure, ecology, regulations, and spatial typologies—into explicit design frameworks. Claude helps you extract actionable insights that directly inform concept development, phasing strategies, and design interventions, bridging the gap between site investigation and design synthesis by creating clear connections between existing conditions and design opportunities.
by az9713
You can input earth retention system parameters—wall type, soil properties, loading conditions, and geometry—and Claude will perform comprehensive stability analysis including bearing capacity, sliding resistance, and overturning checks. The skill generates professional engineering reports with calculated factors of safety, design narratives, and permit-ready documentation that aligns with ASCE 7, ACI 318, and local building code requirements across gravity walls, cantilever walls, soldier pile systems, sheet pile walls, and MSE walls.
by az9713
You can systematically analyze raw XYZ survey coordinates to identify meaningful patterns, validate data quality, and detect surveying errors before they impact design phases. This skill helps you interpret slope gradients, drainage patterns, and terrain complexity while generating constraint reports that communicate site limitations to design teams. You'll produce professional topographic analyses that reveal drainage zones, stability issues, and access constraints invisible in raw measurements.
by az9713
You can submit architectural design documents, floor plans, and project parameters to receive a comprehensive compliance analysis that cross-references IBC, local amendments, zoning ordinances, and ADA accessibility requirements. Claude identifies specific gaps, cites applicable code sections, explains why each gap matters, and provides concrete remediation strategies—all before formal design review submission when corrections are costly and schedule-impacting.
by az9713
You can accelerate HVAC design workflows by inputting building parameters—envelope characteristics, occupancy data, climate conditions, and existing systems—and receiving calculated heating/cooling loads, equipment sizing recommendations, and zone-by-zone analysis. Claude organizes complex building data into actionable design parameters and specification summaries for client review, permit submission, and professional validation against ASHRAE and ACCA standards.
by az9713
This skill enables you to rapidly produce structural analyses for timber members, including bending, shear, compression, and combined stress calculations. You can determine lateral load capacity for shear walls, design connections, evaluate code compliance against NDS standards, assess serviceability limits, and generate calculation frameworks suitable for permit submission and engineer stamping.
by az9713
You can synthesize complex environmental site assessment data into structured remediation recommendations that balance technical feasibility, regulatory compliance, and project economics. This skill guides you through contaminant pathway analysis, remedy alternative evaluation, and the development of defensible remedial action objectives (RAOs) and remedial design specifications suitable for regulatory submittals.
by az9713
Design earth retention systems by inputting soil properties, site geometry, and regulatory requirements. Claude calculates lateral earth pressures using Coulomb, Rankine, and Mononobe-Okabe methods, verifies factors of safety for internal and external stability, compares competing retention concepts (gravity walls, cantilever, anchored, sheet-pile, soil-nail), and generates cost-benefit analysis and regulatory documentation packages ready for permit submission.
by az9713