Summarize Experiment
Extract and summarize experiment results into lightweight markdown reports
What You Can Do
This skill parses experiment metadata and outputs to extract critical metrics from completed machine learning experiments. It reads experiment_summary.yaml to identify runs, pulls final training loss from SLURM logs, extracts accuracy metrics from inspect-ai evaluation files, and generates a clean summary.md report—all while logging the extraction process for reproducibility.
Features
Example Output
Example summary.md output:
Experiment Summary: bert-classification-v2
Run Status
| Run Name | Type | Model | Status | Training Loss | Accuracy |
|---|---|---|---|---|---|
| run-001-lr0.001 | fine-tuned | bert-base | Completed | 0.245 | 0.924 |
| run-002-lr0.0001 | fine-tuned | bert-base | Completed | 0.198 | 0.931 |
| control-baseline | control | bert-base | Completed | — | 0.847 |
Key Findings
- Best accuracy: 0.931 (run-002-lr0.0001, learning rate 0.0001)
- Improvement over baseline: +8.4 percentage points
- Training converged in all runs
Log entry example:
[2024-01-15 14:32:01] Parsing experiment_summary.yaml...
[2024-01-15 14:32:02] Found 3 runs (2 fine-tuned, 1 control)
[2024-01-15 14:32:03] Extracted training loss from run-001: 0.245
[2024-01-15 14:32:04] Extracted accuracy from eval/logs/run-002.eval: 0.931
[2024-01-15 14:32:05] summary.md generated successfully
What's Included
- `summarize-experiment.md` instruction file with full workflow documentation:
- YAML parsing template for reading experiment_summary.yaml structures:
- Python extraction script (parse_eval_log.py) for inspect-ai eval file parsing:
- summary.md template with markdown table and findings sections:
- SLURM log extraction patterns and regular expressions:
Who It's For
- Machine learning researchers conducting hyperparameter tuning experiments
- ML engineers documenting model fine-tuning results for team review
- Research scientists tracking multiple experimental runs for publications
- AI labs automating experiment result documentation workflows
- Data scientists creating reproducible experiment reports
Best For
- Summarizing multi-run fine-tuning experiments with varied hyperparameters
- Extracting metrics from completed inspect-ai evaluation workflows
- Generating quick reference documents comparing control vs. fine-tuned models
- Automating post-experiment documentation after run-experiment completion
- Creating audit trails of experiment execution and metric extraction
You might also like
You can systematically implement, audit, and sustain 5S lean methodology across your manufacturing operations. Claude generates customized audit templates, tracks compliance metrics, creates implementation roadmaps, and produces detailed audit reports—all aligned to your facility's specific processes and standards.
by kylezantos
Analyze production line data to identify bottlenecks, defect patterns, and root causes affecting TV manufacturing operations. Generate detailed corrective action plans with specific interventions, timelines, and expected impact. Transform raw production data into actionable insights for operations teams to improve yield, reduce downtime, and accelerate throughput.
by kylezantos
You can structure and run Kaizen events from start to finish, guiding your team through waste analysis, root-cause identification, and rapid improvement design. The skill generates detailed event agendas, countermeasure documentation, and performance metrics—enabling you to close improvement opportunities within days rather than weeks.
by kylezantos
This skill applies structured decision logic to classify, prioritize, and route manufacturing or post-production defects with consistency and evidence-based reasoning. You provide defect details, and Claude triages them into severity categories, recommends dispositions (scrap, rework, accept-as-is, hold), and documents the decision logic so your quality team can audit, adjust rules, and track trends over time.
by kylezantos
You can systematically identify root causes of production problems using proven troubleshooting frameworks, decision trees, and rapid diagnosis protocols. This skill walks you through impact assessment, root cause analysis, and solution validation to minimize downtime and prevent recurring issues in manufacturing and operations environments.
by kylezantos
You can diagnose root causes of yield loss, identify process drift in real time, and optimize critical parameters using structured troubleshooting frameworks. The skill helps you prioritize corrective actions based on statistical analysis, accelerate problem resolution, and build repeatable optimization workflows that your team can execute consistently across production runs.
by kylezantos
This skill structures and accelerates improvement initiatives using the PDCA (Plan-Do-Check-Act) framework combined with 5 Why analysis and standardized work templates. You'll build data-driven problem statements, identify root causes systematically, and create actionable improvement plans with built-in controls. It enables teams to move from reactive firefighting to proactive, continuous optimization grounded in evidence.
by kylezantos
This skill guides you through systematic root cause analysis for manufacturing defects, helping you move beyond surface symptoms to uncover underlying failure mechanisms. You'll build investigation workflows, interpret production data, develop corrective actions backed by evidence-based reasoning, and prevent recurrence through structured follow-up.
by kylezantos