Experiment Designer
Design statistically valid experiments with hypothesis testing and sample size calculations
What You Can Do
You can design experiments that meet statistical validity standards by defining clear null and alternative hypotheses, calculating required sample sizes based on effect sizes and power, identifying confounding variables to control, and pre-specifying analysis plans that prevent p-hacking and bias. This skill produces complete experiment designs combining scientific rigor with practical feasibility for product testing, marketing validation, and research studies.
Features
creates null and alternative hypotheses with clear statistical framing
determines required participants using effect size, power, and significance levels
maps variables that could bias results and specifies control methods
designs assignment schemes that ensure unbiased group allocation
structures primary and secondary outcomes before data collection to prevent selective reporting
justifies sample sizes and provides confidence in effect detection
estimates execution timeline and required resources for feasibility assessment
Example Output
Example 1: Product A/B Test
- Hypothesis: Adding social proof to checkout increases conversion by 8% (effect size)
- Sample size: 2,400 users per variant (n=4,800 total) to achieve 80% power at 5% significance
- Confounders controlled: device type, traffic source, time of day
- Primary metric: Checkout completion rate; Secondary: revenue per user
- Timeline: 2-week test window
Example 2: Marketing Campaign Study
- Null hypothesis: Email subject line A and B have equal open rates
- Calculation: 6,500 recipients per variant to detect 2% lift with 90% power
- Randomization: Random split at subscriber level, stratified by engagement tier
- Analysis plan: Chi-square test on primary outcome; subgroup analysis by user tenure as pre-specified secondary outcome
What's Included
- SKILL.md instruction file with full experiment design methodology:
- Hypothesis statement template: structured format for null/alternative hypotheses with effect size specifications
- Sample size calculation worksheet: interactive framework with power, significance, and effect size inputs
- Confounder control checklist: systematic list of variables to identify and control strategies by experiment type
- Pre-analysis plan template: captures hypotheses, metrics, subgroups, and analysis procedures before data collection
- Power analysis documentation guide: explains trade-offs between sample size, power, and effect magnitude
Who It's For
- Product managers — designing A/B tests for feature launches and user experience optimization
- Research scientists — planning rigorous studies with statistical validity and reproducibility
- Data scientists — structuring experiments that prevent bias and support causal inference
- Marketing analysts — validating campaign strategies with appropriate sample sizes and controls
- UX researchers — designing behavioral experiments with confounding variable management
Best For
- A/B tests for product features, checkout flows, and user interface changes
- Marketing campaign validation and channel comparison studies
- Randomized controlled trials and quasi-experimental designs
- Sample size justification for funding applications and research proposals
- Pre-analysis plan documentation for open science and reproducibility standards
You might also like
This skill parses experiment metadata and outputs to extract critical metrics from completed machine learning experiments. It reads experiment_summary.yaml to identify runs, pulls final training loss from SLURM logs, extracts accuracy metrics from inspect-ai evaluation files, and generates a clean summary.md report—all while logging the extraction process for reproducibility.
by kylezantos
You can systematically implement, audit, and sustain 5S lean methodology across your manufacturing operations. Claude generates customized audit templates, tracks compliance metrics, creates implementation roadmaps, and produces detailed audit reports—all aligned to your facility's specific processes and standards.
by kylezantos
Analyze production line data to identify bottlenecks, defect patterns, and root causes affecting TV manufacturing operations. Generate detailed corrective action plans with specific interventions, timelines, and expected impact. Transform raw production data into actionable insights for operations teams to improve yield, reduce downtime, and accelerate throughput.
by kylezantos
You can structure and run Kaizen events from start to finish, guiding your team through waste analysis, root-cause identification, and rapid improvement design. The skill generates detailed event agendas, countermeasure documentation, and performance metrics—enabling you to close improvement opportunities within days rather than weeks.
by kylezantos
You can conduct comprehensive literature reviews by simultaneously querying PubMed, arXiv, bioRxiv, and Semantic Scholar, then synthesizing the results into a coherent narrative organized by theme or research question. Claude verifies all citations for accuracy and generates publication-ready markdown and PDF documents with your choice of citation format, complete with PRISMA flow diagrams and research gap analysis.
by kylezantos
This skill applies structured decision logic to classify, prioritize, and route manufacturing or post-production defects with consistency and evidence-based reasoning. You provide defect details, and Claude triages them into severity categories, recommends dispositions (scrap, rework, accept-as-is, hold), and documents the decision logic so your quality team can audit, adjust rules, and track trends over time.
by kylezantos
You can systematically identify root causes of production problems using proven troubleshooting frameworks, decision trees, and rapid diagnosis protocols. This skill walks you through impact assessment, root cause analysis, and solution validation to minimize downtime and prevent recurring issues in manufacturing and operations environments.
by kylezantos
This skill guides you through systematic root cause analysis for manufacturing defects, helping you move beyond surface symptoms to uncover underlying failure mechanisms. You'll build investigation workflows, interpret production data, develop corrective actions backed by evidence-based reasoning, and prevent recurrence through structured follow-up.
by kylezantos