Survey Psychometric Validation & Scale Development
Validate surveys with psychometric rigor and scientific precision
by omkamalSocial Sciences & PolicyPsychology (Applied)Industrial-Organizational PsychologistSurvey/AnalyticsWhat You Can Do
You design, validate, and optimize survey instruments using systematic psychometric analysis. The skill systematically evaluates item quality, tests reliability through Cronbach's alpha and test-retest correlations, assesses construct validity via factor analysis and convergent/discriminant testing, and generates data-driven recommendations for survey improvement. Whether building a new measurement scale or auditing an existing one, you gain scientific evidence that your survey measures what it claims.
Features
Calculate item difficulty, discrimination indices, item-total correlations, and inter-item relationships to identify high-performing and problematic questions
Compute Cronbach's alpha, McDonald's omega, test-retest reliability, and split-half consistency to quantify internal consistency and measurement stability
Perform exploratory and confirmatory factor analysis, evaluate model fit (CFI, RMSEA, TLI), and visualize factor loadings to validate theoretical structure
Test convergent validity with related measures, discriminant validity with unrelated constructs, and criterion validity against external outcomes
Analyze Likert scale performance, evaluate optimal response anchors, detect ceiling/floor effects, and assess whether 5-point vs 7-point scales work better
Identify items with low discrimination, high redundancy, or unclear wording that should be revised, removed, or reordered
Compare psychometric performance across survey formats, response scales, or item phrasings to determine which design performs best
Generate comprehensive psychometric reports with statistical summaries, interpretation, and publication-ready documentation of measurement quality
Example Output
Item Analysis Summary
| Item | Mean | SD | Discrimination | Item-Total r | Recommendation |
|---|---|---|---|---|---|
| Q1: I feel energized at work | 4.2 | 1.1 | 0.58 | 0.62 | Keep |
| Q3: My work feels meaningful | 3.8 | 1.3 | 0.71 | 0.69 | Strengthen—highest discriminator |
| Q7: Feedback is confusing | 2.1 | 1.4 | 0.12 | 0.28 | Revise—ambiguous wording |
Reliability Results
- Cronbach's alpha: 0.82 (95% CI: 0.78–0.86) — Acceptable internal consistency
- McDonald's omega: 0.84 — Reliable latent variable measurement
- Test-retest (4-week): r = 0.76, p < .001 — Stable over time
Factor Structure
Confirmatory Factor Analysis (Single Factor Model):
- CFI = 0.91 (Acceptable; >0.90 threshold)
- RMSEA = 0.068 (Good fit; <0.08)
- TLI = 0.88
Factor Loadings (all p < .001):
- Q1: 0.73 | Q3: 0.81 | Q5: 0.77 | Q9: 0.69
Key Recommendations
✓ Remove Q7 (low discrimination, confusing wording)
✓ Revise Q2 and Q6 (item-total r < 0.40)
✓ Reorder items: place highest-loading items (Q3, Q5) earlier to establish clarity
✓ Consider 7-point scale: pilot data suggests broader item spread would improve discrimination
What's Included
- Item Performance Dashboard: Complete statistics for every survey item including means, standard deviations, discrimination, and correlations with total score
- Reliability Analysis Suite: Multiple reliability methods (Cronbach's alpha, omega, test-retest, split-half) with confidence intervals and interpretation guidelines
- Factor Analysis Workflow: Exploratory and confirmatory factor analysis templates with guidance on determining optimal number of factors and evaluating model fit
- Validity Evidence Framework: Systematic approach to assess convergent, discriminant, and criterion validity against related and unrelated measures
- Optimization Recommendations: Data-driven suggestions for which items to remove, revise, reorder, or strengthen based on statistical performance
- Publication-Ready Report Template: Structured format for psychometric findings with APA-style tables, statistical interpretation, and discussion of measurement quality
Who It's For
- Research psychologists designing personality, ability, or clinical assessment scales
- Survey researchers validating measurement instruments before large-scale deployment
- UX/Product researchers developing usability scales and satisfaction surveys
- HR professionals creating employee engagement, culture, or competency assessments
- Educational researchers building learning outcome or teaching effectiveness measures
Best For
- Validating new survey instruments before publication or deployment
- Analyzing existing survey data for reliability and quality issues
- Comparing psychometric performance across alternative survey designs
- Optimizing items based on statistical performance and theoretical fit
- Generating evidence of construct validity for research papers or reports
You might also like
You can design and validate selection systems that maximize predictive accuracy while minimizing legal risk. The skill applies industrial-organizational psychology principles to create structured assessments, job analyses, and validity evidence that withstand regulatory scrutiny. You'll generate documentation that demonstrates how your selection criteria connect to job performance and comply with EEOC/OFCCP standards.
by omkamal
Systematically integrate psychological and psychiatric assessment results into comprehensive, legally-sound forensic competency opinions. Claude guides you through evidence-based reasoning to address the specific legal standards applicable to your jurisdiction, helping you organize clinical findings into clear, defensible conclusions about cognitive, functional, and volitional capacities relevant to the legal question at hand.
by omkamal
This skill transforms raw clinical findings and assessment data into structured, legally defensible competency evaluation reports. You provide clinical observations, test scores, and relevant history; Claude organizes them into coherent narratives with proper test cross-referencing, expert reasoning frameworks, and standards-compliant conclusions that withstand legal scrutiny.
by omkamal
Build empirically validated competency frameworks and selection assessments grounded in structured job analysis and industrial-organizational psychology principles. Create assessment blueprints with criterion-related validity evidence, competency models aligned to business strategy, and comprehensive documentation that withstands legal scrutiny. Generate adverse impact analyses and reliability metrics to ensure your selection systems are both defensible and fair.
by omkamal
This skill provides comprehensive human factors analysis of vehicle interface designs. You can assess cognitive load, identify driver distraction risks, evaluate NHTSA compliance, and uncover usability issues before design freeze. It combines HCI best practices, automotive safety standards, and workload assessment methodologies to deliver actionable feedback that keeps drivers safe.
by omkamal
Systematically assess automotive human-machine interfaces against usability standards, safety frameworks (ISO 26262, SOTIF), and accessibility guidelines. You'll identify cognitive load risks, driver distraction hazards, and compliance gaps—then receive prioritized remediation recommendations backed by safety impact analysis. The skill works from design descriptions, wireframes, or prototypes to produce comprehensive evaluation reports.
by omkamal
You can systematically analyze aerospace accidents and incidents using evidence-based human factors frameworks like STAMP, SHEL, and Five Whys to identify systemic contributors beyond individual error. The skill pinpoints design flaws, procedural gaps, and organizational factors that created the accident pathway. You'll receive structured findings and specific, actionable interventions for design improvements or procedural changes to prevent recurrence.
by omkamal
You can systematically architect training programs grounded in industrial-organizational psychology principles. This skill guides you through organizational needs analysis, competency framework development, learning objective design, and program evaluation strategy. You'll produce comprehensive training blueprints that align development with business outcomes and job performance requirements.
by omkamal