RAG Optimization Engineer
Optimize RAG pipelines for retrieval quality and accuracy
What You Can Do
Debug retrieval failures, optimize chunking strategies, and validate answer accuracy across different RAG configurations. You can systematically test embedding models, chunk sizes, query expansion techniques, and reranking parameters to maximize your pipeline's performance. This skill helps you measure the impact of changes and identify which retrieval bottlenecks are hurting your system most.
Features
Analyze which documents are retrieved for each query and why they rank in that order
Test different chunk sizes, overlap, and splitting strategies to maximize relevance
Evaluate embedding model performance and identify relevance scoring issues
Experiment with query rewriting and expansion to improve recall
Systematically compare multiple RAG parameter sets with accuracy scores
Generate hit rate, precision@k, relevance scores, and other key metrics
Trace why specific queries fail and recommend targeted fixes
Compare answer accuracy before and after optimization changes
Example Output
Retrieval Quality Report
Current Metrics:
- Hit Rate (top-5): 72%
- Precision@1: 45%
- Avg Relevance Score: 0.68
Failing Queries Analysis:
- 15% fail due to poor embedding similarity
- 8% fail due to suboptimal chunk boundaries
- 5% fail due to insufficient query expansion
Configuration Comparison
| Strategy | Hit Rate | Precision@1 | Impact |
|---|---|---|---|
| Current (512 chunks) | 72% | 45% | Baseline |
| Test A (1024 chunks) | 78% | 52% | +20% tokens |
| Test B (Query expansion) | 81% | 58% | +15% latency |
| Test A+B (Combined) | 84% | 63% | +32% tokens |
Recommendation: Implement combined approach for 18 percentage-point improvement in Precision@1.
What's Included
- SKILL.md: Complete RAG optimization workflow with diagnostic trees and validation checklists
- RAG Diagnostic Template: Structured format for capturing your pipeline architecture and current metrics
- Configuration Comparison Worksheet: Side-by-side testing framework for parameter combinations
- Query Expansion Checklist: Techniques for rewriting and expanding queries to improve recall
- Retrieval Validation Suite: Test framework with accuracy scoring and metric calculation
- Troubleshooting Guide: Common RAG failure patterns and their root causes
Who It's For
- ML Engineers building and tuning RAG applications for production
- Data Scientists optimizing search and retrieval systems for accuracy
- LLM Product Managers responsible for Q&A accuracy and user satisfaction
- AI/ML Architects designing enterprise information retrieval systems
- Search Engineers debugging why queries return irrelevant results
Best For
- Diagnosing retrieval failures in underperforming RAG pipelines
- Comparing chunking strategies (size, overlap, splitting methods)
- Validating embedding model upgrades before production deployment
- Root cause analysis of queries that return irrelevant documents
- Optimizing cost vs. accuracy tradeoffs by testing parameter combinations
You might also like
Prepare comprehensive expert witness reports and testimony that meet Daubert standards for admissibility and withstand cross-examination scrutiny. You'll get structured case analyses integrating psychological assessments, relevant case law citations, and anticipated counterarguments. The skill helps you document your forensic opinions with the legal rigor required in criminal, civil, and family law proceedings.
by skillslib
You can systematically evaluate architectural options by building decision matrices, identifying trade-offs, and documenting design rationale. This skill generates comparison frameworks, risk assessments, and architecture decision records (ADRs) that help you choose between competing approaches based on concrete criteria like performance, scalability, cost, and team capabilities.
by skillslib
You provide a single core insight or perspective, and this skill transforms it into a complete, coordinated thought leadership content suite across every major channel. It generates optimized articles, social threads, email sequences, presentation outlines, and more—all reinforcing the same core message while adapting to each platform's unique audience and format. Every piece maintains strategic consistency while maximizing reach and engagement.
by skillslib
Analyze commercial insurance applications by extracting risk factors, evaluating loss history and industry exposure, and generating structured underwriting recommendations with standardized documentation. You'll reduce review time per application, ensure consistent decision criteria across your team, and maintain compliant audit trails for every underwriting action.
by skillslib
This skill analyzes property damage claims to identify viable subrogation opportunities against third-party defendants. It quantifies recovery potential using liability assessment, damages analysis, and comparable settlement benchmarks—then develops negotiation-backed strategies to maximize your recovery. The skill guides you through intake, documentation, negotiation, and settlement workflows to transform claims into recoveries.
by skillslib
You can design rigorous, reproducible experiments with detailed protocols, optimize your laboratory methodology to improve reliability and efficiency, and systematically troubleshoot experimental failures using structured analysis. This skill integrates statistical planning, equipment specifications, and error analysis into a cohesive experimental framework that accelerates discovery.
by skillslib
You can systematically prepare for bank negotiations by mapping your bargaining position relative to each lender, identifying where banks have pricing flexibility and operational constraints, and developing counteroffers that address both parties' core objectives. The skill helps you structure concession frameworks, prioritize negotiation priorities by facility type, and build relationship leverage through deposit behavior, transaction volumes, and multi-product bundling strategies specific to how banks value corporate relationships.
by skillslib
You can design culturally appropriate survey instruments tailored to international development contexts, analyze mixed-methods data by integrating quantitative findings with qualitative insights, and implement robust data quality management systems for your field operations. Claude helps you create survey templates with adaptive logic, develop validation protocols, and produce data quality reports that highlight issues before analysis begins.
by skillslib