RAG Optimization Engineer
Optimize RAG pipelines for retrieval quality and accuracy
What You Can Do
Debug retrieval failures, optimize chunking strategies, and validate answer accuracy across different RAG configurations. You can systematically test embedding models, chunk sizes, query expansion techniques, and reranking parameters to maximize your pipeline's performance. This skill helps you measure the impact of changes and identify which retrieval bottlenecks are hurting your system most.
Features
Analyze which documents are retrieved for each query and why they rank in that order
Test different chunk sizes, overlap, and splitting strategies to maximize relevance
Evaluate embedding model performance and identify relevance scoring issues
Experiment with query rewriting and expansion to improve recall
Systematically compare multiple RAG parameter sets with accuracy scores
Generate hit rate, precision@k, relevance scores, and other key metrics
Trace why specific queries fail and recommend targeted fixes
Compare answer accuracy before and after optimization changes
Example Output
Retrieval Quality Report
Current Metrics:
- Hit Rate (top-5): 72%
- Precision@1: 45%
- Avg Relevance Score: 0.68
Failing Queries Analysis:
- 15% fail due to poor embedding similarity
- 8% fail due to suboptimal chunk boundaries
- 5% fail due to insufficient query expansion
Configuration Comparison
| Strategy | Hit Rate | Precision@1 | Impact |
|---|---|---|---|
| Current (512 chunks) | 72% | 45% | Baseline |
| Test A (1024 chunks) | 78% | 52% | +20% tokens |
| Test B (Query expansion) | 81% | 58% | +15% latency |
| Test A+B (Combined) | 84% | 63% | +32% tokens |
Recommendation: Implement combined approach for 18 percentage-point improvement in Precision@1.
What's Included
- SKILL.md: Complete RAG optimization workflow with diagnostic trees and validation checklists
- RAG Diagnostic Template: Structured format for capturing your pipeline architecture and current metrics
- Configuration Comparison Worksheet: Side-by-side testing framework for parameter combinations
- Query Expansion Checklist: Techniques for rewriting and expanding queries to improve recall
- Retrieval Validation Suite: Test framework with accuracy scoring and metric calculation
- Troubleshooting Guide: Common RAG failure patterns and their root causes
Who It's For
- ML Engineers building and tuning RAG applications for production
- Data Scientists optimizing search and retrieval systems for accuracy
- LLM Product Managers responsible for Q&A accuracy and user satisfaction
- AI/ML Architects designing enterprise information retrieval systems
- Search Engineers debugging why queries return irrelevant results
Best For
- Diagnosing retrieval failures in underperforming RAG pipelines
- Comparing chunking strategies (size, overlap, splitting methods)
- Validating embedding model upgrades before production deployment
- Root cause analysis of queries that return irrelevant documents
- Optimizing cost vs. accuracy tradeoffs by testing parameter combinations
You might also like
You can design culturally appropriate survey instruments tailored to international development contexts, analyze mixed-methods data by integrating quantitative findings with qualitative insights, and implement robust data quality management systems for your field operations. Claude helps you create survey templates with adaptive logic, develop validation protocols, and produce data quality reports that highlight issues before analysis begins.
by skillslib
Prepare comprehensive expert witness reports and testimony that meet Daubert standards for admissibility and withstand cross-examination scrutiny. You'll get structured case analyses integrating psychological assessments, relevant case law citations, and anticipated counterarguments. The skill helps you document your forensic opinions with the legal rigor required in criminal, civil, and family law proceedings.
by skillslib
You can systematically evaluate architectural options by building decision matrices, identifying trade-offs, and documenting design rationale. This skill generates comparison frameworks, risk assessments, and architecture decision records (ADRs) that help you choose between competing approaches based on concrete criteria like performance, scalability, cost, and team capabilities.
by skillslib
You provide a single core insight or perspective, and this skill transforms it into a complete, coordinated thought leadership content suite across every major channel. It generates optimized articles, social threads, email sequences, presentation outlines, and more—all reinforcing the same core message while adapting to each platform's unique audience and format. Every piece maintains strategic consistency while maximizing reach and engagement.
by skillslib
Analyze commercial insurance applications by extracting risk factors, evaluating loss history and industry exposure, and generating structured underwriting recommendations with standardized documentation. You'll reduce review time per application, ensure consistent decision criteria across your team, and maintain compliant audit trails for every underwriting action.
by skillslib
You can convert raw discovery call notes into polished, multi-section sales proposals that include executive summaries tailored to buyer personas, needs analysis mapped to your solution architecture, tiered pricing options with competitive positioning, ROI projections with financial justification, and realistic implementation timelines. The skill preserves personalization and strategic positioning while automating structural components, letting you respond to RFPs and time-sensitive opportunities without sacrificing quality or consistency.
by skillslib
You can build production-grade vector search systems that retrieve semantically similar documents in sub-millisecond time using AgentDB's optimized HNSW indexing. The skill enables you to implement RAG pipelines, semantic search engines, and intelligent knowledge bases with configurable embedding dimensions, distance metrics (cosine, Euclidean, dot product), and similarity thresholds—all with built-in quantization and caching for massive performance gains.
by moonwell-fi
Turn project descriptions, timelines, and structured data into professional draw.io diagrams instantly. You can generate flowcharts, org charts, Gantt charts, swimlane diagrams, and process flows by describing what you need. Claude creates draw.io XML that you can open directly in the editor or modify further.
by skillslib