Rag Pipeline Builder
Design and optimize end-to-end RAG pipelines with intelligent document retrieval
What You Can Do
You'll design complete RAG architectures from ingestion through inference, making strategic decisions on chunking strategies, embedding models, vector database selection, and retrieval tuning. This skill guides you through all five core pipeline phases—document preprocessing, semantic embedding, vector storage, ranking optimization, and context window assembly—ensuring your system balances accuracy, latency, and cost for domain-specific knowledge retrieval at scale.
Features
Design semantic-aware splitting with overlap, handling different document types (PDFs, markdown, structured data)
Choose between dense embeddings (OpenAI, Cohere, open-source) based on cost, latency, and domain fit
Evaluate and configure PostgreSQL pgvector, Pinecone, Weaviate, Chroma, or Milvus for scalable retrieval
Implement hybrid search, BM25 + semantic fusion, re-ranking, and metadata filtering to maximize relevance
Intelligently pack retrieved chunks within token limits while preserving semantic coherence
Deploy via LangChain, LlamaIndex, or custom Python implementations with minimal scaffolding
Design metrics for retrieval quality (NDCG, MRR) and end-to-end answer accuracy
Example Output
Example 1: E-commerce Product Support RAG
- Chunking: 256-token sections from product manuals with 64-token overlap, preserving section headers
- Embedding: OpenAI text-embedding-3-small for cost efficiency
- Vector Store: Pinecone with product category metadata filtering
- Retrieval: Hybrid (BM25 + semantic) with MMR reranking for diversity
- Output: Customer service bot retrieving exact product specs with 94% accuracy on FAQ queries
Example 2: Legal Document Discovery System
- Chunking: Clause-aware segmentation respecting legal document structure
- Embedding: Self-hosted sentence-transformers for data sovereignty
- Vector Store: PostgreSQL pgvector for on-premise deployment
- Retrieval: Dense passage retrieval + legal metadata filters (document type, jurisdiction, date range)
- Output: Paralegal tool surfacing relevant precedents in 200ms with full citation trails
What's Included
- RAG Pipeline Builder SKILL.md: Complete instruction framework for designing RAG systems
- Chunking Decision Matrix: Comparison of splitting strategies (semantic, fixed-size, recursive) with trade-offs
- Vector Store Comparison Checklist: Evaluation criteria for 6+ vector databases (cost, latency, scalability)
- Retrieval Tuning Workflow: Step-by-step process for testing embeddings, ranking strategies, and hybrid search
- Context Packing Template: Code patterns for intelligent document assembly within token windows
- RAG Evaluation Framework: Metrics and benchmarking approach for retrieval quality
Who It's For
- AI/ML Engineers — Building production RAG systems for customer-facing applications
- Data Scientists — Designing knowledge retrieval architectures for proprietary datasets
- LLM Product Managers — Making technical decisions on embedding and vector storage stack
- Enterprise Architects — Planning document intelligence solutions across organizational knowledge bases
- Prompt Engineers — Optimizing retrieval performance to ground AI responses in domain expertise
Best For
- Designing document-grounded Q&A systems (customer support, FAQ automation)
- Building legal/compliance document discovery tools
- Creating research or scientific paper recommendation engines
- Architecting enterprise knowledge management systems
- Optimizing existing RAG pipelines for speed, cost, or accuracy
- Evaluating and selecting appropriate vector databases and embedding models
You might also like
You can build production-grade vector search systems that retrieve semantically similar documents in sub-millisecond time using AgentDB's optimized HNSW indexing. The skill enables you to implement RAG pipelines, semantic search engines, and intelligent knowledge bases with configurable embedding dimensions, distance metrics (cosine, Euclidean, dot product), and similarity thresholds—all with built-in quantization and caching for massive performance gains.
by moonwell-fi
This skill transforms high-level feature requests into actionable implementation plans by conducting deep requirement interviews, analyzing existing codebases (or planning greenfield projects), and producing comprehensive handoff documents. It distinguishes between brownfield development (with existing code to integrate) and greenfield projects (starting from scratch), ensuring your technical strategy accounts for your actual constraints and patterns.
by moonwell-fi
Diagnose root causes of model performance failures, select optimal evaluation metrics based on your use case and data characteristics, and design comprehensive evaluation strategies. You'll get systematic workflows for analyzing failure modes, statistical significance testing, threshold optimization, and cross-validation planning—turning vague model problems into actionable fixes.
by moonwell-fi
This skill helps you systematically diagnose failures in distributed ML training and serving infrastructure. You provide system logs, metrics, and error traces, and Claude performs structured root-cause analysis to identify the underlying issue—whether it's resource exhaustion, distributed system deadlock, data pipeline corruption, or model serving misconfiguration. You get a detailed diagnosis with remediation steps ranked by likelihood and implementation effort.
by moonwell-fi
Systematically evaluate your Claude prompts using quantitative scoring frameworks to identify weaknesses in clarity, task alignment, and output quality. You'll receive detailed optimization recommendations with A/B testing guidance to improve prompt effectiveness and consistency across different use cases and input variations.
by moonwell-fi
You can systematically diagnose why your RAG system isn't returning relevant results and identify specific misalignments between queries and context. This skill walks you through analyzing retrieval quality, scoring context relevance, and optimizing prompt-retrieval alignment to improve your RAG accuracy. You'll get concrete, actionable recommendations for fixing bottlenecks in your pipeline.
by moonwell-fi
You can prepare your domain-specific datasets, train fine-tuned Claude models optimized for your use case, benchmark performance against base models, and deploy production-ready custom models. This skill automates the entire fine-tuning pipeline from raw data to live inference, complete with cost-benefit analysis and version control.
by moonwell-fi
You can confidently navigate complex statistical analyses by working through structured workflows that match your research question and data type to the right test. The skill guides you through assumption validation, helps you interpret results accurately without common pitfalls, and walks you through regression diagnostics with full transparency about what your findings actually mean.
by moonwell-fi