Api Data Extractor
Extract, transform, and load data from REST and GraphQL APIs with authentication and pagination
What You Can Do
You can pull structured data from any REST or GraphQL endpoint, automatically managing authentication (API keys, OAuth, Basic Auth), pagination across multiple pages, rate limit compliance, and transient failure retries. Claude transforms raw API responses into normalized, schema-validated datasets deduplicated and formatted for immediate downstream analysis or storage.
Features
automatically detects and handles both API types
supports API keys, OAuth tokens, Basic Auth, and custom headers
manages cursor-based, offset, and page-number pagination schemes
respects API quotas and implements exponential backoff retries
validates extracted data against your predefined data schemas
gracefully handles 429, 503, timeout, and network errors
flattens nested structures, renames fields, converts types, deduplicates
reports success rates, row counts, and failed requests with diagnostics
Example Output
Example 1: REST API extraction with pagination
Extracted 2,847 user records from /api/v1/users endpoint
- Applied pagination: 20 records/page × 143 pages
- Schema validation: 100% success (0 failures)
- Deduplicated: 12 duplicate entries removed
- Output: users_2024-01-15.json (2.3 MB)
Example 2: GraphQL transformation
Query: { users { id name email posts { id title } } }
Extracted 1,543 users with nested posts flattened
Transformation applied:
- Nested posts array → separate user_posts table
- email field renamed to contact_email
- created_at timestamp → ISO 8601 format
Output: users.csv (143 KB), user_posts.csv (856 KB)
Example 3: Rate-limited API with retries
Stripe API extraction (100 req/sec limit)
- Initial attempt: 3,200 requests over 45 seconds
- Rate limit hit: 429 responses at request 1,847
- Backoff retry: resumed after 30s delay
- Final success: 5,120 transaction records extracted
- Duration: 2 min 15 sec (with intelligent delays)
What's Included
- api-data-extractor SKILL.md: instruction file with activation signals and best practices
- REST API template: pre-built configuration for common REST endpoints with pagination patterns
- GraphQL query template: example queries and response transformation logic
- Authentication config checklist: secure credential handling patterns for API keys, OAuth, tokens
- Error handling reference: retry strategies, rate limit detection, and failure logging
- Schema validation framework: JSON schema templates for validating extracted datasets
Who It's For
- Backend engineers — building data pipelines and ETL workflows
- Data engineers — extracting datasets from SaaS APIs for warehouses
- API integration specialists — connecting third-party services and syncing data
- Product analysts — pulling user and usage data from analytics or business APIs
- DevOps/Site reliability engineers — automating data collection for monitoring and reporting
Best For
- Multi-page data extraction — handling pagination at scale
- Rate-limited API integrations — respecting quotas and implementing backoff logic
- Data transformation workflows — normalizing and flattening nested API responses
- Schema validation pipelines — ensuring data quality before storage or analysis
- Error-resilient batch jobs — recovering from transient failures and retrying intelligently
You might also like
You can accelerate your entire antenna design workflow—from initial concept to optimized specifications—by leveraging Claude's knowledge of electromagnetic theory, antenna types, and design trade-offs. Claude helps you parameterize designs, identify optimal configurations for your frequency band and application, and prepare detailed briefs for simulation and manufacturing. You'll generate design reviews, impedance-matching solutions, and validation checklists without context-switching to multiple tools.
Rapidly analyze production incidents by correlating observability signals—logs, metrics, traces, and events—to identify the true root cause instead of symptoms. You'll reconstruct incident timelines, quantify impact scope, and generate both immediate remediation steps and long-term prevention measures.
This skill guides you through designing Kalman filters for your specific state estimation and sensor fusion problems. You'll learn to diagnose filter divergence, tune covariance matrices (Q and R), and validate observability—turning mathematical theory into production-ready filters. The skill provides step-by-step workflows, diagnostic tools, and practical templates for common applications like GPS/IMU fusion, SLAM, and aircraft navigation.
Claude analyzes your code diffs and generates standardized commit messages that follow the Conventional Commits specification. The skill automatically determines the correct commit type, scope, and description based on the changes you've made, ensuring your messages are parseable by automation tools while remaining human-readable for code reviewers.
You'll transform high-level feature requirements into a complete development plan covering API design, frontend implementation, testing strategy, and documentation. The skill helps you identify dependencies, parallelize work across frontend and backend teams, and establish quality gates that ensure features ship bug-free with proper error handling and logging.
Transform schema and data changes into production-ready migration packages that protect data integrity and minimize downtime. You can handle column additions, renames, type conversions, and backfills using proven patterns like expand-contract and shadow tables. The skill generates explicit rollback scripts, validation checklists, and phased execution plans that keep applications stable throughout the migration.
You can design end-to-end NGS pipelines tailored to your experiment type (WGS, WES, RNA-seq, targeted), troubleshoot pipeline failures by analyzing QC metrics and logs, and optimize performance by fine-tuning parameters and identifying bottlenecks. This skill handles the full workflow from raw sequencing data quality assessment through variant interpretation and annotation.
You can rapidly design, optimize, and validate chromatographic and LC-MS/MS methods for analyzing drugs and metabolites in biological matrices. This skill guides you through analytical method development workflows—including parameter selection, sensitivity optimization, and regulatory strategy—while ensuring ICH Q2(R2) and FDA compliance throughout the development process.