Video Understand
Extract frames and transcribe video locally with ffmpeg and Whisper
What You Can Do
You can intelligently extract representative frames from videos using scene detection, keyframe analysis, or regular intervals, while simultaneously transcribing audio to text using OpenAI's Whisper model. This skill processes videos entirely offline, giving you frame-by-frame visual context and complete transcripts without relying on external APIs or cloud services.
Features
automatically identifies shot changes and extracts frames at transitions
isolates the most important frames based on visual content analysis
captures frames at regular time intervals for consistent sampling
converts video audio to text using offline speech recognition
control maximum frames extracted to balance detail vs. file size
choose from tiny to large models based on accuracy needs
extract visual content without transcription for faster processing
structured metadata with timestamps, frame paths, and transcription data
Example Output
Scene Detection with Transcription
{
"video": "interview.mp4",
"duration_seconds": 180,
"frames_extracted": 12,
"frames": [
{"timestamp": 0.0, "mode": "scene", "path": "frame_0000.jpg"},
{"timestamp": 15.3, "mode": "scene", "path": "frame_0001.jpg"}
],
"transcription": {
"full_text": "Welcome to the interview. Today we're discussing video production techniques...",
"segments": [
{"start": 0.0, "end": 5.2, "text": "Welcome to the interview."},
{"start": 5.2, "end": 12.1, "text": "Today we're discussing video production techniques..."}
]
}
}
Keyframe-Only Output
✓ Extracted 8 keyframes from video.mp4
✓ Identified critical visual moments
✓ Saved frames: frame_0000.jpg, frame_0001.jpg, ...
What's Included
- SKILL.md: complete setup and CLI reference
- understand_video.py: main script with scene detection, keyframe, and interval extraction modes
- Whisper integration: optional local transcription with configurable model sizes
- CLI flags & options: control frame limits, models, output formats, and processing modes
- JSON schema templates: structured output format for frame metadata and transcription segments
Who It's For
- Video editors — quickly understand video structure and content before detailed editing
- Content creators — extract key moments and auto-generate captions locally
- Researchers — analyze video datasets offline without API costs or privacy concerns
- Accessibility specialists — transcribe video audio to create subtitles and transcripts
- Post-production teams — preview shot sequences and identify editing points visually
Best For
- Transcribing video interviews and podcasts to searchable text
- Extracting thumbnail candidates and key frames for previews
- Analyzing shot composition and scene transitions
- Creating subtitle files from video audio tracks
- Cataloging video library content without external dependencies
You might also like
Upload your asset specifications, render bottlenecks, or performance metrics, and receive a comprehensive optimization roadmap. Claude analyzes polygon topology, texture memory footprint, shader complexity, and LOD strategies specific to your target engine—whether game engines, live VFX systems, or AR/VR pipelines. You'll get concrete recommendations for polycount reduction, UV layout optimization, texture atlasing, and bake-versus-real-time lighting decisions, complete with documentation for handing off to animation and rigging teams.
by dliangthinks
You'll convert narrative scripts into comprehensive shot-by-shot breakdowns that specify camera movement, framing, timing estimates, emotional beats, and editorial cues. The skill generates detailed visual specifications that serve directors, cinematographers, production teams, and editors—eliminating guesswork between creative intent and execution while reducing coverage waste and accelerating the editorial process.
by dliangthinks
This skill generates comprehensive audio design documentation that bridges creative vision and technical execution. You'll produce detailed mixing specifications with priority hierarchies, platform-specific audio budgets, middleware configurations for Wwise/FMOD, and implementation workflows tailored to your target engines (Unity, Unreal, custom). It accelerates pre-production planning, clarifies audio requirements for programmers, and reduces iteration cycles during integration.
by dliangthinks
You can transform raw video footage into structured, professional content by leveraging AI to identify key moments, organize clips, transcribe audio, and augment with voiceovers, overlays, and subtitles. This skill guides you through a proven pipeline that emphasizes compression and smart structuring rather than AI generation—turning hours of footage into minutes of compelling content optimized for any platform.
by dliangthinks
You can instantly transform session details, creative decisions, and technical parameters into standardized mix notes, delivery specifications, and editorial documentation. Claude generates episode-consistent technical specs for dialogue, music, and effects stems across broadcast, streaming, and theatrical formats—capturing your creative intent without manual transcription.
by dliangthinks
This skill accelerates your compositing pipeline by helping you plan multi-layer strategies before node construction, quickly diagnose rendering bottlenecks and technical failures, and structure complex workflows for keying, tracking, and rotoscoping. You get systematic guidance on layer organization, pass management, and optimization techniques that reduce iteration time and improve shot delivery without replacing your creative judgment.
by dliangthinks
You can generate production-ready audio documentation that bridges creative sound design and technical implementation. This skill produces asset inventories with precise technical specifications, engine-specific integration instructions for Unreal, Unity, Godot and proprietary engines, platform-aware optimization strategies for mobile/console/PC, and audio middleware configuration guides that development teams can directly implement without iteration cycles.
by dliangthinks
You can break down video content into individual frames and aligned audio segments at any interval you specify. This skill generates PNG images for visual analysis, extracts corresponding audio clips in AAC format, and creates a JSON manifest that maps every frame to its timestamp and audio file—perfect for preparing video content for detailed review, transcription workflows, or handing off to other agents for automated processing.
by dliangthinks