AMD Developer Hackathon · Track 2

Built for intelligent understanding

A 9-step agentic AI pipeline that samples, transcribes, analyzes, critiques, and optimizes captions for high-impact social clips.

Core Capabilities
Zone-Based Frame Extraction
Intelligently samples 6 key frames across 3 temporal zones (2 Beginning, 2 Middle, 2 Ending) to capture the visual story arc.
Speech-to-Text Transcription
Extracts the audio track and transcribes dialogue using Groq Whisper, applying post-filtering to ignore transcription noise.
Scene Intelligence Parsing
Identifies Scene Type, Mood, Distilled Narrative Story, Emotion, Irony, and builds list tags for high-engagement elements.
5 Distinct Caption Styles
Writes Formal, Sarcastic, Tech Humor, Life Humor, and Social Media captions tailored for specific audiences.
Gemma Critic Evaluation
Analyzes copy using Gemma to evaluate style authenticity, creativity, strength, and human likeness.
Direct YouTube Downloading
Extracts and downloads content directly from YouTube URLs (including Shorts) up to a 2-minute processing ceiling.
The 9-Step Pipeline Timeline
1
Download Video
Downloads assets securely from direct links, YouTube sources, or raw uploads.
2
Extract 6 Zone Frames
Grabs key visual moments from Beginning, Middle, and Ending zones.
3
Extract Audio
Isolates the raw audio stream to prepare for voice processing.
4
Transcribe Audio
Whisper model transcribes dialogue and strips background noise.
5
Scene Intelligence
Gemini Flash parses images and dialogue to extract scene hooks.
6
Caption Strategy
Builds a target audience style brief detailing focus, contrast, and goals.
7
Generate Drafts
Generates 5 distinct style variations in parallel using Gemini.
8
Gemma Editorial Review
Gemma scores draft quality. Triggers individual rewrites for drafts scoring < 8.5.
9
Refine & Finalize
Normalizes grades and passes final strategic captions to the dashboard.
Pipeline Tech Stack
Gemini Flash Gemma 4 Groq Whisper FastAPI ffmpeg yt-dlp Docker