RecipeReel

An AI video-to-recipe converter for cooking content

RecipeReel turns a cooking or tutorial video into one clean, structured recipe. Paste a YouTube, TikTok or Instagram Reels link, or upload a file. It returns:

ingredients with quantities and units
ordered steps, each tagged with the second it happens in the video
prep, cook and total times
equipment and tips
an estimated nutrition panel

The output comes as JSON or as schema.org/Recipe JSON-LD that a recipe site can publish directly.

We built it in three days for Track 3 of the AMD Developer Hackathon: ACT II. The code is MIT-licensed and public on GitHub. A live demo runs the full pipeline in mock mode.

Key features

Transcript and on-screen text fusion — narration gives the order of the steps and the technique. Vision and text recognition on keyframes give the exact amounts that creators show but don't say.
Timestamped steps — every step keeps its start time, so the web app can jump the embedded video to that moment.
Provenance on every field — each ingredient is marked as spoken, on screen or inferred. An unknown quantity stays null instead of being made up.
schema.org Recipe export — a dedicated endpoint returns JSON-LD with ingredients, HowToStep instructions, times, tools and nutrition.
Link or upload — links are downloaded with yt-dlp, and uploaded files go through the same pipeline.
Live progress — jobs run asynchronously and stream each stage over Server-Sent Events.
Bot protection without a third party — a self-hosted Altcha proof-of-work CAPTCHA guards the extraction endpoints. Its challenges are signed, expire and can be used only once.

Architecture and tech stack

Processing pipeline

Each job runs through eight stages:

Ingest the video.
Extract 16 kHz audio with FFmpeg.
Sample about 24 timestamped keyframes.
Transcribe the audio with Whisper-large-v3.
Read the frames with Qwen2.5-VL.
Synthesize one recipe with Gemma at temperature 0, constrained to a JSON schema.
Validate the result, normalize units and deduplicate ingredients.
Store the recipe in SQLite.

Speech and vision are the heavy, throughput-bound stages, so they are built to run on AMD Instinct MI300X GPUs through ROCm. ROCm PyTorch presents itself as CUDA, so the same code runs on a laptop CPU, in CI and on the GPU. A notebook runs both models on the MI300X.

Synthesis goes through one OpenAI-compatible client with interchangeable routes: Gemma 4 on Google AI Studio, a fast model on Fireworks AI, or Gemma self-hosted with vLLM. Switching between them is a config change. In one verified end-to-end run on a real YouTube Short, Gemini 2.5 Flash read the frames and Gemma 4 wrote the recipe.

Getting past YouTube in production

The live server runs on a datacenter IP, and YouTube blocked downloads from it. We tested datacenter and ISP proxies, then moved YouTube ingest to a hosted download API, keeping yt-dlp as the fallback. Raw extractor errors are replaced with a calm message suggesting a direct upload, so bot-block messages never reach users.

Web app

The frontend is a self-contained vanilla HTML, CSS and JavaScript app served by the API. Desktops get an editorial layout, and phones get a full-bleed app layout. It includes recipe detail, cook mode, a searchable library and an add-by-link flow that runs a real extraction job.

Quality

Every stage is a pluggable provider with a mock implementation, so the service always boots. Mock mode drives the real code path from bundled fixtures for demos and CI. The test suite has 24 passing tests, and slim and ROCm Docker images are included.

Why structured recipe data matters

A video-to-recipe converter is only as useful as its output. schema.org Recipe markup makes recipe pages eligible for Google's recipe rich results, and it's the shape meal planners and grocery integrations can consume. That's why RecipeReel produces standard structured data instead of a tidier transcript.

RecipeReel is part of our AI integration and automation work. LectureLoom applies the same pipeline to lecture videos. Legible, Testify and Keystone came out of the same hackathon.