A montage that breathes with the music, written as code一支跟著音樂呼吸的剪輯，以程式寫成
Instead of scrubbing a timeline in an editor, the entire edit is a small program. The song is analysed for tempo, structure and “drops”; a director layer turns that analysis into a list of cuts, layouts and effects; ffmpeg renders it; and each round of feedback is a prompt, not a mouse.
The footage is deliberately meaningless — thirteen unrelated AI-generation test renders. The music supplies all the meaning. The system's job is to feel the track's rise and fall and cut to it: sparse and slow at the open, a relentless grid at the peak, a collapse at the end.
整支片子不是在剪輯軟體裡拖時間軸，而是一支小程式。先分析歌曲的速度、結構與「drop」；再由一層「導演」邏輯把分析轉成剪點、版面與特效清單；最後交給 ffmpeg 算圖。而每一輪修改，靠的是一句話，不是滑鼠。
素材刻意是「無意義」的 —— 十三段彼此無關的 AI 生成測試影片。意義全由音樂提供。系統的工作，是去感受歌曲的起伏並依此下刀：開場稀疏而緩慢，高潮處是密不透風的電視牆，結尾則歸於崩解。
Why edit by prompt at all為什麼要用對話來剪輯
The motivation was a workflow experiment: could a video edit be iterated as fast as code — describe a change in a sentence, see a render, refine — without opening an NLE? For a free-form, non-narrative piece where the goal is energy rather than story, that loop is appealing. There's no continuity to protect, so the machine is free to chop, mirror, stack and corrupt.
The track
Boards of Canada — “Age of Capricorn.” Dark, tape-saturated electronica with an occult undertow. It carries two found-audio samples that set the whole tone: a devotional chant (Paramhansa Yogananda's “Door of My Heart”) and a televangelist's apocalypse monologue (Jack Van Impe's 1990 “A.D. 2000: The End?”). Those two poles — prayer and doomsday broadcast — became the film's concept.
The footage
Thirteen clips, all byproducts of day-job design-technology work: test renders from AI video and generation tools (Veo, AnimateDiff, ReActor + ACE++, InfiniteTalk and others). No shared subject, no shared look — a knitted puppet, a woman on a giant chessboard, subway platforms, a segmentation-mapped figure, product UI. Treated purely as raw signal.
動機是一場工作流程實驗：一支影片能不能像寫程式一樣快速迭代 —— 用一句話描述修改、看算圖結果、再修 —— 而完全不打開剪輯軟體？對於一支自由、非敘事、追求「能量」而非「故事」的片子，這個循環格外吸引人。沒有連戲要顧，機器就能自由地切、鏡射、堆疊與破壞。
這首歌
Boards of Canada 的〈Age of Capricorn〉。陰暗、帶著磁帶飽和感的電子樂，暗藏神祕學的氣息。曲中有兩段取樣定調了全片：一段祈禱吟唱（Paramhansa Yogananda 的〈Door of My Heart〉），與一段電視佈道家的末日獨白（Jack Van Impe 1990 年的〈A.D. 2000: The End?〉）。祈禱與末日廣播這兩極，成了本片的概念核心。
這些素材
十三段影片，全是本職設計技術工作的副產品：來自 AI 影片與生成工具（Veo、AnimateDiff、ReActor + ACE++、InfiniteTalk 等）的測試輸出。沒有共同主題、沒有統一風格 —— 一隻針織玩偶、一位站在巨大棋盤上的女子、地鐵月台、被語意分割的人像、產品介面。它們純粹被當作「訊號」原料。
A liturgy playing on a dying television一場在瀕死電視上播送的儀式
The chant and the sermon suggested a rite. The song's dynamics fall into six movements, so the edit is structured as a liturgy in six acts: Invocation → Procession → Litany → Ascent → Sermon → Benediction. Everything is framed as a corrupted 1970s CRT broadcast — a hijacked transmission where the images surface, repeat and decay like a signal losing hold. The recurring motifs (the knitted idol, the red priestess, chess pieces) behave as “signal ghosts” the broadcast can't stop replaying.
The CRT is not a filter laid on top — it is the narrator. Screens are physical monitors on black; the frame is 4:3; the picture bends over glass, blooms, and rolls when the signal slips.
吟唱與佈道暗示著一場儀式。歌曲的動態剛好落成六個段落，於是整支剪輯被結構成一場六幕儀式：召喚 → 行進 → 連禱 → 上升 → 佈道 → 祝禱終場。一切都被框在一台被劫持的 1970 年代 CRT 廣播裡 —— 影像如同抓不住的訊號，浮現、重複、衰減。反覆出現的母題（針織偶像、紅衣祭司、西洋棋子）像是廣播停不下來、不斷重播的「訊號幽靈」。
CRT 不是疊在最上層的濾鏡 —— 它就是敘事者。畫面是黑底上的實體螢幕；畫幅為 4:3；影像在玻璃上彎曲、暈開，並在訊號失鎖時上下滾動。
The pipeline: analysis in, ffmpeg out流程：分析進，ffmpeg 出
Six small modules turn the song into a render. Analysis and the “edit map” are data; the look and the compositing are ffmpeg filtergraphs; every output is versioned so nothing is overwritten.
六個小模組把歌曲變成成品。分析與「剪輯地圖」是資料；風格與合成是 ffmpeg 濾鏡圖；每次輸出都有版本，不會被覆蓋。
librosa · tempo · beats · sections · band energy · drops
director · acts · cadence ramp · layouts · hero drops · walls
per-cut ffmpeg filtergraphs — monitor mask · curve · glass · roll逐刀 ffmpeg 濾鏡圖 — 螢幕遮罩 · 彎曲 · 玻璃 · 滾動
concat → room / halation pass → mux串接 → 房間／暈光層 → 混流
4:3 · 16:9 · review
- 01_analyze — librosa extracts tempo (117.5 BPM), 456 beats, six section boundaries, per-band energy envelopes, and a robust multi-cue drop detector.
- edit_map — the director layer: maps sections to acts, snaps every cut to the beat grid, chooses a layout per cut, and places hero drops and the walls.
- layouts — a vocabulary of compositions: centered “power” framings (monolith, altar, pair), grids, and the oversized Paik walls.
- crt / make_assets — each screen is a real monitor: a rounded mask, barrel curve, glass rim-light, scanlines; globally a warm amber halation, signal-tied noise, and a vertical-hold roll.
- 03_compose → 04_render — one ffmpeg segment per cut, concatenated, given one “room” pass, then muxed to the untouched song.
- 01_analyze —— 用 librosa 取出速度（117.5 BPM）、456 個拍點、六個段落邊界、各頻段能量包絡，以及一個多線索、穩健的 drop 偵測器。
- edit_map —— 導演層：把段落對應到樂章、把每個剪點對齊拍點、為每一刀挑選版面，並安排「英雄 drop」與電視牆。
- layouts —— 一套構圖語彙：置中的「權力感」框（巨碑、祭壇、對稱雙螢）、格狀牆，以及超大尺寸的白南準電視牆。
- crt / make_assets —— 每個螢幕都是實體顯示器：圓角遮罩、桶形彎曲、玻璃邊緣光、掃描線；全域再加上暖琥珀暈光（halation）、與訊號連動的雜訊，以及垂直滾動。
- 03_compose → 04_render —— 每一刀算成一段 ffmpeg 片段，串接後過一次「房間」風格層，再與未經更動的原曲混流。
Cutting to the rise and fall依著起伏下刀
The whole edit is a machine for tracking the track's energy. Four devices do the work:
- Cadence ramp. Cut length shrinks act by act — 12s holds in the Invocation down to ~0.6s at the Sermon — so the cutting itself accelerates with the music.
- Drop-driven hero shots. A detector scores every beat by sub-bass impact, a preceding build/riser, and sustain. The most dramatic releases (1:55, 2:11, 3:21) each land on a single, big image — the red priestess or the idol — so the release breathes.
- The phasing lull. In the phaser-synth passage right after the main drop, the walking-feet clip plays at its original speed — a real-time breath against the fast cutting.
- The Paik walls. Three escalating monitor walls (≈10 → 16 → 24 screens) with a camera pull-back land on the biggest swells; the last unveils the full wall just before the collapse.
Underneath, the CRT corruption itself is music-driven: chromatic aberration and bloom widen with the bass, noise thickens in the quiet passages and clears at the peak.
整支剪輯就是一台追蹤歌曲能量的機器，由四個手法完成：
- 節奏遞增。剪點長度逐幕縮短 —— 從召喚段的 12 秒長鏡，一路壓到佈道段約 0.6 秒 —— 讓剪輯本身隨音樂加速。
- 由 drop 觸發的英雄鏡頭。偵測器以低頻衝擊、前導的鋪陳/riser、以及延續度為每個拍點評分。最戲劇性的釋放點（1:55、2:11、3:21）各自落在一個放大的單一畫面 —— 紅衣祭司或偶像 —— 讓釋放得以呼吸。
- 相位段的留白。在主 drop 之後的 phaser 合成器段落，走路腳步那段素材以原速播放 —— 對比快速剪接的一次即時呼吸。
- 白南準電視牆。三面逐步升級的螢幕牆（約 10 → 16 → 24 台）配上鏡頭拉遠，落在最大的能量湧動處；最後一面在崩解前揭示整牆。
更底層地，連 CRT 的訊號破壞本身也由音樂驅動：色差與暈光隨低頻加寬，雜訊在安靜段落變厚、在高潮處轉清。
Six rounds, each one a conversation六輪迭代，每輪都是一次對話
Every version was a prompt with notes, a render, and a review. Older renders are archived so any version can be compared side by side.
每個版本都是「一段帶註記的指令 → 一次算圖 → 一次檢視」。舊的算圖都被封存，任何版本都能並排比較。
The spine. Music analysis, six-act structure, beat-locked cutting, first CRT look — proof the loop works.骨架。音樂分析、六幕結構、對拍剪接、第一版 CRT 風格 —— 驗證整個循環可行。
Physical monitors + Paik walls. Per-screen curvature and the escalating video-wall climax with camera pull-backs.實體螢幕 + 電視牆。每個螢幕的彎曲，以及配合鏡頭拉遠、逐步升級的電視牆高潮。
Solid black, round corners, halation. Feeds clipped to rounded masks on true black; warm amber glow; fixed A/V drift.純黑、圓角、暈光。畫面裁進圓角遮罩、置於純黑上；暖琥珀輝光；修正影音飄移。
Subtler look + negative space. Lighter halation, faint edge-light, floating compositions, ascent CRT roll, signal-tied noise; non-wall scenes locked static.更收斂的風格 + 留白。更淡的暈光、微弱邊緣光、漂浮構圖、上升段的 CRT 滾動、與訊號連動的雜訊；非電視牆場景固定為靜止。
Projection of power. Negative space redesigned to centered, bilaterally symmetric framings; 8px corners; ending locked to the AnimateDiff idol.權力的投射。留白重新設計為置中、左右對稱的構圖；8px 圓角；結尾固定為 AnimateDiff 偶像。
Music-driven footage. Hero shots on the detected drops; the walking clip at original speed in the phasing passage; the row-of-three equalized.音樂驅動的選材。在偵測到的 drop 上放英雄鏡頭；相位段的走路素材以原速播放；三連格改為等分。
What editing-by-prompt is good and bad at對話式剪輯的長處與短處
Where it wins
它的長處
- Speed of iteration — a note becomes a render without touching a timeline.
- Reproducible & versioned — the edit is code; any version rebuilds exactly.
- Music-driven automation — 156 beat-accurate cuts placed by analysis, not by hand.
- Systemic changes are cheap — “make all corners smaller” is one line, not 156 edits.
- 迭代速度 —— 一句註記就能變成一次算圖，完全不碰時間軸。
- 可重現、有版本 —— 剪輯即程式；任何版本都能精確重建。
- 音樂驅動的自動化 —— 156 個對拍剪點由分析決定，而非手工擺放。
- 系統性修改很便宜 —— 「把所有圓角改小」是一行程式，不是 156 次手動修改。
Where it hurts
它的短處
- No real-time scrub — you review a render, not a live playhead.
- Render waits — each full pass is minutes, not instant feedback.
- Filtergraph complexity — the look lives in dense ffmpeg strings that must be de-risked in small tests.
- Taste still human — “projection of power,” “too magenta” — the agent proposes, a human eye decides.
- 無法即時刷播 —— 你檢視的是算圖結果，不是即時播放頭。
- 算圖等待 —— 每次完整算圖是以分鐘計，非即時回饋。
- 濾鏡圖的複雜度 —— 風格藏在密集的 ffmpeg 字串裡，得靠小規模測試逐一排除風險。
- 品味仍靠人 —— 「權力的投射」、「太洋紅了」—— 代理提案，人眼定奪。
The honest conclusion: this suits energy-matched, generative, non-narrative montage extremely well — and would fight you on a dialogue scene that needs a human on a J-cut. Here, the constraints were a gift.
誠實的結論：這套方法非常適合能量導向、生成式、非敘事的剪輯 —— 但若是需要人工處理 J-cut 的對白場景，它會跟你作對。在這支片子裡，這些限制反而是禮物。
The whole render, four commands整支片子，四行指令
The pipeline lives in build/; music analysis runs in a pinned Python 3.12 venv (librosa needs it). One command chain rebuilds any version.
整條流程放在 build/；音樂分析跑在固定的 Python 3.12 虛擬環境（librosa 需要）。一串指令即可重建任何版本。
# analyse the song → cache/analysis.json (once) build/.venv/bin/python build/01_analyze.py # build the edit map → cache/editmap.json build/.venv/bin/python build/edit_map.py # compose every cut, then render + mux into versions/v6/ build/.venv/bin/python build/03_compose.py build/.venv/bin/python build/04_render.py --version v6 --tag master
Dependencies: ffmpeg 7.1, and (in the venv) librosa · soundfile · scipy · numpy · Pillow. Look-dev is fast — render any slice with --cut A B and read a frame back before committing to a full pass.
相依套件：ffmpeg 7.1，以及（虛擬環境內）librosa · soundfile · scipy · numpy · Pillow。試看很快 —— 用 --cut A B 算任一小段、抓一格回看，確認無誤再跑完整算圖。