Plan B: Text-to-Video Storyboards (Pro)

Jimeng / Seedance / Kling

Lock a CHARACTER DNA block at the top (Identity / Face / Hair / Outfit / body / vibe / no-drift items); change only the 'scene block' (place / time / action) per shot. Order video prompts by priority: composition → subject → camera move → overall mood, which sharply reduces element conflicts. Keep a fixed negative: no different eye color / no extra characters / no watermark. Log a manifest (prompt / seed / model version / reference image) for reproducibility.

💡Concrete constraints like 'low-angle upward shot, fluid motion, no distortion' beat stacks of adjectives.

Midjourney / Jimeng + ComfyUI

Tier 1 (no training, 70-80%): generate front/side/back character sheets as references (Midjourney --cref / Jimeng reference), plain simple background. Tier 2 (no training, ~95%): ComfyUI chain InstantID + IP-Adapter FaceID → ControlNet OpenPose → FaceDetailer; CFG 4-5, IP-Adapter weight 0.6-0.8, fixed seed. Tier 3 (trained, best): train a LoRA on 20-30 multi-angle shots (1024², 1500-3000 steps, lr 1e-4~5e-5). For cross-shot video, lock identity with first/last frames + interpolation.

💡90% of 'face melt' comes from busy references or extreme head-on shots. Make the character sheet clean first.

CapCut / FFmpeg

Assemble the whole film from storyboard stills + voiceover to check pacing, shot transitions and timing BEFORE spending money on video generation. This step is nearly free and the most cost-effective iteration point — fixing pacing here can halve your API bill.

💡Animatic-first is the professional consensus: preview in stills, then generate. Never the reverse.

Kling / Jimeng / Vidu / Runway / Seedance

5-8 seconds per shot, using the first-frame image + the tool's subject library ('@' the character) + voice reference. Picks: Kling (Chinese storytelling, physical realism), Jimeng (all-in-one, anime), Vidu (multi-subject consistency), Runway (director-level camera), Seedance (cinematic long takes). On failure (face melt / bad physics), fix the first frame or prompt and retry — cheaper than fixing in post.

💡Generate 2-3 candidates per shot and pick; keep seeds fixed for reproducibility.

ElevenLabs / MiniMax + Suno

Tag emotions per character when dubbing (e.g. [angrily]); lip sync relies on punctuation. Generate BGM with Suno, keep ambience as its own bed layer. For singing scenes, separate vocals with Demucs before driving lip sync to avoid 'mouth flailing'. Lip-sync audio prep: 16-24kHz mono WAV, trim leading silence, add ~0.2s tail, normalize to -3dB peak.

💡Music at ~30% of voice volume; mixing ambience and music as separate layers instantly raises quality.

FFmpeg / CapCut / DaVinci

Drive assembly from a timeline spec (film_spec: cuts / music spans / ambience / color / fades) instead of hand-stitching. Triple QC gates: ① Whisper-back each narration line, similarity ≥0.85 or re-synthesize; ② CLIP image-text consistency check to reject drifted keyframes early; ③ final self-review (ffprobe validation, frame sampling, audio level, captions). Resume on interruption; fix a single line with a minimal-repair tool instead of re-rendering.

💡Turn QC into deterministic machine-checkable rules, not eyeballing. Keep storyboard, voice samples and manifest for regression.