Parallel Secret: 11 Days, 2-Hour Film via Multi-Machine Task Racing

Producing a two-hour-and-twenty-minute film in eleven days sounds like a fantasy, but when you distribute the generation of 1,029 shots across three computers for parallel processing, that number becomes plausible. In single-machine serial mode, every shot must pass through five stages: character image generation, location image generation, scene synthesis, first-and-last-frame generation, and Jimeng video generation. Assuming each shot averages thirty minutes, a single machine processing all 1,029 shots would need 514 hours—at a nonstop 24 hours per day, that is at least twenty-one days. Yet *First Avenue* was actually completed in eleven days, which means the system did not linearly feed tasks one by one to a single machine but instead compressed the total time by nearly half through multi-machine concurrency. However, merely treating three machines as a 3x-speed serial device would yield around seven days—so why eleven days and not shorter? The answer lies in task granularity, state scheduling, and the safety trade-offs imposed by idempotency locks; that is the real crux of understanding parallel architecture.

The task queue is the heart of the entire parallel system. In the `dbvideo_process` table, after 1,029 storyboard records are generated, they are not pre-assigned to any specific computer—they all remain in a pending state. Each of the three machines runs a task-polling client that sends a task-grabbing request to the database every few seconds, selects one record currently in the pending state, flips its status to in-progress, and then executes the full generation pipeline for that shot. The task-grabbing process is entirely stateless: each machine keeps no local context across shots. It only needs to know which record to process next and from which tables to read the associated character, location, and style fields. This stateless design makes task distribution extremely simple—every machine is an equal worker; whoever is idle proactively pulls from the pool, with no central scheduler performing complex load balancing. Instead, each `process` record in the database itself acts as a state machine, transitioning through the state field among pending, in-progress, completed, and failed-retry, allowing multiple machines to avoid duplicate task acquisition through atomic row-level updates in the database.

But the prerequisite for tasks being concurrently grabbed is that data dependencies have been thoroughly decoupled. In *First Avenue*, the texts of the sixty-nine chapters are mutually independent: characters from one chapter do not appear in another, and locations from one chapter do not undergo dramatic changes in another. This independence is achieved through the design of the `atctoscene` function, which parses each chapter individually, extracting characters into `dbvideo_roles`, locations into `dbvideo_places`, and then generating that chapter's storyboard records. Because the character sets of each chapter do not overlap, three machines can simultaneously process text-parsing tasks for different chapters—machine A parses chapter four, machine B parses chapter fifty, machine C parses chapter sixty-nine—each invoking `atctoscene` without interference. However, this also introduces the greatest hazard in parallel safety: if the same machine or different machines attempt to process the same chapter concurrently, duplicate character records, duplicate location records, and duplicate storyboard records will be created, leading to chaos in the generated video assets. To prevent this, the `atctoscene` function has a built-in idempotency lock: once `atctoscene` begins executing for a chapter, the database locks the corresponding chapter record, and any other machine detecting that the chapter's parsing task is already locked will skip it or wait until the lock is released. This means the same chapter can never be re-run, while different chapters can parallelize freely—this rule is the cornerstone that allows the parallel architecture to accelerate while maintaining data consistency.

The idempotency lock, on the surface, restricts duplicate processing of the same chapter; in reality, it protects the global consistency of the entire parallel system. Without this lock, if machine A and machine B both grabbed the parsing task for chapter five simultaneously, each would independently read the source text, generate character and location records, and insert two identical storyboard records into the `process` table. When later generating video, the same visuals would be created twice, wasting compute, possibly causing file overwrites, or producing garbled video content. With the idempotency lock, although concurrent processing of the same chapter is sacrificed, the system gains the freedom to parallelize safely within correct data boundaries. For *First Avenue*, all sixty-nine chapters can have sixty-nine tasks simultaneously claimed by different machines; each machine processes one or more chapters, and once all are complete, the 1,029 records in the `process` table are finalized. Then, in the storyboard generation stage, machines A, B, and C no longer care about chapter ownership—they only look at the status of `process` records. Thus, they can pull storyboard tasks from different chapters at any time, further dispersing the workload and refining task granularity down to the single-shot level. This fine-grained task allocation makes far better use of idle compute across the three machines than fixed chapter assignment, because each shot's processing time varies—some shots require complex scene synthesis, while others are simple sky backdrops. If chapters were fixed to specific machines, one machine might finish its chapters while another remains busy, wasting idle capacity.

The actual eleven-day duration can be reverse-derived from the speedup ratio. Assuming a single machine serially completes the entire film in 514 hours, the ideal parallel speedup with three machines is 3x, giving a theoretical time of roughly 171 hours. Converting to sixteen working hours per day, that is 10.7 days—approximately eleven days. This matches the actual eleven days almost exactly, indicating that the system achieved near-theoretical parallel efficiency. But there is a hidden precondition: the three machines were not running at full load around the clock, because one person cannot operate three machines simultaneously, and manual intervention time is also included within those eleven days. Therefore, in practical terms, each machine's effective working time may have been only twelve to fourteen hours per day. This puts the total available machine time across three machines at roughly 462 hours—slightly lower than the 514 hours needed by a single machine serially—yet the tasks were still completed within eleven days. This shows that multi-machine parallelism is not simply dividing tasks by three; rather, through stateless task distribution, each machine immediately claims the next shot after finishing one, eliminating the idle waiting for manual assignment found in traditional batch processing. Meanwhile, because the task queue is dynamic, when one machine encounters a generation failure for a particular shot, it does not block subsequent tasks—it simply sets that record's status to failed-retry and moves on to claim another shot. This elasticity prevents the system's overall throughput from degrading due to individual anomalies.

What is truly impressive is that the system's handling of locations also fits the parallel logic perfectly. `dbvideo_places` is not a location dictionary but one row per storyboard. This means a location may appear in multiple storyboards, but in the database it corresponds to multiple rows, each serving a specific shot context. This design allows different machines processing different storyboards to avoid sharing or locking a global description of the same location; instead, each reads its own row's location information and prompts, going about its own business. This effectively transforms shared resources into independent resources through data redundancy, eliminating the lock-waiting and deadlock risks that arise from contending for the same data row in a parallel environment. In the "Five Elements Dragon Flag" chapter of *First Avenue*, Dragon Flag Hall appears as the primary scene across dozens of shots, and each shot has its own independent row in the `places` table. The prompt content across rows may be identical, but they are stored as separate row values. Thus, while machine A processes a character close-up in Dragon Flag Hall, machine B simultaneously processes a full establishing shot of the hall—they read different rows, experience no conflict, and require no database row lock to serialize access. This space-for-time data redundancy strategy aligns perfectly with the fundamental principles of parallel computing.

Finally, looking at the results: the three-computer parallel architecture paired with the task queue produced a finished film in eleven days—two hours and twenty minutes, 2560x1440 resolution, 30 fps, 7.7 Mbps bitrate, with 87.2% audio coverage. Behind these numbers is the realization of machine utilization pushed to its extreme. Every computer during that period was almost never idle, because the pending-task pool remained non-empty at all times; after completing one shot, a machine immediately pulled the next, with only millisecond-level gaps in between. The core idea of this task-grabbing mode is to let tasks find machines, not machines find tasks. The bottleneck of manual scheduling is completely removed. Even if the machines differ in performance, faster machines will grab more tasks and slower ones fewer, and overall completion time is determined by the cumulative capability of all machines rather than being dragged down by the slowest one. This is the most elegant expression of parallel computing in AI generation: no complex pre-allocation, no shattering of overall data—just turning the task table into a shared mine, letting each machine's excavator freely extract. The idempotency lock guards the boundaries of critical zones, ensuring every vein of ore is neither over-mined nor missed. As more machines are added, the throughput of this task-grabbing mode approaches ideal linear scaling—and that is the core secret of using engineering methods to break through the limits of human creation.