How 1,029 Storyboard Tasks Are Grabbed and Processed in Parallel Across Three Computers

Three computers completing image generation and video generation for 1,029 shots sounds like a fantasy, but the actual working principle is not that one machine processes the complete sequence; rather, all tasks are broken into countless atomic-level job units, and each computer continuously grabs pending records from the task queue in a task-grabbing mode. The core of this mechanism is not how sophisticated the parallel computing is, but that tasks are scientifically enumerated and state-managed, so that each computer knows at any moment what it should do and also knows what the other computers are doing, avoiding duplicate work and idle resources. The Yuncao Academy system stores every pending image generation task and video generation task in database tables and controls who is executing, who has completed, and who is waiting through status fields. All three computers run the same client program, which scans the task table every few seconds, executes tasks immediately after grabbing them, updates the status upon completion, and grabs the next task. In actual deployment, the three computers are labeled Host A, Host B, and Host C. Host A is a workstation equipped with a high-performance graphics card, responsible for drawrole synchronous direct-call character image generation, because its single-image generation speed is fast with low latency, making it suitable for handling high-concurrency requests. Host B and Host C have slightly weaker configurations, but they excel in large memory and stable operation, and they mainly handle placeaiwords asynchronous-branch location image generation and scene compositing images. Video generation tasks are uniformly pushed to the Jimeng platform, which does not consume local computing power. Each computer runs a desktop client. After the client starts, it connects to the Yuncao Academy database and reads a configuration file that records the worker_id of the current machine and the task type it is specialized in. Host A's worker_id is 001 and its task type is character image. Host B's worker_id is 002 and its task type is location image. Host C's worker_id is 003 and its task type is scene compositing. However, this does not mean Host A only processes character images; rather, it has priority in character image tasks, and when there are no character image tasks, it will also grab other task types. The core of task grabbing is the status field in the dbvideo_arcpictask table. Status has three values: 0 means pending, 1 means submitted and being processed, and 2 means completed. Whenever a computer is idle, it executes an SQL update statement: UPDATE dbvideo_arcpictask SET status = 1, worker_id = '003' WHERE status = 0 AND tasktype = 'scene' ORDER BY taskid ASC LIMIT 1. Because UPDATE is atomic at the database level, only one computer can successfully update the same row at any given moment. When other computers execute the same UPDATE, they cannot match rows with status = 0, so they automatically switch to other tasks with different statuses. This database-row-lock-based task-grabbing method is extremely stable, requiring no additional message queue middleware and no complex distributed locks. Each computer also periodically executes SELECT COUNT(*) FROM dbvideo_arcpictask WHERE status = 1 AND worker_id = '001', for the purpose of detecting its own unfinished tasks that are being executed. If a task has not been completed after the predetermined timeout threshold, the computer actively resets the status to 0 and releases the worker_id, allowing other computers to grab the task again. When three computers process in parallel, the complete workflow of each shot is split into four stages. The first stage is the image generation task, which generates the first frame and last frame images. There are 1,029 shots in total, and each shot requires two images, totaling 2,058 image tasks. Among them, Host A handles about 800 character image generation tasks, Host B handles about 300 pure text-to-image location images, and Host C handles about 1,000 scene compositing images. After the image generation tasks are completed, the firstpic and secondpic fields in the arcpictask record are backfilled with the actual paths and the status is set to completed. The second stage is pulling the shots whose images have been completed and generating video prompt tasks. These tasks are uniformly written into the dbvideo_process table, with the status set to ready, indicating that they are waiting for video generation. The third stage is submitting video generation tasks to the Jimeng platform. The client scans records in dbvideo_process with status = ready, calls the Jimeng API to send the aiwords and the first and last frame image paths, and after successful submission, stores the returned task_id in the vdurl field and updates the status to submitted. The fourth stage is polling the video generation status. The client calls the Jimeng query interface every five seconds. When video generation is complete, the system stores the returned video metadata in the dbvideo_videometa table and marks the corresponding scenecode as done. But how do two computers avoid submitting a video generation request for the same shot at the same time? The answer is a small field named task_lock in the dbvideo_process table. Its default value is 0. Before submitting, the client executes an UPDATE dbvideo_process SET task_lock = 1 WHERE scenecode = 'SCN000245' AND task_lock = 0. Only the client that succeeds in the update is qualified to submit that shot to Jimeng. When another client attempts to update the same row of the task, it fails because task_lock is no longer 0, and it automatically skips that shot and continues searching for the next unlocked task. This lightweight locking mechanism guarantees that even if three computers scan the same scenecode at the same time, only one can successfully submit, producing no duplicate tasks. Moreover, if that client suddenly loses power after submitting, other computers will automatically take over after the lock times out. This lock timeout mechanism is maintained through the last_update_time field. The current time is recorded at the same time as the update is executed. When other computers scan and find that last_update_time is more than ten minutes old, they reset task_lock to 0 and re-grab the task. In the actual operation of "The First Avenue," Host A took an average of twenty seconds per character image, and all 800 character images took about 4.4 hours to complete. Host B's pure text-to-image location images took an average of thirty-five seconds each, and 300 location images took about 2.9 hours. Host C's scene compositing images took an average of forty seconds each, and 1,000 compositing images took about 11.1 hours, making this the bottleneck of the entire pipeline. To avoid Host A and Host B sitting idle after completing their own tasks, dynamic task type switching logic was written into the client. When Host A detects that the character image task queue is empty, it automatically changes its own worker_type to scene and begins attempting to grab scene compositing tasks. Similarly, Host B also switches to scene tasks after completing its tasks. In this way, all three machines are fully committed to scene compositing images in the later stage, and the actual total completion time was nearly three times faster than a single machine. The initial estimate was 30 hours for a single machine to process everything, but with three machines running in parallel, all image generation and video submission were actually completed in 10 hours and 40 minutes. During this period, three single-machine freezes occurred, but after the frozen machine restarted and re-scanned the task table, it found that the already-processed tasks had normal statuses, and the unprocessed tasks, because their status was still 0, were automatically picked up by other computers, requiring no manual intervention at all. During the video generation stage, because tasks were queued in the Jimeng cloud, local computing power was no longer consumed. However, due to the large volume of tasks, the Jimeng platform's queue sometimes became backed up. The client did not simply wait idle; instead, it continued generating image generation tasks for the next batch of shots, because arcpictask task generation and video submission are asynchronously decoupled. You can have Host B specifically monitor the Jimeng return status while the video submission queue has not yet cleared, while Host A and Host C continue generating images for the next chapter. The entire process ultimately covered all sixty-nine chapters of the book, 1,029 shots, with a finished runtime of 2 hours and 20 minutes. The workload completed by each of the three computers is recorded in the dbvideo_worklog table. Host A completed 342 image generation tasks, Host B completed 231 image generation tasks, and Host C completed 456 image generation tasks. Together, the three machines submitted 1,029 video tasks, retried 67 times after failures, and all tasks were completed within eleven days. The essence of this parallel mechanism is that every shot becomes an independently schedulable job, and any computer can join or exit at any time without affecting overall progress.