I Switched My AI Anime Storyboards to "One Panel, One Line" and Made 28 Episodes in Two Days

Once making an episode got easy, my time started going into deciding "what to make."

Over the two days of October 8–9, I made short animations for 7 series, 4 episodes each. That's 28 episodes in total, about 26 minutes. This is a production log from after I switched to assembling storyboards with code and changed how I handle the art and the voice acting.

The channel is called "異境劇場｜IKYO STORIES": vertical animations of roughly 30 seconds to a minute and a half per episode, posted on YouTube Shorts.

https://www.youtube.com/@ikyostories ↗

I wrote before about throwing away all 6 of the first ones I made. Since then I've changed my approach twice more, and I'm finally able to produce at volume. That said, while all 28 episodes passed the automated checks, I haven't finished checking all of them by ear yet.

# It used to be handmade, one shot at a time

At first I drew one image per shot, then added motion with Canvas. For a sword swing, I'd break it down from raising the sword to finishing the swing into several images, then add speed lines. It looked good, but each episode took a ridiculous amount of time.

So I made the call all at once: "Let's turn everything into manga pages." Panels fly in one by one, and the speech bubbles double as subtitles. Hands reaching in from outside the panel, pose switches, mouths flapping along with the voice: the motion got narrowed down to just these few types.

# Storyboards are "one panel, one line"

The single most effective change was assembling the storyboards with code.

In the script, each panel gets "what's in the shot," "who's speaking," and "how it's presented." For a scene where someone's true identity is revealed, I specify that presentation and add the line plus acting directions. I call it "one panel, one line," but it isn't literally about matching up line counts. As long as each panel's info is written as one bundle, how the pages get split and where the bubbles go can all be changed later with code.

The rest is left to code: splitting pages, deciding image sizes and bubble positions. Mouth shapes, which way the bubble tails point, sound effects, and thumbnails are all generated from the same data too.

The goal was "storyboarding from 2 hours down to 30 minutes," but I haven't actually measured it under the same conditions yet. Still, when I'm writing scripts, I don't have to think about layout at all, and that alone makes life a lot easier.

# New images via API, partial redraws on my own PC

Images get drawn in two places:

• New images go via API to an external image generation service (Image 2.5). About 16 seconds per image, 4 at a time. I attach character reference images every time so the faces don't drift
• Partial redraws, like mouth open/close or background swaps, go to an image model running on my own PC (RTX 4070)

At first I drew everything on my own PC, but I kept getting weird text on signboards and poses it just couldn't draw, so there were tons of rejects. Since moving new images to the API, all the new art for an episode is ready in about 2 minutes.

I changed the art style too. Glossy thick-paint rendering, blurred backgrounds, backlighting: that's what "AI-ness" actually is. So I switched to flat colors with hard shadows, like a screenshot from a 2010s TV anime.

# With a free GPU, 4 seconds a line

For voices, I run Irodori, a speech synthesis model that's really strong at Japanese, on my own PC. When the GPU is free, a line takes about 4 seconds to read. An episode has 11–29 lines, so the voice acting alone is done in a few minutes.

I first make one voice sample per character, then read every line in that voice. Each line can also get acting directions like "quietly, speaking fast."

There were some sticking points too:

• I couldn't get a deep male voice. Writing "taciturn, cold" doesn't make the voice any lower. In the end I made the sample with a large model, specifying "a deep, resonant adult male voice, past his voice break," and finally got the pitch I wanted
• A god I described as "small-bodied" ended up with a child's voice
• For a shy girl I wrote "very quietly," and got a super high, super thin voice. Changing it to "a lowish, calm voice" fixed it

I also measure pitch when making voice samples. If it's off from the target by more than 1.4×, the sample gets rejected and the process stops right there.

# Read too fast, short lines turn to mush

The first version felt slow. There were 0.85–1.45 second gaps between lines, and the time when a line was actually playing was only about 70% of the total.

So I cut the gaps in half and sped up the delivery. The model already has a "reading length" setting; at 0.8 it gets you roughly 1.25× speed without changing the pitch. One episode went from 58 seconds to 44 seconds.

I tried to see if it could go any faster, and 0.7 is the limit. Go to 0.66 or below and push the speed further, and short lines like "...Who is this?" turn to mush no matter which voice you use.

To check whether lines are intelligible, I feed every line to Whisper, the speech recognition model, and compare the result against the script. Names are easy to mispronounce, so in the dialogue text only, I write names in kana.

# Passing the automated checks doesn't get you out of listening

Every episode goes through the same set of checks:

• Do the speech bubbles cover any character's face
• Does any text spill out of the frame
• Is each line loud enough
• Does the speech recognition output match the script
• Is the loudness of the whole video within the required range
• Can the invisible watermark embedded in the video be read back

For example, episode 1 of To Me, Who Loses My Memory Every 24 Hours is 62.85 seconds with 27 lines. The speech bubbles didn't cover any of the 21 detected faces, and no text spilled out.

But speech recognition only matched the script on 21 of the 27 lines. The other 6 weren't just differences in kanji spelling; some were recognized as entirely different words. Those got flagged as "needs review" in the check log.

Anything with errors doesn't get published. But passing the checks doesn't mean it's done. A program that only measures loudness and compares text can't judge the acting in the lines or the balance with the BGM. I have to listen to that part myself. Review of the 28 episodes is still in progress.

Once making an episode got easy, my time started going into deciding "what to make." Next time I'll write about having AI write 155 project pitches and then having 8 AI judges pick from them.