Conference Presentation, Keynote, Product Demonstration
Designing Characters with Deep Learning: Spellbrush (W18) - YC Gaming Tech Talks 2020
- SpellRush, a Y Combinator-backed company, develops deep learning tools to scale art production for games and media, addressing the industry trend where art consumes 50–70% of AAA game production budgets.
- The company's AI technology can generate high-quality character illustrations in under two seconds, a speed comparable to 2–15 hours of professional manual labor, and can generate hundreds of character variations in that same timeframe.
- The core technology utilizes Generative Adversarial Networks (GANs) comprising a generator network (which learns to create art) and a discriminator network (which evaluates authenticity), trained over millions of iterations to produce images indistinguishable from human work.
- The system uses "latent space" noise inputs to allow precise control over generated outputs, enabling rapid variations in character expressions, colors, and illustration styles without manual redrawing.
- The company trained its initial models on a crawled dataset of 10 million publicly available anime-style images, which presented significant demographic skew: approximately 1 to 6 female-to-male character ratio and less than 3% representation of darker skin tones.
- SpellRush actively mitigated dataset bias by engineering the model to generate darker skin tones and male characters at higher frequencies than the raw internet data supports, aiming for better real-world representation and solving the scarcity of high-quality male art in the anime industry.
- The company recently transitioned from cloud infrastructure to an on-premises "DIY supercomputer" located in their office to reduce training costs, featuring a 42U rack with a top-of-rack router, 200+ cores, 20+ GPUs, and 40+ terabytes of flash storage.
- Training a single model currently costs between $3,000 and $4,000, a price derived from the 7–10 day training duration required, which is significantly cheaper than the estimated $10–$24 per hour cost of comparable cloud instances (e.g., AWS P3 16xlarge).
- The technical architecture utilizes a custom internal language called "NetGen" to define GAN architectures, which compiles into TensorFlow low-level operations, runs within Singularity containers, and is scheduled on the cluster via Slurm.
- Monitoring for the system includes custom data piping to Prometheus and Grafana, alongside TensorBoard for tracking loss functions during the training process.
- Additional research areas include automated animation using the Live2D Spine workflow and super-resolution techniques, demonstrated by fully generated and animated characters.
- SpellRush is hiring a 2D animator, motion designer, real-time VFX artist, and an AI research intern for the upcoming winter; the team currently consists of five people and aims to add one new hire immediately.
- The company is building the world's first AI-illustrated game, leveraging their internal tools to manage the asset pipeline for this specific project.