newsfilter
← Interviews
Sequoia Capital1 min

The Problem With Testing AI Architectures at Small Scale | Jerry Tworek, Core Automation

Jerry Tworek

The event critiques the industry's conventional practice of validating model architectures on small datasets, arguing that this approach fails to capture the true potential of scalable systems. It specifically highlights that reinforcement learning models require a minimum threshold of compute to exhibit meaningful capabilities, suggesting that lower-scale testing often yields uninteresting results. Consequently, the organization advocates for a paradigm shift toward large-scale initial evaluations to ensure architectural research accurately reflects future performance.