Guido

The True Cost of Compute

Guido

Training large language models now requires astronomical computational resources costing tens of millions of dollars, with startups often allocating over 80% of their capital to secure the specialized hardware needed for training phases that consume six times more operations than inference. While the absolute expense of training is projected to rise as the industry expands, experts anticipate that a looming scarcity of high-quality human-generated data will soon become a more significant constraint than compute availability. Consequently, this dynamic creates a competitive environment where well-funded entrants can overcome capital barriers, provided they navigate the diminishing returns of scaling model size without matching data quantities.