Conference Presentation
Focusing on data quality over quantity, Metalware built a foundation model with less compute.
- Metalware, a company lacking internal PhD-level machine learning expertise, joined the batch with the objective of building a co-pilot specifically for hardware design.
- The company adopted a data strategy prioritizing high quality over volume, sourcing and scanning textbook figures and hardware information to serve as their primary input.
- Leveraging this curated dataset, Metalware successfully utilized the significantly smaller GPT-2.5 model (approx. 1 billion parameters) instead of larger alternatives like GPT-4 (approx. 1 trillion parameters).
- This approach allowed Metalware to reduce computational resource requirements while still achieving functional results, demonstrating that constrained tasks with high-quality data can effectively replace the need for massive models.