Conference Presentation
Every CIO will have to answer for every token | Factory's Matan Grinberg
- The factory router enables dynamic model selection based on specific tasks, optimizing for efficiency rather than using a single "frontier" model for all queries.
- Enterprises face a critical, unaddressed challenge within the next 12 months: tailoring token allocation to specific roles rather than applying blanket caps across an organization.
- Current token management practices often lack granularity, potentially assigning identical limits to distinct roles such as product managers writing code versus engineers building critical infrastructure.
- CIOs must determine the optimal placement of incremental tokens, a decision framework that is currently undefined for many large-scale organizations.
- Routing logic allows organizations to bypass expensive generalist models (e.g., Opus) in favor of specialized, fine-tuned alternatives for legacy systems like COBOL code bases.
- Specific organizational units can be assigned distinct models based on their workflow:
- "Vibe coding" teams are routed to lightweight models like Gemini Flash.
- COBOL-specific tasks are routed to custom fine-tuned models optimized for that language.
- High-reliability requirements can be met by generating code with OpenAI, testing it with Anthropic, and reviewing it with Gemini.
- The strategy prioritizes matching model capability and cost to the specific constraints and output requirements of different parts of the organization.