'The Future of Software With GenAI' by Marek Kalnik, Group Partner Theodo | RAISE Summit 2024

Market Adoption and Scale

• GitHub Copilot has sold over 1 million paid licenses, while VS Code Marketplace reports 25.5 million installations of code generation tools.
• Current estimates indicate the vast majority of the global developer workforce has been exposed to AI tooling, though definitions of "developer" vary and often exclude ops roles.
• Marek Kalnyk (CTO of BAM mobile, Teodo Group) asserts that teams not utilizing Generative AI daily are effectively lagging behind industry standards.

Evolution of Software Production

• The 20th-century factory analogy illustrates that simply replacing human labor with electric motors (early AI adoption) yields limited gains until the system itself is redesigned (e.g., Ford's assembly lines).
• Present-day AI integration is currently characterized by "co-pilot" assistance rather than "autopilot," where AI augments human tasks rather than replacing the entire workflow architecture.
• Teodo Group observed that AI consistently achieves near 100% success on specific tasks like API calls, page generation, and data bootstrapping.
• Internal testing at Teodo showed a 0.37 similarity score between AI-generated code and human-written code for ticket-based tasks, indicating that minor rework allows AI to handle the bulk of development.

Emerging AI Capabilities and Benchmarks

• Devin: A new autonomous agent that solved 13–14% of real-world open-source tasks autonomously while iterating and fixing errors.
• Microsoft AutoDev: Achieved over 90% correctness on algorithmic coding tests (Human Eval), outperforming many junior candidates, though these tests do not reflect complex real-world scenarios.
• Task categorization is shifting into three tiers: tasks requiring human supervision, tasks AI can execute with minor rework, and tasks capable of full autonomy.
• Human-only development tasks are projected to plateau as AI adoption increases, necessitating a shift in team focus toward higher-level logic and design.

Future Architecture and Business Impact

• Future development models may utilize a "Product Builder" who manages a platform to generate code via AI, reserving human teams for complex problem-solving and quality control.
• Companies are expected to build in-house AI engineering teams to manage security-sensitive, custom-built platforms.
• Two-tiered systems are predicted to emerge: AI handling execution alongside human teams focused on software design, quality assurance, and custom AI asset creation.
• Strategic business outcomes include reduced software costs, improved legacy system maintenance, and the potential for personalized, user-generated software products.