The Trust Phase: What Does it actually Take to Trust Agentic NetOps - ILOG SA
Explicit Guidance:
- No specific financial guidance (revenue, earnings, margins, capex, etc.) or quantitative outlook for IBM was provided in this transcript. The discussion focuses on product capabilities, market research data, and implementation timelines rather than financial projections.
Business-Segment Outlook:
- Adoption Timelines: Research indicates that 29% of enterprises plan to deploy AI-powered network management tools within the next 12 months.
- Trust Trajectory: Only 31% of IT professionals currently "completely trust" AI network management tools, while 64% operate with a "somewhat trusting" (verify) mindset. Trust levels are expected to correlate directly with increased day-to-day operational usage and ROI.
- Operational Maturity: The market is shifting from questioning "capability" (2024-early 2025) to demanding "reliability and auditability" (late 2025-2026). Buyers are now prioritizing questions on "Will this do it again tomorrow?" and "How do I know when it's wrong?"
- Segment Focus: DDI (DNS, DHCP, IP management) is identified as a specific domain where AI adoption is maturing, with 55% of respondents using AI tools and 10% willing to use fully autonomous agents, though the majority prefer agentic (assisted) operations.
- Outcome Metrics: Network Operations Center (NOC) leaders are moving away from measuring feature adoption (e.g., MCP tools, LLM types) toward outcome metrics: time-to-diagnosis, reduction in service-impacting events, and governed change success rates (targeted at 100% audit trails).
Market and Competitive Outlook:
- Vendor Requirement: AI capability is now a "requirement" for 55% of vendors being evaluated for network management tools; it serves as a signal of innovation.
- Competitive Differentiator: The market is shifting from "black box" AI to transparent architectures with visible data attribution, reasoning trajectories, and decision trees. Vendors failing to provide audit trails and evaluation suites face higher barriers to trust.
- Autonomy Limitations: Full autonomy in DDI is limited (<6% desired) and in network operations generally (19.9% for reviewing changes) due to data quality and risk concerns; the near-term standard remains "human-in-the-loop" for high-risk tasks.
- Tool Ecosystem: A proliferation of Model Context Protocol (MCP) tools and A2A sessions is occurring, but market participants warn that excessive tool counts without clear utility increase hallucination risks and context window fatigue.
- Evaluation Standards: Buyers are increasingly demanding "golden data sets," deterministic code checks, and online evaluations (real-time scoring) as proof of product reliability, moving beyond vendor demos.
Macro / Industry Outlook:
- Economic Barriers: Security and compliance risks are the #1 barrier to AI adoption in enterprise IT, slightly ahead of budget constraints.
- Security Evolution: Security reviews are stalling deployments because current Role-Based Access Control (RBAC) is insufficient; the industry is moving toward Attribute-Based Access Control (ABAC) to embed "judgment" into agent permissions.
- Data Quality as a Bottleneck: 51.5% of organizations are only "somewhat confident" in their data quality; key blockers include data collection errors (36%), storage/retention costs (36%), inconsistent formatting (35%), and blind spots in network monitoring.
- Human Error vs. AI Error: Human manual errors currently drive 23–27% of network incidents; while AI is imperfect, industry perception suggests human error remains a higher risk than AI automation.
- Regulatory/Compliance: Regulatory compliance tasks are seeing high acceptance of AI (20–30% willing to allow AI to drive compliance), but "verify" remains the dominant mode.
Strategic Initiatives:
- Product Roadmap (IBM Network Intelligence - INI): Developing a "persistent operational agentic layer" that sits on top of existing observability tools, not replacing them.
- Trust Architecture: Implementing a four-stage loop (Sense, Reason, Govern, Act) with specific controls:
- Sense: Aggregating telemetry into validated observations/symptoms before model ingestion.
- Reason: Using a time-series foundation model ensemble with ontological grounding (knowledge graphs) under development.
- Govern: Mandatory approvals for config changes, scoped tools, and banned commands.
- Act: Generating Ansible playbooks pre-validated against digital twins (e.g., Forward Network Predict) before execution.
- Evaluation Framework: Launching continuous "online evals" where every tool call and agent trace is instrumented with OpenTelemetry for real-time scoring and drift monitoring.
- Data Integrity Initiatives: Actively incubating "ontological grounding" via knowledge graphs to introduce determinism into probabilistic agent reasoning.
- Collaboration Features: Enabling "Human Contribution" where senior engineers can correct an agent's reasoning trajectory, feeding tribal knowledge back into the system to improve future outcomes.
- Audit Transparency: Providing customers with visibility into false positive rates, hallucination rates, tool error frequencies, and cost-per-investigation via an evaluation suite (using IBM "Bob" AI Judge).
Risks and Headwinds to Outlook:
- Data Quality Deficiencies: Fragmented data, inconsistent formatting, and "blind spots" in network observability prevent AI from making reliable decisions.
- Non-Determinism: LLM behavior changes with new data, model upgrades, or prompt changes; a 95% reliable step compounded over 20 steps results in ~36% end-to-end reliability.
- Security & Access Control: Lack of granular, context-aware access (ABAC) poses a significant barrier to production deployment.
- Hallucination & Tool Errors: Adding more tools increases the surface area for errors and hallucinations; tool error rates in test labs were observed at 6%.
- Integration Complexity: Difficulty in maintaining uniformity across large, heterogeneous networks (e.g., 40,000 sites with unauthorized hardware/config changes) limits full autonomy.
- Cost of AI: AI introduces new OpEx (tokens, inference, tool calls) that must be tracked and optimized per incident resolution.
Management Tone on the Future:
- Confidence Level: Moderate to high regarding the capability of AI models, but cautious and pragmatic regarding autonomy and trust.
- Strategic Shift: Management views the current phase as a transition from "magic" to "engineering," where the focus is on harnessing non-determinism through rigorous controls, audits, and validation.
- Customer Expectations: High skepticism regarding vendor claims; success depends on demonstrating transparency, audit trails, and tangible business outcomes rather than technical features.
- Optimism on Reliability: Belief that reliability can be engineered into the system through "governed actions" and digital twin validation, even if the underlying models remain probabilistic.
Miscellaneous:
- Timeframe References: "Today" (immediate adoption), "Next 12 months" (deployment horizon for 29% of enterprises), "2025" (market focus on capability), "2026" (market focus on trust/audit).
- Specific Metrics Cited: 55% AI requirement, 29% 12-month deployment, 6% full autonomy in DDI, 23-27% human error incidents, 6% tool error rate in INI test labs (1,900 investigations), ~0.2–0.7 cents per audit cost.
- Key Concepts: "Trust but verify," "Ontological grounding," "Human-in-the-loop," "Digital twin validation," "ABAC vs. RBAC."
- Company
- ILOG SA
- Keywords
- ILOG SA