Panel, Conference Presentation
Big Data as Driver of Disruptive Technology Breakthroughs
Milken InstituteSumant Mandal, Andrew Appel, Vanessa Colella, Vikas Kapoor, David A. Steinberg, Andreas Weigend
- Technological advancements driven by Moore's Law and decreasing storage costs are enabling the execution of previously day-long reports in milliseconds, shifting the strategic focus from data volume to asking meaningful questions and solving data governance problems.
- The long-term objective is to leverage machine learning to extrapolate 2D data into 3D non-hypothesis-based solutions, automate decision-making workflows to reduce operational expenses, and streamline processes such as supply chains, risk management, and customer retention.
- Most large enterprises face significant challenges integrating fragmented data across hundreds or thousands of legacy systems that were not stitched together despite expectations of a five-year cycle, necessitating a shift from building barriers to removing them for an end-to-end unified framework.
- Walmart exemplifies aggressive data utilization by altering website prices 10,000 times daily to maintain an 80th percentile low-price aspiration, while retail and industrial sectors aim to drive growth and product efficiency through customer exposure clusters and automated decisions.
- Global data creation rates are accelerating such that companies generate more data in a single day than in the previous 30 years, with specific regional variances in maturity, such as China's development of social credit scores and real-time public security reporting by 2020 versus the US and Europe.
- Privacy regulations in Europe are expected to become stricter with fines reaching up to 4% of revenues for data loss, creating a complex landscape where varying country laws hinder global data flow and force a focus on consumer-centric data control.
- Emerging business models include SaaS platforms for maintaining customer relationships, specialized services transforming messy data into insights, and data governance as a top priority for companies with over a billion dollars in revenue to address legal ambiguity and enforce provenance.
- Strategic opportunities exist in healthcare due to record silos, in developing markets where users trade data access for services like free phones in India, and in the "left tail" risk management mindset prevalent in Europe compared to the growth focus in San Francisco.
- Trust remains a critical barrier where technical experts differ from business users, and while consumers can block ads or remove cookies, they cannot guarantee removal from data lists once shared, requiring institutions to build 360-degree consumer views integrating mobile, credit, and context data.
- Future prospects include automating decision workflows to win business through actionable data, with large legacy firms potentially growing slower due to inertia while startups and smaller entities leverage agility, though real machine learning with curiosity remains a distant goal.