Michael Jordan

Distinction of Inferential Goals in Big Data

• Particle physics (e.g., Higgs boson detection) relies on strict hypothesis testing with small Type I and Type II errors, utilizing massive data volumes for definitive yes/no conclusions.
• Exploratory fields like genomics and astronomy involve "zillions of hypotheses," prioritizing False Discovery Rate (FDR) control where 90% accuracy is often acceptable.
• Technology-driven personalization models require controlling the $L_\infty$ norm (worst-case error) rather than the $L_2$ norm, as poor service for even a small minority of users leads to customer churn and reputational damage.

Engineering and Statistical Mismatch

• Current industry practice involves "building software and hoping it works" due to a lack of fundamental principles for systems that must maintain constant latency while handling exponentially growing data ($n$).
• Computer science complexity theory (e.g., $O(n^3)$, $O(n \log n)$) and statistical decision theory (risk, dimension) currently operate as "oil and water" with no unified framework linking computational runtime to statistical risk.
• Future systems must integrate "computational thinking" (modularity, abstraction, robustness) with "inferential thinking" (sampling patterns, robustness) to solve grand challenges like scalable, privacy-preserving personalization.

Privacy and Inference Integration

• Current differential privacy theory focuses on descriptive statistics (protecting specific database records) rather than statistical inference (protecting inferences about the underlying population generating the data).
• New minimax theory developed by the speaker and colleagues (Duchi, Wainwright) unifies statistical decision theory with differential privacy.
  • Key finding: The effective sample size required for differential privacy scales with the data points multiplied by the privacy parameter $\alpha^2$, divided by the problem dimension.
• This framework provides equations where the number of data points, privacy constraints, and statistical dimension are explicitly balanced.

The "Bag of Little Bootstraps" Solution

• Problem: Standard frequentist bootstrapping requires resampling the entire dataset (e.g., 1 terabyte) with replacement thousands of times, creating prohibitive I/O bottlenecks (approx. 632 GB per resample) that prevent parallelization at scale.
• Solution: The "Bag of Little Bootstraps" algorithm subsamples the large dataset into small fragments ($B \ll N$) and performs full bootstrapping on these fragments.
  • Theoretical justification: The distribution of statistics from subsampled data converges to the distribution of the full population without requiring $B=N$.
  • Computational advantage: Allows massive parallelization across cloud nodes (e.g., Spark) by processing tiny footprints with full resampling counts.
• Performance Results:
  • Standard sequential bootstrap on 1 TB data (Amazon EC2) required ~15,000 seconds to achieve error bar stability.
  • Bag of Little Bootstraps achieved superior error bar quality in approximately 200–300 seconds.
• Implication: Frequentist error bars provide necessary uncertainty quantification for real-world decisions (e.g., medical diagnostics) without the sensitivity to priors required by Bayesian methods, and the new algorithm makes this feasible for big data.