Mathematical Approaches to Image Processing with Carola Schönlieb

Research Background and Evolution

• The researcher began in Vienna studying partial differential equations (PDEs) modeling natural phenomena in physics and biology, specifically the Cahn-Hilliard equation for phase separation in metallic alloys.
• Early work focused on stability analysis of stationary states, determining how systems react to perturbations in initial conditions.
• A pivotal moment occurred while reading about UCLA researchers (Andrea Bertozzi's group) applying the Cahn-Hilliard equation to image restoration, predating commercial tools like Photoshop's "content-aware fill."
• The researcher's PhD shifted focus to image restoration, maintaining a reliance on differential equations but applying them to replace damaged image regions.
• Postdoctoral research expanded into "inverse imaging problems," where direct images are not observed but must be reconstructed from transformed measurements (e.g., X-ray projections in CT or MRI).

Technical Methodologies: Inverse Problems and Denoising

• Data Limitations: Inverse problems face inherent data scarcity; high-resolution reconstruction requires many measurements (e.g., X-ray line integrals), which conflicts with patient safety constraints in medical imaging regarding radiation exposure.
• Noise Integration: Denoising is integrated directly into the reconstruction algorithm rather than treated as a separate post-processing step.
• Edge Preservation: Traditional handcrafted algorithms (e.g., Total Variation regularization, Median filtering) prioritize preserving sharp edges (high-frequency discontinuities) over blurring them, contrasting with Fourier-based methods that remove high frequencies indiscriminately.
• Handcrafted vs. Neural Networks:
  • Deep neural networks now often outperform handcrafted models in denoising for images similar to their training data.
  • Neural networks struggle with generalization when presented with data types unseen during training (e.g., a network trained on animal photos failing on CT scans).
  • Handcrafted models retain value for interpretability and mathematical guarantees, such as provable stability against perturbations.
• Adversarial Robustness: Neural networks are susceptible to adversarial errors where small, consistent perturbations across different scanner models can cause catastrophic classification failures.
• Current Research Direction: The researcher is exploring "bi-level optimization" to parametrize handcrafted models with a small number of learnable parameters (e.g., 10 vs. millions), retaining interpretability while leveraging data-driven learning.

Practical Applications and Collaborations

• Biomedical Imaging:
  • MRI & CT: Collaborations with Addenbrooke's Hospital (Cambridge) and university clinicians focus on maximizing resolution from limited data in dynamic processes (time-varying objects).
  • Chemical Engineering: Partnering with the Magnetic Resonance Research Center to model dynamic fluid processes in tubes, requiring high temporal and spatial resolution with minimal measurement data.
• Environmental Science:
  • Forest Health: Collaborations with plant scientists using airborne hyperspectral and multispectral imaging (200+ light spectrum channels) to identify material properties and detect invasive species.
  • 3D Modeling: Utilizing LiDAR measurements to generate 3D models of trees from flight data, rather than relying on standard photography.
• Art History and Conservation:
  • Virtual Restoration: Partnerships with the Fitzwilliam Museum allow for "virtual restoration" of fragile illuminated manuscripts, creating digital templates to show original states or remove overpaint without physical intervention.
  • Exhibition Success: A 2023 exhibition titled "Color" showcased a page where digital processing successfully removed manual overpainting, presenting the original and restored versions side-by-side.

Theoretical Challenges and Future Directions

• Optimization Strategies: Training neural networks often employs stochastic optimization (randomly sampling subsets of data) to prevent overfitting to the finite training set and improve generalization to unseen infinite distributions.
• Structural Priors: A major research goal is introducing mathematical structure into neural networks to enable error estimates and stability proofs currently unavailable in "black box" deep learning models.
• Computational Scale: Work is conducted sequentially to manage computational loads, processing data bit-by-bit rather than feeding massive datasets simultaneously.
• Limitations of Enhancement: While machine learning can enhance low-resolution CCTV footage, researchers caution that such upscaling relies on probabilistic matching and cannot guarantee factual accuracy or reveal true details.
• Resource Recommendations: The researcher advises prospective entrants to study foundational work from UCLA (e.g., S. Osher, M. Perona, S. Sapiro) and review classical introductory texts before engaging with modern research.