Conference Presentation, Keynote, Panel
Matthew Turk
- The speaker expects a limited audience understanding of augmented reality (AR) distinctions from virtual reality, predicting widespread public recognition via familiar examples like the digital "first down" marker on football broadcasts.
- AR capabilities formerly attributed to superheroes, such as x-ray vision and teleportation, are anticipated to become achievable, with applications projected across medicine, dentistry, retail, gaming, military, and entertainment.
- Specific medical use cases include enabling professionals to visualize internal anatomy to assess surgical readiness and identifying dental needs, while a primary project titled "Being There" aims to facilitate remote physical collaboration.
- The "Being There" system facilitates "teleportation" for experts to virtually assist novices, requiring independent viewpoints and shared spatial reference frames while explicitly excluding physical manipulation of the remote environment.
- Mobile telecollaboration is designed to be unobtrusive and device-agnostic, utilizing common smartphones to capture 3D scene models for remote experts who can explore, zoom, and freeze viewpoints.
- The system architecture merges SLAM for 3D data with panoramic tracking for 2D data, automatically transitioning between 3D motion (essential matrix) and 2D motion (homography) to create a hybrid model for navigation and annotation.
- Annotation rendering challenges are addressed by moving beyond static "spray paint" models to a "gesture enhanced by surface normal" approach that infers user intent to ensure spatial correctness across different perspectives.
- Research validation involved 88 users on Amazon Mechanical Turk, confirming that hybrid models and intent-inferred rendering support 3D modeling, remote navigation, and world-stabilized annotations.
- Current technical limitations include handling dynamic scenes and non-rigid objects, which remain unsolved challenges for the framework's evolution.
- A commercialization effort via a startup named Cognate targets the U.S. field service sector, which comprises over 5.5 million technicians facing a demographic shift toward inexperienced workers and aging experts.
- Economic projections suggest the technology can reduce equipment downtime to save tens or hundreds of thousands of dollars daily, while decreasing travel costs and increasing customer satisfaction.
- The long-term business strategy initiates with business-to-business market penetration, with subsequent phases planned for business-to-consumer and consumer-to-consumer remote collaboration.