TensorFlow Tutorial (Sherry Moore, Google Brain)

Introduction and Team Context

• Sherry Moore (Google Brain) delivers a tutorial on TensorFlow, designed to transition users from research to production.
• Wolf G verified installations across all platforms; product manager Zach is present for feature requests.
• The session aims to provide attendees with tools to build models for image recognition, audio, and art generation.
• TensorFlow has over 32,000 GitHub stars, 14,000 forks, and 8,000 contributions from 400 developers since its open-source release in November.
• The library is optimized for asynchronous, event-driven models where computation fires when data is ready.

Core Architecture and Design Philosophy

• TensorFlow separates the front-end (graph construction in C++, Python, or other languages) from the core execution system (runtime).
• Data is held in tensors (multidimensional arrays similar to NumPy ndarray).
• The computation graph consists of nodes: oval nodes for computation (e.g., matrix multiplication, convolution) and rectangular nodes for data flow.
• The system is modular, allowing parallel development and easy upgrades while maintaining API stability.
• It supports portability across CPUs, GPUs, TPUs, and mobile devices (iOS, Android, Raspberry Pi).
• TensorFlow is designed to minimize code rewriting, allowing research prototypes to be directly pasted into production environments.

Real-World Google Applications

• Image Recognition: Uses Inception (1,000-class recognition) and ResNet; easily retrained for custom datasets like pets or family members.
• Voice/Text: "Smart Reply" handled over 10% of mobile email responses (as of February); initially learned to reply "I love you" before refining.
• Gaming: Develops agents that learn to play games and generate game scenarios (AlphaGo context).
• Art: Powers "Deep Dream" for image synthesis and captioning.
• Open-source models include Inception, captioning, language models on a billion words, and CIFAR-10 ResNet.

Lab 1: Linear Regression Implementation

• Objective: Predict weights ($W$) and bias ($b$) for a mystery linear equation using input $X$ and output $Y$ samples.
• Key components introduced: Input data definition, inference graph construction, loss function, optimizer (Gradient Descent), and session execution.
• Variables: Stateful nodes (square nodes) hold weights/biases; gradients are applied here during training to minimize loss.
• TensorFlow uses a session to execute the graph, distinguishing it from imperative libraries that run code immediately upon typing.
• Optimizers can be discovered via API documentation or IDE tab-completion (e.g., momentum, Adam).

Lab 2: MNIST Digit Classification

• Goal: Build a neural network to recognize handwritten digits from the National Institute of Standards and Technology (NIST) dataset.
• Network Structure: Two hidden layers plus a linear layer producing logits.
• Key Concepts Taught:
  • Placeholders: Define input structures (images/labels) to allow dynamic data feeding without rebuilding the graph.
  • Checkpoints (Saver): Saves network state (weights/biases) to disk, enabling recovery from interruptions or continuous evaluation.
  • Global Step: Tracks training progress to associate checkpoints with specific iteration counts.
  • Evaluation: Validates the model against a held-out set to ensure performance before deployment.
• Data preprocessing is critical; images must be scaled (0-1 range) to match the MNIST training distribution.

Community and Contribution

• TensorFlow is open source; the team actively welcomes code contributions, model submissions, and feature requests via pull requests or direct contact with the product manager.
• High-level APIs (Keras, tf.contrib, TFLearn, Slim, Pretty Tensor) are built on core APIs but users are encouraged to build custom solutions if needed.
• The "research to production" pipeline allows researchers to share code that production teams can immediately prioritize.

Q&A Summary: Platform Support and Roadmap

• C++ API: Available for inference (e.g., label_image.cc); training libraries in C++ are less complete than Python but improving. Custom Python layers can be converted to C++ for inference.
• Windows/iOS Support: Actively being developed; currently hindered by Bazel build system limitations. Expected once Bazel supports Windows.
• TPUs: Not yet available for external testing; timeline for Google Cloud availability is not set.
• Distributed Storage: Plans exist for integration with Hadoop/HDFS for distributed training, but no specific timeline.
• Mobile Training: Training large models (e.g., Inception) on phones is discouraged due to memory and compute constraints; inference is fully supported on mobile.
• Serving & Languages: TensorFlow Serving currently supports C++ and Python; Go and other language front-ends are in progress or supported via contributions.
• Model Packaging: Models can be compiled into single binaries (constants) for efficient inference in containers or standalone executables.