Summary
Highlights
Professor Fei-Fei Li introduces CS231N, highlighting the interdisciplinary nature of AI, especially computer vision and deep learning. She emphasizes that vision is a cornerstone of intelligence and discusses how the course will focus on the core intersection of computer vision and deep learning, touching upon its connections to various fields like natural language processing, robotics, mathematics, and even medicine and law.
The lecture delves into the deep history of vision, starting 540 million years ago with the Cambrian explosion and the onset of sight in early animals. It explains how vision drove the evolution of intelligence and complex nervous systems. The discussion then fast-forwards to human civilization, examining early efforts to build 'seeing machines' by thinkers like Leonardo da Vinci and the fundamental challenge of recovering 3D information from 2D images, an ill-posed problem nature solved with multiple eyes.
The historical journey continues with key milestones in computer vision, including the seminal neuroscience experiments by Hubel and Wiesel in the 1950s, revealing hierarchical processing in the visual cortex. The first PhD thesis in computer vision by Larry Roberts in 1963 and an ambitious MIT summer project aiming to 'solve computer vision' are also discussed. The 1970s saw David Marr's systematic approach to visual processing, proposing stages from primal sketch to 3D representation. Despite these efforts, early computer vision struggled with limited data and computational power, leading to an 'AI winter' in the 1980s.
During the AI winter, cognitive and neuroscience continued to advance, providing crucial insights into human visual processing. Studies showed the importance of context in scene understanding and the remarkable speed of human object detection (150 milliseconds for categorization). These findings highlighted the need for computer vision to focus on fundamental problems like object recognition in natural settings. This era saw pioneering work in object recognition, like generalized cylinders and face detection, which eventually found practical applications in digital cameras.
Concurrently with computer vision's development, a separate thread of research in neural networks was evolving. Early concepts like perceptrons and Fukushima's hand-designed Neocognitron laid the groundwork. The breakthrough of backpropagation in 1986 provided a principled learning rule, enabling more effective training of neural networks. Yann LeCun's convolutional neural network (LeNet-5) in the 1990s demonstrated practical applications, but deep learning still faced limitations due to a lack of data. The creation of ImageNet with 15 million images across 22,000 categories, and the associated challenge (ILSVRC), addressed this data scarcity. The 2012 ImageNet challenge, won by AlexNet, marked the rebirth of modern AI and the deep learning revolution, showcasing the power of convolutional neural networks with sufficient data and computational resources (GPUs).
Since 2012, deep learning has exploded, leading to significant advancements in computer vision, including object detection, image segmentation, video classification, and multimodal understanding. Applications span medical imaging, scientific discovery, and environmental monitoring. The lecture also highlights the emergence of generative AI, with examples like image captioning, style transfer, and advanced image generation (DALL-E, Midjourney). The exponential growth in computational power (driven by GPUs) and data, coupled with algorithmic advancements, has propelled AI into a 'global warming period.' The presenter also touches upon the societal implications of AI, including bias in algorithms and ethical considerations, but expresses excitement for applications in medicine and healthcare, emphasizing the continued nuance and complexity of human vision.
Professor Adelli outlines the course structure, divided into four main topics: deep learning basics, perceiving and understanding the visual world, large-scale distributed training, and generative/interactive visual intelligence, concluding with human-centered applications. The initial lectures will cover fundamental concepts like linear classifiers, regularization, and optimization, leading into detailed discussions of neural networks, including CNNs, RNNs, and transformers. The course will explore advanced tasks like semantic segmentation, object detection, and video understanding, and delve into self-supervised learning, generative models (diffusion models), vision-language models, and 3D vision. Learning objectives include formalizing computer vision tasks, developing and training vision models, and understanding the field's current state and future trajectory. The next session will focus on image classification and linear classifiers.