CVPR 2021 Tutorial: All About Self-Driving
CVPR 2021 | Full-Day
|
Zoom Livestream
| Sli.do Q&A
| YouTube Livestream (Part 1)
| YouTube Livestream (Part 2)
Thanks to everyone who attended, and see you all in 2022!
Individual recordings of the sessions and other materials will be made available after CVPR. In the meantime, feel free to check out the (recorded) YouTube live streams above and last year's recordings.
Organizer & Contributors
Bin Yang
Waabi & University of Toronto
Shenlong Wang
University of Toronto
Yuwen Xiong
Waabi & University of Toronto
Simon Suo
Waabi & University of Toronto
James Tu
Waabi & University of Toronto
Joyce Yang
Waabi & University of Toronto
Julieta Martinez
Waabi
Kelvin Wong
Waabi & University of Toronto
Justin Liang
Waabi
Wenyuan Zeng
Waabi & University of Toronto
Lunjun Zhang
Waabi & University of Toronto
Chris Zhang
Waabi & University of Toronto
Sean Segal
Waabi & University of Toronto
Davi Frossard
Waabi & University of Toronto
Sergio Casas
Waabi & University of Toronto
Andrei Bârsan
Waabi & University of Toronto
Siva Manivasagam
Waabi & University of Toronto
Ze Yang
Waabi & University of Toronto
Abbas Sadat
Waabi
Jingkang Wang
Waabi & University of Toronto
Quinlan Sykora
Waabi & University of Toronto
Overview
A full day tutorial covering all aspects of self-driving. This tutorial will provide the necessary background for understanding the different tasks and associated challenges, the different sensors and data sources one can use and how to exploit them, as well as how to formulate the relevant algorithmic problems such that efficient learning and inference is possible. We will first introduce the self-driving problem setting and the existing solutions both top down from a high level perspective and bottom up from technology and algorithm specific manner in a detailed fashion. We will then extrapolate from the state of the art and discuss where the challenges and open problems are, and where we need to head towards to provide a scalable, safe and affordable self-driving solution.
Program & Recordings
All times are in Eastern Daylight Time (EDT).
Each session will have five minutes of Q&A at the end. Please use the sli.do link or the Zoom Seminar to submit questions
Morning (Eastern US Time)
1. Opening & Intro to Autonomy Systems
Speaker: Raquel Urtasun
- Modular approaches
- End-to-end approaches
- Overview of tutorial
2. Hardware
Speakers: Davi Frossard and Andrei Bârsan
- Sensor coverage
- Wheel encoders
- LiDARs
- Cameras
- Microphone
- Ultrasound
- RADAR
- GPS
- Real-Time Kinematic (RTK) Systems
3. Autonomy: Perception
Speaker: Bin Yang
- 3D perception from LiDAR
- 3D perception from Camera
- Sensor fusion (LiDAR, camera, RADAR, maps)
- Output representations
- Open-set perception
- Latency-aware models
4. Autonomy: Prediction
Speakers: Sergio Casas and Simon Suo
How do we forecast the future motion for actors in a scene?- Prediction in the traditional autonomy pipeline
- Learning rich representation of the scene
- Capturing multiple futures
- Scene-consistent prediction
- Instance-free perception & prediction
Break ☕
5. Autonomy: Motion Planning
Speakers: Abbas Sadat and Wenyuan Zeng
- End-to-End (E2E) planning motivation
- Input modalities
- Model architectures
- Output representations
- Learning paradigms
- Interpretable neural planners
- Reactive planning and contingency planning
Afternoon (Eastern US Time)
6. Autonomy: Vehicle-to-Vehicle Communication (V2V)
Speakers: Siva Manivasagam and James Tu
- Vehicle-to-Vehicle perception
- Robustness in Vehicle-to-Vehicle communication
7. Datasets and Metrics
Speaker: Sean Segal
- Self-driving datasets
- Dataset curation (tagging, coverage, active learning)
- Evaluation
Break ☕
8. Simulation
Speakers: Kelvin Wong, Simon Suo, Siva Manivasagam, Ze Yang, and Shenlong Wang
- Asset reconstruction (background, rigid, dynamic)
- Scene Generation
- Traffic Simulation
- LiDAR Simulation
- Image Simulation
9. Adversarial ML
Speakers: James Tu and Jingkang Wang
- Introduction to adversarial examples
- Adversarial robustness in perception systems
- Adversarial robustness in V2V communication
- Generating safety-critical scenarios
Break 🍵
10. Building HD Maps
Speakers: Justin Liang, Anqi Joyce Yang, and Quinlan Sykora
- Topological maps
- HD maps
- Simultaneous Localization and Mapping (SLAM)
- Online mapping
- Multi-agent routing
11. Localization
Speakers: Shenlong Wang, Julieta Martinez, and Andrei Bârsan
Understand how self-driving vehicles robustly establish their precise position within HD maps in order to leverage them for safe and efficient autonomous driving. Topics:- Overview of localization for SDVs
- Probabilistic formulation & Monte Carlo localization
- Online localization with maps
- Joint localization, perception & prediction
- Global localization