Exponential Progress of AI: Moore’s Law, Bitter Lesson, and the Future of Computation

Discussion of exponential progress in AI and computation, including Moore’s Law and the Bitter Lesson by Rich Sutton. This was part of the AI paper club on our Discord. Join here: https://discord.gg/8RwBPRs

Bitter Lesson: http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Slides for this video: https://bit.ly/2T4SeHt
References sheet: https://bit.ly/bitter-lesson
Lex + AI Podcast Discord: https://discord.gg/8RwBPRs

OUTLINE:
0:00 – Overview
0:37 – Bitter Lesson by Rich Sutton
6:55 – Contentions and opposing views
9:10 – Is evolution a part of search, learning, or something else?
10:51 – Bitter Lesson argument summary
11:42 – Moore’s Law
13:37 – Global compute capacity
15:43 – Massively parallel computation
16:41 – GPUs and ASICs
17:17 – Quantum computing and neuromorphic computing
19:25 – Neuralink and brain-computer interfaces
21:28 – Deep learning efficiency
22:57 – Open questions for exponential improvement of AI
28:22 – Conclusion

CONNECT:
– Subscribe to this YouTube channel
– Twitter: https://twitter.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– Medium: https://medium.com/@lexfridman