Logistics
Driving Forces Behind AI
Three key players
Computation
- cloud compute
- GPUs
Data
Web data is a powerful source to get more general intelligence.
Algorithms
Old school algorithms, but then with enough data scales.
Key Trends in AI
- moving away from symbolics: don’t code, learn
- moving away from small networks: deep learning
- LLMs popularizing machine learning
- more ethical AI
Scale Emergence
Capabilities in LMs emerge after certain scale: i.e. there’s a sudden improvement in performance after a while.