#144 Why is Bayesian Deep Learning so Powerful, with Maurizio Filippone

Sign up for Alex's first live cohort, about Hierarchical Model building! Get 25% off "Building AI Applications for Data Scientists and Software Engineers"

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch! Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work! Visit our Patreon page to unlock exclusive Bayesian swag ;) Takeaways: Why GPs still matter: Gaussian Processes remain a go-to for function estimation, active learning, and experimental design – especially when calibrated uncertainty is non-negotiable. Scaling GP inference: Variational methods with inducing points (as in GPflow) make GPs practical on larger datasets without throwing away principled Bayes. MCMC in practice: Clever parameterizations and gradient-based samplers tighten mixing and efficiency; use MCMC when you need gold-standard posteriors. Bayesian deep learning, pragmatically: Stochastic-gradient training and approximate posteriors bring Bayesian ideas to neural networks at scale. Uncertainty that ships: Monte Carlo dropout and related tricks provide fast, usable uncertainty – even if they’re approximations. Model complexity ≠ model quality: Understanding capacity, priors, and inductive bias is key to getting trustworthy predictions. Deep Gaussian Processes: Layered GPs offer flexibility for complex functions, with clear trade-offs in interpretability and compute. Generative models through a Bayesian lens: GANs and friends benefit from explicit priors and uncertainty – useful for safety and downstream decisions. Tooling that matters: Frameworks like GPflow lower the friction from idea to implementation, encouraging reproducible, well-tested modeling. Where we’re headed: The future of ML is uncertainty-aware by default – integrating UQ tightly into optimization, design, and deployment.

Chapters: 08:44 Function Estimation and Bayesian Deep Learning 10:41 Understanding Deep Gaussian Processes 25:17 Choosing Between Deep GPs and Neural Networks 32:01 Interpretability and Practical Tools for GPs 43:52 Variational Methods in Gaussian Processes 54:44 Deep Neural Networks and Bayesian Inference 01:06:13 The Future of Bayesian Deep Learning 01:12:28 Advice for Aspiring Researchers