PPP | Probabilistic Programming Primer | Bayesian Stats | PyMC3

A course by Peadar Coyle, a core PyMC3 contributor and founder of Aflorithmic Labs
Introduction
An Introduction to Probabilistic Programming
29 mins
A modern Bayesian Workflow.pdf
1.33 MB
Slack Channel Link
Github Repository for code
Hands on introduction to PyMC3
In this section I'll look at some real world techniques with PyMC3

  • AB Testing
  • Change point detection 
  • Installing PyMC3.

AB testing is a classical technique in modern day e-commerce or marketing analytics jobs. Alternatively it can be useful in biology or chemistry or any experimental design process. 

Change point detection is useful in fraud detection, electricity markets modelling, process analysis and in dev ops - predicting when systems will get overloaded in data centres or in the cloud.
How do I install PyMC3?
How do I do Bayesian AB Testing? - Probabilistic Programming Primer -
Probabilistic Programming Primer: Bayesian Changepoint Detection
Bayesian_AB_Testing.ipynb
184 KB
Case Study 1- Bayesian_Changepoint_Detection.ipynb
194 KB
How to build a Logistic Regression model the Bayesian way
10 mins
1.5 - Puppy Steps_version_2.mp4
5 mins
Hands on introduction to PyMC3: Quiz
Notes_on_Chains.pdf
128 KB
What is PyMC3 and how do I get started?
Firstly we need to introduce, what is PyMC3?

In this section we'll explain what the key syntax is, and how to use PyMC3. 

What is PyMC3


This section will take us through what PyMC3 is. In particular we'll talk about how to install/ use and get up and running with this library. 

We'll touch on 
  • What Bayesian Statistics and Probabilistic Programming are
  • What MCMC algorithms are
  • What use cases in industry are improved by a better understanding of uncertainty.

In the rest of the course we'll dive deeper into each of these topics.
1. Introduction to PyMC3.ipynb
305 KB
pymc3_compared_deep_learning.pdf
64 KB
ppp_intro_to_pymc3_1
2 mins
ppp_intro_to_pymc3_1 Edited.mp4
5 mins
ppp_intro_to_pymc3_2 Edited.mp4
2 mins
ppp_intro_to_pymc3_3 Edited.mp4
9 mins
ppp_intro_to_pymc3_4 Edited.mp4
7 mins
ppp_intro_to_pymc3_5_version
5 mins
What is MCMC?
An explanation of what MCMC (Markov-Chain-Monte Carlo) is and why you should care. I'll also explain things like what NUTS (No-U-Turn-Sampler) is and this will inform our future work on how to diagnose model performance. 

Why should you care?

The underlying machinery of Probabilistic Programming is either a variational inference approach (largely out of scope for this course, but we'll talk about it) or a MCMC sampling approach. Like in any applied statistics topic you do need to have an understanding of how the underlying machinery works. In particular to debug it. 

This section of the course provides an applied and practical focus on what MCMC samplers are. There'll be some Math but it'll be hacker focused. 

Curriculum

  • Metropolis sampling
  • Gradient-based sampling methods
2. Markov Chain Monte Carlo.ipynb
23.1 KB
3.1_what_is_mcmc.mp4
4 mins
3.2_what_is_mcmc.mp4
6 mins
3.3 - What is MCMC_version_2.mp4
5 mins
3.4 - What is MCMC_version_2.mp4
4 mins
3.5 - What is MCMC_version_2.mp4
5 mins
3.6 - What is MCMC_version_2.mp4
3 mins
Introduction to Theano
PyMC3 is based upon Theano, so it's useful to have some familiarity with it when doing work. 

Theano is a package that allows us to define functions involving array operations and linear algebra. When we define a PyMC3 model, we implicitly build up a Theano function from the space of our parameters to their posterior probability density up to a constant factor. We then use symbolic manipulations of this function to also get access to its gradient.

This part of the course will discuss how to use theano, how to debug it and some tricks for handling some of the idiosyncrasies of Theano. 
3. Theano.ipynb
403 KB
4.1 - Intro to Theano_version_2.mp4
5 mins
4.2 - Intro to Theano_version_2.mp4
4 mins
4.3 - Intro to Theano_version_2.mp4
5 mins
4.4_Intro_to_Theano_version_2.mp4
6 mins
4.5 - Intro to Theano_version_2.mp4
4 mins
Introduction to Theano: Quiz
Hierarchical Models (Multilevel models)
In this section I provide a case study into a hierarchical model in sports analytics.

I apply Bayesian Statistics to modelling the six nations in Rugby.

This application is a good one for Bayesian statistics, we have priors and not much data - therefore we should be prepared to model in a way that gives us uncertainty quantification - which is Bayesian Statistics. 
5.1 - Rugby Analytics_version_2.mp4
5 mins
ppp_rugby_analytics_2.mp4
12 mins
ppp_rugby_analytics_3.mp4
3 mins
ppp_self_driving_cars.mp4
19 mins
5.4 - Self-Driving Cars.mp4
12 mins
Model building with PyMC3
I explain how to build models with PyMC3 and give you a deep dive into the underlying API. 

The Model Building with PyMC3 course will include 
  • Specifying priors and likelihoods
  • Deterministic variables
  • Factor potentials
  • Custom variables
  • Step methods
  • Generalized linear models
  • Missing Data
Model Building with PyMC3.ipynb
310 KB
6.1 - Model Building.mp4
6 mins
6.2 - Model Building.mp4
7 mins
6.3 - Model Building.mp4
6 mins
6.4 - Model Building.mp4
12 mins
Other Probabilistic Programming Languages and Tools
PyMC3 is just one of many Probabilistic Programming Languages.

The aim with this section is to discuss languages/tools such as 
  • Pyro
  • Stan
  • Edward
  • Tensorflow Probability
  • Arviz

All languages will have examples and explanations of the syntax.
Rainier
Pyro
ppp_rainier.mp4
15 mins
7.1 - Pyro Example.mp4
9 mins
7.2 - ArviZ.mp4
8 mins
Real world examples of Probabilistic Programming
This section will include examples of real-world 
Acknowledgements
Bayesian Decision Making applied to Supply Chain
18 mins
Bayesian Machine Learning
*CONGRATULATIONS ON FINISHING THE COURSE* You can retrieve a certificate by this google form
You finished the course!!!
New functionality - PyMC3 3.9
I decided to do some new lectures on new functionality in PyMC3 3.9. Here I take us through the 'Data' class. Which allows you to better plot your data, and better predict models on unseen data. 
Shared_variables_part_1.mp4
5 mins
Shared_Variables_section_3.mp4
5 mins
Shared_Variables_Section_2.mp4
4 mins