Bayesian Modelling with Regression ( From A to Z ) with R

Make sure you fasten your seat belt before you get into this race . I have also added another external link if you want to take your time to do some additional warm up with R.

OK ! You cannot wait to start ! It`s normal ! but how about to go for some jogging and do some push ups and pull ups and get ready for the spectacular journey.

I would talk very briefly about the data that we would work on , the Monotone Regression and Generalized Additive Models in R in non-Bayesian framework , as you guessed correctly , I used R for modelling.

I have also included here and external recourse which in details delineates the two schools of Bayesian methods , Parametric and non-parametric . Please have a cup of coffee , play a nice music and read this and do not forget to have fun also.

In this tutorial , we are getting started to think Bayesian , we build up a scenario ! how about to start to see if we can build a probabilistic model which can predict the height of woman from her weight ? What do we need to know in order to get started.

Things are getting more exciting , we are modelling our very first model in Bayesian and we would wonder which algorithm to choose , what do you think ? Do you prefer " sampling" or "optimizing"?

Now , it seems we are almost done to build up a Bayesian regression model , but , you may wonder what may have been the difference with the non_Bayesian one , in case , if you have an eye for fine details.

Wait a minute ! how can you be sure ? Rene Descartes once said ," I doubt , therefore , I am ." . then I have some doubts about what you are telling me about the things that I really like to hear !

No worries ! if you are that type of guys that look around with doubts , this lecture would be very interesting as we would talk about the "Credible Intervals"

However , if you are more advanced , and have heard about the "confidence interval" before and you wonder , doubt is doubt , then what is the difference between these two , make sure to watch this video , as you are going to see a new world is opening just right in front of you.

Now , you feel comfortable to have some doubts about the inference that you have made and you feel safe to call yourself Bayesian , now that would be the time to quantify your doubts !

Imagine sitting around a table , surrounded by researchers who have been working on the same problem as you , but from a different point of view ! yes, you are right , they are all non_Bayesian ! how to compare your results with them ? who more doubtful ! are your doubts stem from the same nature ? let`s see !   

Can you control your doubts ? Can you decide how much doubt is necessary for a particular situation ? how much narrow you can look into the things around yourself ?

Have you ever thought that the things you know from the past , it may change what you may see in the future ? Do you think that the future is being shaped from the current sate of your mind ?

If you do not believe in this , you may change your idea when you enter into the Bayesian universe , your prior experience and perception may play a role into what may become your perception of future.

Are you ready to change your past belief in order to change your future ? Do you thank that even can be possible ?

Although , it may seem impossible to you , but no worries ! you are a Bayesian , your universe is somehow different .

Have you ever tried to portrait your past and your future  on a canvas with colorful paints and then take a step back , sip at your coffee and dive into a deep reflection ? is there too much contrast ? Do you need to make some changes ?

Have you ever read any science fiction books? have you ever tried in your imagination to build up a machine that can construct human beings ?

If so , are you willing to be the first brave volunteer to replicate yourself ? Is your copy looks as the same as you ? If , there are too much variations and differences then , maybe you need to work more on your machine !

How long ago you watched Sherlock Holmes ? in the world of Bayesian , you may think of yourself as Sherlock Holmes.

“Life can only be understood backwards; but it must be lived forwards.”

― Søren Kierkegaard

I have no words to add , as above saying is all matters with the posterior sampling.

The sky is the limit ! Feel free to take as many samples as you want from the huge , vast ocean of the posterior density ! Are these replicates of you resembles your original image ? take a closer look at your pictures , are they you ? Can you relate to the saying below ?

“It's amazing how a little tomorrow can make up for a whole lot of yesterday.”

Johhn Guare, Landscape of the Body

Do you think that you mastered linearity in your life ? now , forget all about the linearity and let`s be all nonlinear in our assumptions .

"I think readers either love or hate nonlinear storytelling, and it's true that it can be more difficult, both to write and to read. "

Chloe Benjamin

Do you also believe that the non-linearity is the most difficult one ?

All that we need is to be able to have a better view from the top ! can I offer you the histogram ?

“You can’t connect the dots looking forward; you can only connect them looking backwards. So you have to trust that the dots will somehow connect in your future. You have to trust in something – your gut, destiny, life, karma, whatever. This approach has never let me down, and it has made all the difference in my life.”

Steve Jobs

Histogram , did not give a perfect view from the top ? do not give up , you can try boxplot , all for free , do not even bother to pay a dime !

“Perspectives are like batteries. You can see the positive or the negative, and they’ll keep you charged up, if you replace them often enough.”

Curtis Tyrone Jones

For 1000 times , you have been doing the same thing , now , dare to see what is the result of all this hard labor ! 

“I dare do all that may become a man;

Who dares do more, is none”

William Shakespeare, Macbeth

Expand your horizon and extend your probabilistic model from one variate into multiple ones ! Do you have the confidence to do so ? If not sure read the quote below:

“No one can attain infinite wisdom with finite horizons!”

Mehmet Murat ildan

The fact that you looked into the future by a deep dive into the vast ocean of the posterior inference , does not hinder you not to enjoy while you are deep inside and watch the beauty deep inside , do not hold on to the cute horse sea beneath , turn your head and enjoy the beauty surrounding you. Let`s see what your inference on the whole parameters of interest look like.

“There’s nothing more beautiful than the way the ocean refuses to stop kissing the shoreline, no matter how many times it’s sent away.”

Sarah Kay

Do you believe in this that every interaction is an opportunity to learn ?

How would you portrait your interactions on the canvas of your mind ? No worries , if you are not that much imaginative , you can watch this lecture to see at least how to visualize the interactions in R.

What is this obsession with this Monte Carlo ? I am not talking about the casino in Monaco , if you are not sure what I am talking about , you better try to solve analytically some high dimensional integrals , if you get stuck , no worries , the solution is offered at this lecture.

"Nothing is impossible, the word itself says 'I'm possible'!"

Audrey Hepburn

Shadow ! Can you recognize me from my shadow ? Does my shadow tell you all about me ? you seem thoughtful ! But here , I am here to tell you that after watching this lecture you can recognize me even from my shadow . That is the beauty of the Bayesian philosophy.

“I embrace my shadow self. Shadows give depth and dimension to my life. I believe in embracing my duality, in learning to let darkness and light, peacefully co-exist, as illumination.”

Jaeda DeWalt

Are you sure that my shadow is going to tell all about me ? How about to call an emergency room doctor to do a diagnostic test ! Do I need a prescription ?

“Your Shadow is a dark omen, a powerful teacher that reveals to you the places in your life where you are energetically blocked. When you continue to ignore these signs, you perpetuate the cycle of your suffering.”

Mateo Sol, Awakened Empath: The Ultimate Guide to Emotional, Psychological and Spiritual Healing

Who said correlation implies the causation ? Now , let`s quickly get rid off the samples that are correlated.

“My science teacher said that just because two things happened together didn't mean one was because of the other, or as she put it: correlation does not imply causation.”

Ezekiel Kwaymullina, Catching Teller Crow

The backbone ! This would lead us to the summit of the modelling we have pursued so far ! Great job and keep aspiring to go even further.

“Intelligence without ambition is a bird without wings.”

Walter H. Cottingham

Now , we would run a diagnostic test for the importance weights that we sampled before.

We both are fortune tellers ! who can better foresee the future ?

I can give you a tool , a measure , let`s name it measure of the predictive accuracy !

You got confused ! how to calculate it ? wait a minute , can you play the lecture ?

“Fortune-telling was quantum betting, a competitive scrying of variably likely outcomes.”

China Miéville, Kraken

This lecture is an applied method for the efficient approximate leave-one-out cross-validation (LOO) for Bayesian models fit using Markov chain Monte Carlo.

We are getting much closer to reveal who is telling the truth and who is foreseeing future better than the others . Have you forgotten about the importance weights ? Do you still remember how we fitted them into the Pareto distribution ? if so , go back to the previous tutorials and make sure to enjoy to get to know these marvelous techniques.

William Shakespeare, Sonnet 17.

Who will believe my verse in time to come,
If it were filled with your most high deserts?
Though yet heaven knows it is but as a tomb
Which hides your life, and shows not half your parts …

How about the graphical diagnostic tools to compare two predictive models ? can you rely on your keen eyes ? do you prefer the mathematical summary of comparing two models or you trust your visual senses ?

Now that we are almost towards the end of our journey , let us see how we can after choosing the best model harness the splendid predictive power of the fit model . Do not forget have fun !

Now ! we are going to wrap up this incredible journey by plotting the uncertainty , I have always been myself fascinated by the credible regions around the fitted model , how about you ?

“Well, here at last, dear friends, on the shores of the Sea comes the end of our fellowship in Middle-earth. Go in peace! I will not say: do not weep; for not all tears are an evil.”

J.R.R. Tolkien, The Return of the King

“Every meeting led to a parting, and so it would, as long as life was mortal. In every meeting there was some of the sorrow of parting, but in everything parting there was some of the joy of meeting as well.”

Cassandra Clare, Clockwork Princess