Browsed by
Author: Antoine Rebecq

Rugby World Cup explainer using data

Rugby World Cup explainer using data

Last week, a stereotypical “French” ceremony opened the 10th Rugby World Cup in Stade de France, in the suburbs of Paris, France. As a small boy growing up in the southern half of France, I developed a strong interest for the sport. Now being an adult living and working in North America, where barely anyone has ever heard the word “Rugby”, I now rarely have anyone else to talk to about Antoine Dupont’s (captain of the French team and best…

Read More Read More

Using R to build predictions for UEFA Euro 2020

Using R to build predictions for UEFA Euro 2020

Last friday, Euro 2020, one of the biggest events in International soccer, was kicked off by the inaugural match between Italy and Turkey (Italy won it 3-0). Euros (short for European Championships) are usually held every 4 years, but because of he-who-must-not-be-named, last year’s edition was postponed to this summer, while keeping the name “Euro 2020” (much like the Tokyo Olympics). 4 5 years ago, for Euro 2016, I basically wanted to try some cool methods based on splines on…

Read More Read More

Have you checked your features distributions lately?

Have you checked your features distributions lately?

tl;dr Trying to debug a poorly performing machine learning model, I discovered that the distribution of one of the features varied from one date to another. I used a simple and neat affine rescaling. This simple quality improvement brought down the model’s prediction error by a factor 8 Data quality trumps any algorithm I was recently working on a cool dataset that looked unusually friendly. It was tidy, neat, interesting… the kind of things that you rarely encounter in the wild!…

Read More Read More

Causal Inference cheat sheet for data scientists

Causal Inference cheat sheet for data scientists

Being able to make causal claims is a key business value for any data science team, no matter their size.Quick analytics (in other words, descriptive statistics) are the bread and butter of any good data analyst working on quick cycles with their product team to understand their users. But sometimes some important questions arise that need more precise answers. Business value sometimes means distinguishing what is true insights from what is incidental noise. Insights that will hold up versus temporary marketing…

Read More Read More

The Mrs. White probability puzzle

The Mrs. White probability puzzle

tl;dr -I don’t remember how many games of Clue I’ve played but I do remember being surprised by Mrs White being the murderer in only 2 of those games. Can you give an estimate and an upper bound for the number of games I have played?We solve this problem by using Bayes theorem and discussing the data generation mechanism, and illustrate the solution with R. Making use of external information with Bayes theorem Having been raised a frequentist, I first…

Read More Read More

[Sampling] Big data and sampling in Ottawa

[Sampling] Big data and sampling in Ottawa

Tomorrow (November 7th), I’ll give a talk at the Statistics Canada Symposium on survey sampling and big data. I’ll show how techniques that were developed at official statistics institutes can now be used in the context of big data and machine learning, and add a lot of value. I’ll show some examples with: A/B testing Tracking design Calibration in Machine Learning Network analysis User feedback Bring survey sampling techniques into big data de Antoine Rebecq   And really glad to…

Read More Read More

Bad recommendations, good algorithm

Bad recommendations, good algorithm

If you’ve ever shopped online (*cough* Amazon *cough*), you’ve probably experienced the “vacuum cleaner effect”. You carefully buy one expensive item (e.g. a vacuum cleaner) and then you receive dozens of recommendations for other vacuum cleaners to buy: by email, everywhere on the retailer’s website, or sometimes in the ads you see on other websites. In other terms, Amazon is a 1 trillion dollar company that employs hundreds of data scientists and is incapable of understanding that if you bought…

Read More Read More

Weighting tricks for machine learning with Icarus – Part 1

Weighting tricks for machine learning with Icarus – Part 1

Calibration in survey sampling is a wonderful tool, and today I want to show you how we can use it in some Machine Learning applications, using the R package Icarus. And because ’tis the season, what better than a soccer dataset to illustrate this? The data and code are located on this gitlab repo: https://gitlab.com/haroine/weighting-ml First, let’s start by installing and loading icarus and nnet, the two packages needed in this tutorial, from CRAN (if necessary): install.packages(c(“icarus”,”nnet”)) library(icarus) library(nnet) Then…

Read More Read More