Machine Learning and Other Topics

TrueSkill Part 1: The Algorithm

DJ Rich

Posted: February 18, 2025

A walkthrough of the classic, effective and beautiful TrueSkill algorithm.

Algorithmic Operations: Lessons from Bandit Algorithms

DJ Rich, and Howe Wang

Posted: November 18, 2024

Taking inspiration from bandit algorithms, we propose a framework for effective algorithmic operations.

When does the Delta Method approximation work?

DJ Rich

Posted: November 01, 2023

I explore when the Delta Method approximation works and fails.

Bias-Variance Trade-Off

DJ Rich

Posted: July 08, 2018

The bias-variance trade-off is a rare insight into the challenge of generalization.

Information Theory and Entropy

DJ Rich

Posted: June 18, 2018

Entropy and its related concepts quantify the otherwise abstract concept of information. A tour reveals its relationship to information, binary encodings and uncertainty. Most intuitively, we're left with a simple analogy to 2D areas.

Generalized Linear Models

DJ Rich

Posted: June 12, 2018

A Generalized Linear Model, if viewed without knowledge of their motivation, can be a confusing tool. It's easier to understand if seen as a two knob generalization of linear regression.

Jensen's Inequality

DJ Rich

Posted: May 31, 2018

A visual makes Jensen's Inequality obvious and intuitive.

The Fisher Information

DJ Rich

Posted: May 16, 2018

The Fisher Information quantifies the information an observation carries for a parameter. The quantification becomes intuitive once we see it measuring a certain geometric quality.

Bayesian Optimization

DJ Rich

Posted: April 12, 2018

When optimizing a slow-to-evaluate and non-differentiable function, one may think random sampling is the only option--a naive approach likely to disappoint. However, Bayesian optimization, a clever exploit of the function assumed smoothness, disconfirms these intuitions.

The Exponential Family

DJ Rich

Posted: March 18, 2018

The exponential family is a generalization of distributions, inclusive of many familiar ones plus a universe of others. The general form brings elegant properties, illuminating all distributions within. In this post, we discuss what it is, how it applies and some of its properties.