Writing

Part 2: Exact Inference

Part 2: Exact Inference

DJ Rich

Given a Probabilistic Graphical Model, exact inference algorithms exploit factorization and caching to answer questions about the system it represents.

Part 1: An Introduction to Probabilistic Graphical Models

Part 1: An Introduction to Probabilistic Graphical Models

DJ Rich

Probabilistic Graphical Models are born from a remarkable synthesis of probability theory and graph theory. They are among our most powerful tools for managing nature's baffling mixture of uncertainty and complexity.

Bias-Variance Trade-Off

Bias-Variance Trade-Off

DJ Rich

The bias-variance trade-off is a rare insight into the challenge of generalization.

Information Theory and Entropy

Information Theory and Entropy

DJ Rich

Entropy and its related concepts quantify the otherwise abstract concept of information. A tour reveals its relationship to information, binary encodings and uncertainty. Most intuitively, we're left with a simple analogy to 2D areas.

Generalized Linear Models

Generalized Linear Models

DJ Rich

A Generalized Linear Model, if viewed without knowledge of their motivation, can be a confusing tool. It's easier to understand if seen as a two knob generalization of linear regression.

Jensen's Inequality

Jensen's Inequality

DJ Rich

A visual makes Jensen's Inequality obvious and intuitive.

The Fisher Information

The Fisher Information

DJ Rich

The Fisher Information quantifies the information an observation carries for a parameter. The quantification becomes intuitive once we see it measuring a certain geometric quality.

The Copula and 2008

The Copula and 2008

DJ Rich

The copula provides a clever means for mixing and matching a set of marginal distributions with the joint-only mechanism of a joint distribution. However, its elegance and utility have been a dangerous lure.

The Matrix Inversion Lemma

The Matrix Inversion Lemma

DJ Rich

The Matrix Inversion Lemma looks intimidating, but it is easy to know when it applies. Doing so offers considerable computational speed ups.

Bayesian Optimization

Bayesian Optimization

DJ Rich

When optimizing a slow-to-evaluate and non-differentiable function, one may think random sampling is the only option--a naive approach likely to disappoint. However, Bayesian optimization, a clever exploit of the function assumed smoothness, disconfirms these intuitions.