Machine Learning and Other Topics
![When does the Delta Method approximation work?](/assets/images/posts/when_does_the_delta_method_approximation_work/when_does_the_delta_method_approximation_work.png)
When does the Delta Method approximation work?
I explore when the Delta Method approximation works and fails.
![Bias-Variance Trade-Off](/assets/images/posts/bias_variance_tradeoff/BiasVariance.png)
Bias-Variance Trade-Off
The bias-variance trade-off is a rare insight into the challenge of generalization.
![Information Theory and Entropy](/assets/images/posts/entropy_information_theory/cover.png)
Information Theory and Entropy
Entropy and its related concepts quantify the otherwise abstract concept of information. A tour reveals its relationship to information, binary encodings and uncertainty. Most intuitively, we're left with a simple analogy to 2D areas.
![Generalized Linear Models](/assets/images/posts/generalized_linear_models/equations.png)
Generalized Linear Models
A Generalized Linear Model, if viewed without knowledge of their motivation, can be a confusing tool. It's easier to understand if seen as a two knob generalization of linear regression.
![Jensen's Inequality](/assets/images/posts/jensens_inequality/EfX.png)
![The Fisher Information](/assets/images/posts/the_fisher_information/logliksims.png)
The Fisher Information
The Fisher Information quantifies the information an observation carries for a parameter. The quantification becomes intuitive once we see it measuring a certain geometric quality.
![Bayesian Optimization](/assets/images/posts/bayesian_optimization/no_activation.png)
Bayesian Optimization
When optimizing a slow-to-evaluate and non-differentiable function, one may think random sampling is the only option--a naive approach likely to disappoint. However, Bayesian optimization, a clever exploit of the function assumed smoothness, disconfirms these intuitions.
![The Exponential Family](/assets/images/posts/exponential_family/pic_1.png)
The Exponential Family
The exponential family is a generalization of distributions, inclusive of many familiar ones plus a universe of others. The general form brings elegant properties, illuminating all distributions within. In this post, we discuss what it is, how it applies and some of its properties.
![Motivating the Gini Impurity Metric](/assets/images/posts/gini_impurity/p2.png)
Motivating the Gini Impurity Metric
We reveal the gini impurity metric as the destination of a few natural steps.
![The Trace as a Measure of Complexity](/assets/images/posts/trace_as_a_measure_of_complexity/p3.png)
The Trace as a Measure of Complexity
For a class of models, the trace provides a measure of model complexity that's useful for managing the bias variance trade-off.
![A Brief Explanation and Application of Gaussian Processes](/assets/images/posts/a_brief_look_at_gaussian_processes/weight_predicted.png)
A Brief Explanation and Application of Gaussian Processes
A clever and useful technique for inferring distributions over infinite functions using finite observations.