I graded my own ML option forecasts. Here's the Brier score.
In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade Helium's published prob_itm forecasts when the June 20…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
tutorial
typescript
webdev
Meta-Analysis: Between-Study Heterogeneity
Adapted from an appendix of my MS thesis. Between-Study Heterogeneity The extent to which true effect sizes vary within a meta-analysis is called betw…
Gaussian Process Classification
Adapted from an appendix of my MS thesis. Classification We have considered regression problems where the targets are real valued. Classification prob…
Markov Chain Monte Carlo: Theoretical Foundations
Adapted from an appendix of my MS thesis. Markov Chain Monte Carlo Almost as soon as computers were invented, they were used for simulation. Markov ch…
Modeling the Expected Value of a Sealed Card Box (and Where the Number Quietly Lies)
A friend messaged me a photo of a sealed booster box last month with one question: "worth it?" He'd already decided, really. The chase card in that se…
When SuSiE Says '95% Confident', Is It?
Benchmarking the Honesty of Fine-Mapping Credible Sets Fine-mapping has a promise built into its output, and almost nobody checks whether the promise …
Your bandit's exploration floor probably violates its own floor
Most multi-armed bandit / A-B allocation systems add a minimum exploration weight : every arm should get at least, say, 5% of traffic, so no variant i…
A model with R-squared near 0 can still give valid 90% prediction intervals - here's why (and the catch)
I recently calibrated a recovery-rate model that had only two weak features. Its point accuracy was almost nothing — R² basically zero. I expected its…
Conformal prediction silently breaks under drift - and how to make it hold
Conformal prediction is the easiest way to put a calibrated uncertainty band around any model: wrap a point predictor, and you get intervals with a fi…
Stop Shipping ML Models With Bare Floats: A Deep Dive Into Statistically Rigorous Model Evaluation
Stop Shipping ML Models With Bare Floats Every week, somewhere, a team makes a deployment decision that looks like this: Model A: AUROC = 0.847 Model …
Power analysis for LLM evals: how big does your eval set need to be to catch a 5% regression?
TL;DR: Most eval sets are sized by "what we had lying around", not by what they can actually detect. If your eval set is 50 traces and you are trying …
A Scatter-Plot Explorer for World Statistics — Log Scales and Hand-Rolled Pearson Correlation
"Do countries with higher GDP per capita also have longer life expectancy?" I built a tool that lets you explore questions like that across 48 countri…
We put confidence intervals on our LLM-judge scores. The error bars ate three weeks of "trend"
We track weekly agreement between an LLM judge and human labels (Cohen's kappa) on a sample of production traces. For three weeks the point estimates …
Review: A Symbolic Representation of Time Series, with Implications for Streaming Algorithms
In [1], the authors present a method for constructing a symbolic (nominal) representation for real-valued time series data. A symbolic representation …
Graduate Statistics Problem Sets
I put my coursework from SIUe's Master's in Mathematics program up on the problem sets section of this site. Five courses from 2021-2022 that formed t…
Model Selection for Weibull Series Systems: When Simpler Models Suffice
When can you safely use a simpler model for a series system? I ran extensive simulation studies with likelihood ratio tests to get a quantitative answ…
why Cohen's kappa drifts week to week (and what to do about it)
If your LLM-as-judge calibration kappa moves around week to week and you cannot explain it from labeller behavior, the usual cause is the marginal dis…
I Built CausalLens — A Free, Open-Source Causal Impact Calculator for Time Series (5 Methods, Zero Setup)
I want to show you a tool I just open-sourced. It's called CausalLens, and it answers one specific question that most analytics stacks get completely …
Cointegration and Pairs Trading: When Time Series Move Together
Pairs trading rests on a simple idea: find two assets that move together, wait for them to diverge, and bet on convergence. The hard part is defining …
We built a mahjong dangerous-tile predictor calibrated on 4.97M real hands
The problem At a real mahjong table, software can't help you mid-hand. The question — "is this tile safe to discard?" — comes up every few turns, and …
Understanding Correlation in PHP: Pearson vs Spearman vs Kendall Tau
Correlation helps you understand whether two variables move together and how strongly they are related. In this article, you'll learn how Pearson, Spe…
The AI audit rep-curve: why 1 run gives you 67 percent reliability
For most of 2025, the standard AI-search audit I saw from peer agencies looked the same: run a list of prompts once each, screenshot the outputs, code…
Lending's Old Faithful: How a 1958 Breakthrough Still Holds Off the AI Rush
Series: Building an Explainable AI Underwriter This is Part 2 . ↜ Part 1: Why Explainable AI Matters in Underwriting ↝ Part 3: Human Intervention Thre…