How accurately can complex option trades be signed? First grading against exchange truth
To build a dealer book from the options tape you must decide, print by print, who bought. For simple orders the quote rule does that. But on the day…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
- игры
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
Correlation Is Pairwise. Multicollinearity Isn't.
Almost every regression project I've seen in the last decade starts the same way. Load the data, df.corr() , plot the heatmap, scan for red squares,…
Running Multiple A/B Tests at Once: When Overlap Is Safe
Every experimentation program hits this question within its first few months: test A is running on the homepage, the checkout team wants to launch t…
Checking your A/B test every morning makes it lie to you
You launch the test on Monday. You check the dashboard each morning, because of course you do. On Thursday it reads p = 0.04, you call it, and you s…
Multi-Armed Bandit Testing: How It Works and When to Use
A multi-armed bandit is an algorithm that decides how to split traffic across several variations while a test is still running, shifting more visito…
How to Choose a Minimum Detectable Effect (MDE)
You have opened your sample-size calculator, entered your baseline conversion rate, kept the defaults for significance and power, and now the calcul…
You Don't Need a Math PhD for Data Science — You Need to Stop Skipping the Boring Step
Ask someone switching into data science what worries them and it's almost always the same thing: do I know enough math? So they spend weeks on gradi…
I read the metric libraries of five widely-used eval tools. The metric was never the hard part.
Every LLM eval tool sells you the same headline: a big bag of ready-made metrics. Fifty of them. Seventy. Pick one, call evaluate(), get a number. T…
Your eval monitor fired on four days this week. At your sample size, that was the most likely count
A team I compared notes with recently runs continuous evals on production traffic: an LLM judge scores a sample of traces, the scores land on a dash…
Part 6 - STATISTICS
Bernoulli, Binomial Distribution & Central Limit Theorem Explained from First Principles When I first started learning probability, I came acros…
Your A/B test has three goals and they disagree. Now what?
Every A/B testing tutorial ends the same way: run the test, wait for significance, ship the winner. Then you run a real test and variant B converts…
Part 5 - STATISTICS
Non-Gaussian Distributions Explained from First Principles (Beginner Friendly) As we all know, the real-world dataset is not normalized , but most o…
Your eval's confidence interval assumes independent examples. Yours are clustered.
Every binomial confidence interval you have ever computed on an eval pass rate, Wald, Wilson, Clopper-Pearson, all of them, rests on one assumption:…
An LLM judge is a biased instrument, not a measurement
Last month I shipped an eval that ranked two prompt variants. Variant A won by four points. A teammate reran the same eval the next morning and Vari…
I graded my own ML option forecasts. Here's the Brier score.
In May I promised — out loud, on the internet, where people can screenshot you — that I'd grade Helium's published prob_itm forecasts when the June…
Meta-Analysis: Between-Study Heterogeneity
Adapted from an appendix of my MS thesis. Between-Study Heterogeneity The extent to which true effect sizes vary within a meta-analysis is called be…
Gaussian Process Classification
Adapted from an appendix of my MS thesis. Classification We have considered regression problems where the targets are real valued. Classification pr…
Markov Chain Monte Carlo: Theoretical Foundations
Adapted from an appendix of my MS thesis. Markov Chain Monte Carlo Almost as soon as computers were invented, they were used for simulation. Markov…
Modeling the Expected Value of a Sealed Card Box (and Where the Number Quietly Lies)
A friend messaged me a photo of a sealed booster box last month with one question: "worth it?" He'd already decided, really. The chase card in that…
When SuSiE Says '95% Confident', Is It?
Benchmarking the Honesty of Fine-Mapping Credible Sets Fine-mapping has a promise built into its output, and almost nobody checks whether the promis…
Your bandit's exploration floor probably violates its own floor
Most multi-armed bandit / A-B allocation systems add a minimum exploration weight : every arm should get at least, say, 5% of traffic, so no variant…
A model with R-squared near 0 can still give valid 90% prediction intervals - here's why (and the catch)
I recently calibrated a recovery-rate model that had only two weak features. Its point accuracy was almost nothing — R² basically zero. I expected i…
Conformal prediction silently breaks under drift - and how to make it hold
Conformal prediction is the easiest way to put a calibrated uncertainty band around any model: wrap a point predictor, and you get intervals with a…
Stop Shipping ML Models With Bare Floats: A Deep Dive Into Statistically Rigorous Model Evaluation
Stop Shipping ML Models With Bare Floats Every week, somewhere, a team makes a deployment decision that looks like this: Model A: AUROC = 0.847 Mode…
Power analysis for LLM evals: how big does your eval set need to be to catch a 5% regression?
TL;DR: Most eval sets are sized by "what we had lying around", not by what they can actually detect. If your eval set is 50 traces and you are tryin…
A Scatter-Plot Explorer for World Statistics — Log Scales and Hand-Rolled Pearson Correlation
"Do countries with higher GDP per capita also have longer life expectancy?" I built a tool that lets you explore questions like that across 48 count…
We put confidence intervals on our LLM-judge scores. The error bars ate three weeks of "trend"
We track weekly agreement between an LLM judge and human labels (Cohen's kappa) on a sample of production traces. For three weeks the point estimate…
Review: A Symbolic Representation of Time Series, with Implications for Streaming Algorithms
In [1], the authors present a method for constructing a symbolic (nominal) representation for real-valued time series data. A symbolic representatio…
Graduate Statistics Problem Sets
I put my coursework from SIUe's Master's in Mathematics program up on the problem sets section of this site. Five courses from 2021-2022 that formed…
Model Selection for Weibull Series Systems: When Simpler Models Suffice
When can you safely use a simpler model for a series system? I ran extensive simulation studies with likelihood ratio tests to get a quantitative an…