Testing

Storm Sensei's whole job is warning you about dangerous weather before it gets to you. That only matters if the warnings are actually accurate — so instead of just claiming they are, we tested them the way a scientist would: we wrote down exactly how we'd measure success before we ran the test, checked our results against real historical storms, and published what we found — including the parts that weren't perfect.

This is a summary of the two independent studies we performed: one on our lightning detection engine, one on our rain detection engine. The full details, including the statistics, are in these papers: Lightning Detection and Rain Detection.

What we tested

Storm Sensei watches for two kinds of danger separately: lightning and heavy rain. We treat them as two different signals — rain can make an alert feel more urgent, but it can never water down a lightning warning. Safety-wise, lightning always leads.

Our AI continuously interprets the raw data in real time and alerts you in plain language when it sees something you should be aware of — but the underlying decision about how serious a threat is never rests on the AI alone. A separate, deterministic safety check (the thing these tests were actually about) sets a hard floor for alert severity based on real-time data like strike distance and storm speed. Our AI only adds context on top of that floor — it can never talk the system into downplaying a real threat.

Where and when

We tested against real storms from five very different parts of the country — coastal Florida, the Colorado high plains, Texas, and Pennsylvania (including Buck Hill Falls, where Storm Sensei currently operates). We picked places with genuinely different storm patterns on purpose, so a strong result in one place couldn't hide a weak spot somewhere else.

For lightning, we checked our system against 295 real, independently confirmed storms and 759 real periods with no storms at all, going back to 2018. For rain, we used that same real-world dataset, cross-checked against actual radar records going back to 2020 (the earliest year public radar archives are available), which left us 178 real rain events and 519 confirmed dry periods to test against.

Why you should trust these tests

A few things we did specifically to make sure this was a fair, honest test and not just a number that looks good:

We decided how we'd grade ourselves before we looked at the results. No moving the goalposts after the fact.
We tested the actual detection engine, not a simulation of it. The exact same code that decides your alert level is what got tested against real storms — running on satellite lightning data alone, which is the weakest single sensor the app uses. The live app adds ground-network lightning, weather-service warnings, forecast data, and — where a venue has one — an on-site sensor on top, and those can only raise an alert level, never lower it. So the detection rate and lead time are a floor, not a ceiling — more sensors can only catch more storms, and catch them earlier. The false-alarm count works the other way: the same rule that can only push an alert up is what could produce an alarm the test never had a chance to produce. That figure describes the satellite-only setup, which is exactly what background alerts run on. The in-app display adds ground-network lightning on a wider band, and no test covers that combination.
We didn't get to grade our own "no storm" days. Every quiet period had to independently prove itself storm-free using four separate outside checks — weather service records, thunder reports, radar, and official storm logs — before it counted.
We reported results for every region separately, not just averaged together. So a good number in one place can't cover for a weak one somewhere else.
We manually double-checked individual storms by hand — including every single miss and false alarm — to make sure the automated results actually held up.
We tell you where the weaknesses are. Smaller sample in some regions, limited radar history before 2020, open questions we haven't settled yet — all disclosed plainly in the full papers, not hidden in fine print.

What we found

Lightning detection:

We correctly detected 7 out of every 10 real storms overall (72.2%), and a similar share (74.0%) of the storms our satellite sensor could actually see (some storms are outside what any satellite can observe — that's a real limitation, not a flaw in our system)
Zero false alarms. Across 759 independently certified quiet periods, the system never once reached warning level or above. Five of those windows produced a low-level caution signal, each reviewed individually and each defensible. This is measured on satellite data alone — the same configuration background alerts run on. See the white paper for what it does and doesn't cover.
When we did catch a storm, we gave people an average of 36.5 minutes of warning
At Buck Hill Falls specifically, where the app is actually being used today, detection was 61.8% — a real, honest number for our first live location, and the one we think matters most right now

These lightning numbers were revised slightly downward on September 23, 2026. While tightening the storm-speed math behind our alerts, we found it had been counting one especially severe alert — pointing at a storm sixteen miles away — as a correct detection. It wasn't one; it was a bad distance estimate. Fixing it cost 0.7 percentage points off the detection rate. We'd rather publish that number than the flattering one.

The lightning numbers above were revised again on October 2, 2026, this time upward. We found that our background alerts treated lightning as most urgent only within 5 miles, not the 6 miles we publish, and after a pre-registered test we widened it to 6 miles. The figures on this page are measured at 6 miles. Our quiet-day tests cannot tell us how often the wider distance might raise a false alarm, so we are monitoring that live; the white paper has the details.

Rain detection:

When rain was heavy enough to actually matter, we caught it 97% of the time
We had zero false alarms at that same "this actually matters" level, across every single quiet day we tested
Rain and lightning never got confused with each other — across over 15,000 real moments we checked, rain never once made a lightning warning look less urgent than it should have

Read the full papers

Both papers include the complete statistics, region-by-region breakdowns, and an honest accounting of every limitation we found. The underlying data is available if you'd like to check our work yourself.

This page is just a summary. If anything here ever seems to disagree with the full papers, the full papers are what's accurate.