Around this time last year, we were just recuperating from the 2013 Colorado floods. The five-day natural disaster forced more than 11,000 evacuations and damaged more than 19,000 homes. In support of the Colorado Flood Threat Bulletin (FTB), a daily forecasting tool, we used Twitter to broadcast these forecasts to the general population. Historically used only by the state’s emergency management community, Twitter pushed FTB forecasts to 40 new Colorado cities and four times the number of people.

What makes FTB flood posts unique is that they’re not the product of just radar and machines, there’s an actual meteorologist behind every daily post. In a world where automated services are the norm, we believe that incorporating a human element makes the FTB flood reports better than automated weather reports.

In an effort to find out just how valuable FTB forecasts are, we compared its flood reports to newly developed objective forecasting tools based on weather model data. Our results found that, while there are areas for improvement, the FTB substantially outperformed purely objective forecast tools.

Measuring the Importance of a Human Forecaster

Comparing FTB forecasts to our other model tools, we found that while other tools are comparable in determining flood threat days, the comparison does not account for location – an obviously important piece of an accurate forecast’s puzzle.

What we did find was that 74 percent of all flood-related reports fell within threat areas highlighted by the FTB. The FTB forecasts were far more accurate than the 4 to 46 percent location accuracy of our other tools.

The caveat to this accuracy was the necessity to issue large flood threat areas, particularly because our meteorologists could not, on some days, rule out the possibility of strictly isolated, potentially heavy, rainfall (a common cause of flash floods in Colorado). In turn, this created relatively large “false alarm” regions, where an FTB flood threat was posted but neither a flood nor a very heavy rainfall event occurred. For example, the FTB average for an issued flood threat was more than 25 thousand square miles, while the average of those issued by the purely objective tools was closer to 3.4 thousand square miles.

Making the FTB Even More Effective

This type of honest, objective model comparison must be embraced, as it is the tool of the future and helped us realize areas for FTB improvement, like decreasing the false alarm area.

The FTB substantially outperformed the objective models in correctly forecasting flood reports, and while no single comparative forecasting product was able to consistently beat the FTB, others did provide good estimates of maximum daily precipitation and flood threat area.

What this research truly highlights is that the human forecaster is far from obsolete. A good thing to know in an industry where accuracy is everything.