![\"120701_rpts_filtered.gif\"](\"https:\/\/inside.nssl.noaa.gov\/ewp\/wp-content\/uploads\/sites\/22\/2012\/12\/120701_rpts_filtered.gif1_.png\" "\\"120701_rpts_filtered.gif\\"")
<\/a><\/p>\nWow!\u00a0 A whopping 522 wind reports and 212 hail reports.\u00a0 That must have been an exceptionally-bad severe weather day.\u00a0 (It actually was the day of the big Ohio Valley to East Coast derecho from last July, a very impactful event).<\/p>\n
But what makes a storm report?\u00a0 Somebody calls in, or uses some kind of software (e.g., Spotter Network<\/a>), to report that the winds were X<\/em> mph or the hail was Y<\/em> inches in diameter from some location and at some time from within the severe thunderstorm.\u00a0 But the severe weather event is actually impacting an area surrounding the location from which the report was generated, and has been and will occur over the time interval representing the lifetime of the storm.\u00a0 It is highly unlikely that a hail report represented only a single stone falling at that location, or that the wind report represented a single wind gust local to that single location, and there were no other severe wind gusts anywhere else nor at any other time during the storm.\u00a0 Each of these reports represent only a single sample<\/em> of an event that covers a two-dimensional space over a time period.<\/p>\n If you recall from this blog entry<\/a>, the official Probability Of Detection (POD) is computed to be the number of reports that were within warning polygons over the total number of reports (inside and outside polygons).\u00a0 It’s easy to see that to effectively improve a office’s overall POD for a time period (e.g., one year), they only need to increase the number of reports that are covered by the warning polygons issued by that office during that time period.\u00a0 One way to do this is to cast a wide net<\/a>, and issue larger and longer-duration warning polygons. \u00a0 But another way to artificially improve POD is to simply increase the number of reports within storms via aggressive report gathering.\u00a0 Let’s consider a severe weather event like this one:<\/p>\n Look at all those (presumably) severe-sized hail stones.\u00a0 We can make a report on each one, at each time they fell.\u00a0 After about an hour of counting and collecting (before they all melted), this observer found 5,462 hail stones that were greater than 1″ in diameter.\u00a0 Beautiful – the Probability Of Detection is going to go way up!\u00a0 We can also count all the damaged trees as well to add hundreds of wind reports.\u00a0 Do you see the problem here?\u00a0 Are you getting tired of my extrapolations to infinity?\u00a0 Yes, there are literally an infinite number of severe weather reports that can be gleaned from this event (technically, there is a finite<\/em> number of severe-size hail stones fell in this storm, but who’s really counting that gigantic number?).\u00a0 But let’s scale this back.\u00a0 Here’s a scenario in which a particular warning is verified two different ways:<\/p>\n Each warning polygon verifies, so no false alarms.\u00a0 For the scenario on the top, there is one hit added to all reports for the time period (maybe a year’s worth of warning), but for the bottom scenario, there are seven<\/em> hits added to the statistics.<\/p>\n But wait, doesn’t the NWS Verification Branch filter storm reports that are in close proximity in space and time when computing warning statistics?\u00a0 Wouldn’t those seven hits be reduced to a smaller amount?\u00a0 They use a filter of 10 miles and 15 minutes to avoid my hypothetical over-reporting scenario.\u00a0 But that really doesn’t address the issue entirely.\u00a0 One can still try to fill every 10 mile and 15 minute window with a hail or wind report in order to maximize their POD.\u00a0 But if you think about it, that’s not really a bad idea.\u00a0 In essence, you are filling a grid with a 10 mile and 15 minute resolution with as much information known about the storm as possible.\u00a0 But this works only <\/em>if you also call into every 10-miles\/15-minute grid point inside and outside every storm.\u00a0 Forecasters rarely do this (and realistically can’t), because of workload issues, and because only one report within a warning polygon is all that is needed to avoid that warning from being labelled a false alarm (again, cast the wide net<\/a> so that one can increase their chance of getting a report within the warning).<\/span><\/p>\n CORRECTION (1\/10\/2013)<\/span>:\u00a0 I just learned that the 10 mile \/ 15 minute filtering was only done in the era of county-based<\/em> warning verification, and is not done for storm-based verification.\u00a0 Therefore, my arguments against the current verification methodology where hit rates and POD can be stacked by gathering more storm reports is further bolstered.\u00a0 More information is in the NWS Directive on forecast and warning verification<\/a>.<\/p>\n If we knew exactly what was happening within the storm at all times and locations at every grid point (in our case, every 1 km and 1 minute), we’d have a very robust verification grid to use for the geospatial warning verification methodology.\u00a0 But we really don’t know exactly what’s is happening everywhere all the time because it is nearly impossible to collect all those data points.\u00a0 The Severe Hazards Analysis and Verification Experiment (SHAVE) <\/a>is attempting to improve on the report density in time and space.\u00a0 But their resources are also finite, and they don’t have the staffing to call into every thunderstorm.\u00a0 Their high-resolution data set is very useful, but limited to only the storms they’ve called.\u00a0 What could we do to broaden the report database so that we have a better idea of the full scope of the impact of every storm?\u00a0 One concept is proxy verification<\/em>, in which some other remotely-sensed method is used to make a reasonable approximation of the coverage of severe weather within a storm, like so:<\/p>\n<\/a><\/p>\n<\/a><\/p>\n