There are several drawbacks of the “Grid Point” method that we need to address. First, how do we account for grid points that are downstream of an observation event that are expected to be covered by that event on a future grid? Should these be considered false alarm grid points? And how about grid points that are behind an event that has already passed that location? Second, the previous method has no easy way to compute lead times for specific grid point locations. To address these issues, we’ve developed the second method of geospatial warning verification, namely the “Truth Event” method.
A “truth event” is defined as a continuous time period a grid point is under a warning(s) and/or a tornado observation(s) surrounded by at least one minute of neither:
FALSE EVENT: If grid point remains in “false” condition throughout event (only forecast grid becomes “> 0”). These grid points do not receive an observation of a hazard, but were warned.
MISS EVENT: If grid point remains in “miss condition” throughout event (only observation grid becomes “> 0”). These grid points were not warned, but received an observation of a hazard.
HIT EVENT: If grid point experiences a “hit condition” for at least 1 minute during event (fcst and obs are both “> 0”). These grid points were warned AND received an observation of a hazard.
“Hit Events” can then be comprised of several different scenarios. The most common scenario would be this: 1) a warning is issued for a particular grid point, 2) a hazard observation impacts that grid point after warning issuance, 3) the hazard observation leaves the grid point while the warning remains in effect, and 4) the warning expires or is cancelled for that grid point. For these scenarios, the grid points will be in FALSE condition prior to and after the hazard passes over that location. For the Truth Event method, these conditions are not considered FALSE, but instead are depicted as “LEAD TIME” and “DEPARTURE TIME”, respectively. To see this graphically:
Hopefully this makes some sense. What you are seeing above are two snapshots of the grids at two times, t1 and t2. Let’s say we were to look at the “truth event” for one of the grid points in the figure, perhaps one that is right near the letter “L” in “LEAD TIME” on the t1 image on the left:
The truth event is defined by starting and ending time of the warning. Since the warning was issued prior to the observation impacting the grid point, you get positive LEAD TIME. This verification method rewards larger lead time. Although there is some discussion about how much lead time might be “too much”, we will table that discussion for now, and revisit it on a later blog entry. Note that if an observation impacts a grid point prior to a warning, then we can measure negative LEAD TIME, which is considered not good. But if an observation impacts a grid point that is never warned, then no LEAD TIME is recorded. This differs from the current NWS verification method, which records a zero (0) LEAD TIME for missed events.
This verification method also allows us to analyze a new kind of metric which we will call DEPARTURE TIME. This is the amount of time that a grid point remains under a warning after the threat has already passed by. Ideally, the DEPARTURE TIME should be zero – the warning is canceled or expires immediately just after the threat has passed. Positive DEPARTURE TIME, as shown in the above examples, is chosen to represent the condition when the warning remains in effect after the threat has passed (a FALSE condition, in a sense). Negative DEPARTURE TIME is chosen to represent the condition when the warning has expired before the threat has passed (a MISS condition, in a sense). This truth event scenario depicts negative LEAD TIME and negative DEPARTURE TIME, the warning was late plus the warning was canceled too early:
We can also analyze a third kind of metric which we will call FALSE ALARM TIME. This is for truth events that remain in FALSE EVENT condition through their time period. The time line of that kind of truth event is shown here:
More soon…
Greg Stumpf, CIMMS and NWS/MDL