{"id":3243,"date":"2011-11-22T17:34:13","date_gmt":"2011-11-22T22:34:13","guid":{"rendered":"https:\/\/hwt.nssl.noaa.gov\/ewp\/internal\/blog\/?p=3243"},"modified":"2017-06-23T14:37:40","modified_gmt":"2017-06-23T19:37:40","slug":"precise-threat-area-identification-and-tracking-part-iii-how-good-can-tim-be","status":"publish","type":"post","link":"https:\/\/inside.nssl.noaa.gov\/ewp\/2011\/11\/22\/precise-threat-area-identification-and-tracking-part-iii-how-good-can-tim-be\/","title":{"rendered":"Precise Threat Area Identification and Tracking Part III: How good can TIM be?"},"content":{"rendered":"

For my presentation at the 2011 NWA Annual Meeting in Birmingham, I wanted to introduce the concept of geospatial warning verification and then apply it to a new warning methodology, namely Threats-In-Motion (TIM).\u00a0 To best illustrate the concept of TIM, I’ve been using the scenario of a long-track isolated storm.\u00a0 To run some verification numbers, I needed a real life case, so I chose the Tuscaloosa-Birmingham Alabama long-track tornadic supercell from 27 April 2011 (hereafter, the \u201cTCL storm\u201d), shown in earlier blog posts, and I’ll repeat some of the figures and statistics for comparison.\u00a0 This storm produced two long-tracked tornadoes within Birmingham’s County Warning Area (CWA).<\/p>\n

The choice of that storm is not meant to be controversial.\u00a0 After all, the local WFO did a tremendous job with warning operations, given the tools and they had and within the confines of NWS warning policy and directives.\u00a0 Recall the statistics from the NWS Performance Management site:<\/p>\n

Probability Of Detection = 1.00<\/strong><\/p>\n

False Alarm Ratio = 0.00<\/strong><\/p>\n

Critical Success Index = 1.00<\/strong><\/p>\n

Percent Event Warned = 0.99<\/strong><\/p>\n

Average Lead Time = 22.1 minutes<\/strong><\/p>\n

These numbers are all well-above the national average.\u00a0 We then compared these numbers to those found using the geospatial verification methodology for “truth events”:<\/p>\n

CSI = 0.1971<\/strong><\/p>\n

HSS = 0.2933 <\/strong><\/p>\n

Average Lead Time (lt) = 22.9 minutes<\/strong><\/p>\n

Average Departure Time (dt) = 15.2 minutes<\/strong><\/p>\n

Average False Time (ft) = 39.8 minutes<\/strong><\/p>\n

And recall the animation of the NWS warnings I provided in an earlier blog post:<\/p>\n

\"warn-NWS-loop-fast\"<\/a><\/p>\n

What if we were to apply the Threats-In-Motion (TIM) technique to this event?\u00a0 Recall from this blog entry<\/a> that I had developed a system that could automatically place the default WarnGen polygons at the locations of human-generated mesocyclone centroids starting at the same minute that the WFO issued its first Tornado Warning on the storm as it was entering Alabama.\u00a0 I used a re-warning interval of 60 minutes.\u00a0 If you adjust the re-warning interval to one minute (the same temporal resolution of the verification grids), you now have a polygon that moves with the threat, or TIM.\u00a0 The animation:<\/p>\n

\"warn-TIM-10km-loop\"<\/a><\/p>\n

Let’s look at the verification numbers for the TIM warnings:<\/p>\n

CSI = 0.2345<\/strong><\/p>\n

HSS = 0.3493 <\/strong><\/p>\n

Average Lead Time (lt) = 51.2 minutes<\/strong><\/p>\n

Average Departure Time (dt) = 0.8 minutes<\/strong><\/p>\n

Average False Time (ft) = 23.1 minutes<\/strong><\/p>\n

 <\/p>\n

Comparing the values between the NWS warnings and the TIM warnings visually on a bar graph:<\/p>\n

\"NWA-nl-fig-3\"<\/a><\/p>\n

All of these numbers point to a remarkable improvement using the Threats-In-Motion concept of translating the warning polygons with the storm, a truly storm-based<\/em> warning paradigm.\u00a0 Let’s also take a look at how these values are distributed in several ways.\u00a0 First, let’s examine Lead Time.\u00a0 The average values for all grid points impacted by the tornado (plus the 5 km “splat”) are more than doubled<\/em> for the TIM warnings.\u00a0 How are these values distributed on a histogram:<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

For the TIM polygons, there are a lot more values of Lead Time above 40 minutes.\u00a0 How do these compare geospatially?<\/p>\n

\"LT-NWS-map\"<\/a><\/p>\n

\"LT-TIM-map\"<\/a><\/p>\n

Note that the sharp discontinuities of Lead Time (from nearly zero minutes to over 40 minutes) at the downstream edges of the NWS warnings (indicated by yellow arrows in the top figure) are virtually eliminated with the TIM warnings.\u00a0 You do see a few discontinuities with TIM; these small differences are caused by changes in the storm motions at those warned times.<\/p>\n

Moving on to Departure Time, the average values for all grid points impacted by the tornado (plus the 5 km “splat”) are reduced to nearly zero<\/em> for the TIM warnings.\u00a0 How are these values distributed on a histogram:<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

And geospatially:<\/p>\n

\"DT-NWS-map\"<\/a><\/p>\n

\"DT-TIM-map\"<\/a><\/p>\n

With the TIM polygons, the departure times across the path length of both tornadoes is pretty much less than 3 minutes everywhere<\/em>.\u00a0 Whereas, the NWS polygon Departure Times are much greater, and there are some areas still under the NWS warning more than 30 minutes after<\/em> the threat had passed.<\/p>\n

Finally, looking at False Alarm Time distributions both on histograms and geospatially:<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"FT-NWS-map\"<\/a><\/p>\n

\"FT-TIM-map\"<\/a><\/p>\n

There are some pretty large areas within the NWS polygons that are under false alarm for over 50 minutes at a time, even though these warnings verified “perfectly”.\u00a0 In comparison, the TIM warnings have a much smaller average false alarm times for areas outside the tornado path (about a 42% reduction).\u00a0 However, there are a number of grid points that are affected by the moving polygons for just a few minutes.\u00a0 How would we communicate a warning that is issued and then canceled just a few minutes later?\u00a0 That is food for future thought.\u00a0 I’ve also calculated an additional metric, the False Alarm Area (FAA), the number of grid points (1 km2<\/sup>) that were warned falsely.\u00a0 Comparing the two:<\/p>\n

NWS FAA = 10,304 km2<\/sup><\/strong><\/strong><\/p>\n

TIM FAA = 8,103 km2<\/sup><\/strong><\/strong><\/p>\n

There is a 21% reduction in false alarm area with the TIM warnings, much less than the reduction in false alarm time.\u00a0 The reduction in false alarm area is more a function in the size of the warning polygons.\u00a0 The WarnGen defaults were used for our TIM polygons, but it appears that the NWS polygons were made a little larger than the defaults.<\/p>\n

So, for this one spectacular long-tracked isolated supercell case, the TIM method shows tremendous improvement in many areas.\u00a0 But there are still some unanswered questions:<\/p>\n