{"id":393,"date":"2020-04-30T20:19:40","date_gmt":"2020-05-01T01:19:40","guid":{"rendered":"https:\/\/inside.nssl.noaa.gov\/phi-em\/?p=393"},"modified":"2020-04-30T20:20:42","modified_gmt":"2020-05-01T01:20:42","slug":"what-have-we-learned-a-quick-glance-at-emergency-managers-use-of-probabilistic-hazard-information-phi","status":"publish","type":"post","link":"https:\/\/inside.nssl.noaa.gov\/phi-em\/2020\/04\/what-have-we-learned-a-quick-glance-at-emergency-managers-use-of-probabilistic-hazard-information-phi\/","title":{"rendered":"What have we learned? A quick glance at emergency managers\u2019 use of Probabilistic Hazard Information (PHI)"},"content":{"rendered":"

The Emergency Manager (EM) experiment has hosted over 40 emergency managers since its inception in 2016, including 19 EMs in 2019!\u00a0 Before coming to the NOAA HWT housed in the NOAA National Severe Storms Laboratory in Norman, OK to participate in the Experiment, EMs completed a pretest questionnaire.\u00a0 In particular, they answered questions about their demographics, weather product use, and decision-making styles.\u00a0\u00a0<\/span><\/p>\n

As a whole, our EMs have been predominantly white (91%), male (71%), college graduates (75%), with degrees ranging from Associate degrees to Doctoral degrees, who work primarily for government agencies (86%).\u00a0 The majority of participants indicated working as EMs for more than 10 years.\u00a0 They represent a variety of jurisdictions and services\u2014cities, counties, state governments, hospitals, utilities, and federal agencies\u2014from across the US.<\/span><\/p>\n

EMs completed various questions assessing their decision-making styles and preferences.\u00a0 One measure, the Subjective Numeracy Scale (Fagerlin et al. 2007), assessed EMs\u2019 self-reported abilities and preferences for working with numbers over words.\u00a0 Sample questions included, \u201cHow good are you at working with percentages?\u201d and \u201cWhen you hear a weather forecast, do you prefer predictions using <\/span>percentages <\/b>(e.g., \u2018there will be a 0% chance of rain today\u2019) or predictions using only <\/span>words<\/b> (e.g., \u2018there is a small chance of rain today\u2019)?\u201d\u00a0 Thirty-nine EMs answered these questions.\u00a0 As a whole, EMs self-reported high numeracy, <\/span>Mean<\/span><\/i> = 4.48 (out of 6), <\/span>Standard Deviation <\/span><\/i>(<\/span>s<\/span><\/i>) = 0.98, which seemed to be driven by a preference for receiving numerical over verbal information, <\/span>M<\/span><\/i> = 4.66, <\/span>s<\/span><\/i> = 0.99 more than self-reported ability for working with numbers, <\/span>M<\/span><\/i> = 4.30, <\/span>s<\/span><\/i> = 1.36.<\/span><\/p>\n

For a more detailed description of the Experiment, please see our previous posts here<\/a> and here<\/a>.\u00a0 Let\u2019s dive right into the good stuff!<\/span><\/p>\n

Was PHI useful?\u00a0 EMs overwhelmingly reported that PHI was useful for their decisions, <\/span>M<\/span><\/i> = 6.84 (out of 7), <\/span>s<\/span><\/i> = 0.99, <\/span>n<\/span><\/i> = 19.\u00a0 Specifically, it delivered information quickly, <\/span>M<\/span><\/i> = 6.78 (out of 7), <\/span>s<\/span><\/i> = 0.43, <\/span>n<\/span><\/i> = 18; made EMs more confident in their decisions, <\/span>M<\/span><\/i> = 6.72 (out of 7), <\/span>s<\/span><\/i> = 0.58, <\/span>n<\/span><\/i> = 18; delivered pertinent information, <\/span>M<\/span><\/i> = 6.60 (out of 7), <\/span>s<\/span><\/i> = 0.70, <\/span>n<\/span><\/i> = 10; and was easy to use, <\/span>M<\/span><\/i> = 6.28 (out of 7), <\/span>s<\/span><\/i> = 0.75, <\/span>n<\/span><\/i> = 18.\u00a0 Through discussions, EMs elaborated that one of the nice aspects of PHI was the ability to pinpoint where the most damaging impacts could be expected and give people more time to prepare.\u00a0 Their warnings and communications could be more nuanced in that they were better able to indicate which areas should take immediate shelter, which areas should stay on high alert and be ready to shelter soon, and which areas had a bit of time before they could see potential impacts.<\/span><\/p>\n

What is it about PHI that is most important\/useful?\u00a0 Does that depend on the hazard to which they were responding?\u00a0 For both tornadoes and severe thunderstorms (high wind\/hail in particular), similar elements of the PHI and the Enhanced Data Display (EDD; i.e., the tool used to deliver PHI to EMs; see below)<\/span><\/p>\n

\"\"
This is a picture of what the Enhanced Data Display (EDD) tool looks like from the user’s perspective<\/figcaption><\/figure>\n

emerged as highly important: the forecaster discussion box, time of arrival estimate, hazard probability of occurrence, having one-hour lead time, and the PHI plume coloring scheme.\u00a0 Of these elements, two were significantly different in their importance across hazards.\u00a0 First, the importance of <\/span>hazard probability of occurrence<\/b> for understanding and communicating about tornadoes (<\/span>M<\/span><\/i> = .90) was significantly higher than for severe thunderstorms (<\/span>M<\/span><\/i> = .86), <\/span>t<\/span><\/i>(39) = 2.48, <\/span>p<\/span><\/i> = .02.\u00a0 Second, the importance of the <\/span>PHI plume coloring scheme<\/b> for understanding and communicating about tornadoes (<\/span>M<\/span><\/i> = .83) was significantly higher than for severe thunderstorms (<\/span>M<\/span><\/i> = .79), <\/span>t<\/span><\/i>(39) = 2.70, <\/span>p<\/span><\/i> = .01.\u00a0 The graphic below depicts these elements in the EDD and the data tables depict the differences in these PHI and EDD elements.<\/span><\/p>\n

\"\"
This is another view of the EDD tool with clear labels of the featured elements (e.g., the Forecaster Discussion box and Time of Arrival estimate).<\/figcaption><\/figure>\n
\"\"
This graph depicts participants’ ratings of the importance of each PHI element to their understanding and communication of Tornado hazards<\/figcaption><\/figure>\n
\"\"
This graph depicts participants’ ratings of the importance of each PHI element to their understanding and communication of Severe Thunderstorm hazards.<\/figcaption><\/figure>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

Can EMs successfully manage all of the different types of probabilistic information that were presented to them?\u00a0 This research aim was unique for the 2019 Experiments because EMs were presented with multiple products along the continuum of severe weather events, each conveying different probabilistic information.\u00a0 On a scale of 1 (extremely easy) to 5 (extremely difficult), EMs found it very easy to interpret all of the different kinds of probabilities they received for each case, <\/span>M<\/span><\/i> = 1.36, <\/span>s<\/span><\/i> = 0.51, <\/span>n<\/span><\/i> = 11.\u00a0 In discussions, EMs would note that even though the probabilities differed in the ways in which they were derived (e.g., climatology based vs. storm based) or what they were communicating\u2014among other dimensions\u2014, they \u201call made sense together\u201d and \u201ctold a complete story.\u201d\u00a0 EMs said having the complete picture helped, and having all of the probabilistic information helped them be more confident in making some decisions sooner while holding off on some other decisions.<\/span><\/p>\n

So, after immersion in our probabilistic information world, did EMs walk away preferring probabilistic information?\u00a0 Well, EMs actually had an overwhelming preference (95% of those asked; <\/span>n<\/span><\/i> = 20) for a <\/span>combination<\/span><\/i> of probability and text information.\u00a0 Even among those not asked this question in the posttest survey, discussions revealed support for this preference.\u00a0 In particular, EMs noted that, while they may prefer receiving probabilistic information for their own personal use (to help with interpreting the incoming storms), they would not want to pass probabilistic information on to their constituents (e.g., their bosses, their publics, their peers, etc.).\u00a0 Therefore, they would also need textual information that is ready to pass on to others for their consumption.<\/span><\/p>\n

Lastly, a really cool feature of the EDD is the user\u2019s ability to customize the different layers that are shown (e.g., radar, map, PHI plumes) based on their personal preferences.\u00a0 These layers included the color schemes used for the PHI plumes and the particular hazard plumes that are displayed.\u00a0 Participants predominantly (70% of those asked) preferred a monochromatic color scheme over a rainbow color scheme; this color scheme separated each hazard by color with varying probability levels displayed as gradients of the assigned color.\u00a0 For example, tornado PHI objects were red, where higher probabilities were represented by deeper\/darker shades of red and lower probabilities were represented by fainter\/lighter shades of red.\u00a0 Severe wind\/hail PHI objects were yellow with the same probability coloring concepts.\u00a0 EMs noted the monochromatic plumes were \u201cless cluttered\/confusing,\u201d \u201ceasy to discern on the map,\u201d and \u201c[didn\u2019t] blur out the radar.\u201d\u00a0 They also noted the rainbow-colored PHI plumes were \u201ctoo busy,\u201d \u201cdistracting,\u201d \u201c[easily] confused with radar.\u201d\u00a0 EMs noted difficulty trying to tell differing hazard plumes apart when they were all rainbow-colored.\u00a0 Thus, the monochromatic plumes were preferred because it was easier to distinguish between threats and ostensibly easier to discern the probabilistic information.<\/span><\/p>\n

\"\"
This picture depicts the monochromatic PHI coloring scheme; the red coloring is used for Tornado PHI objects.<\/figcaption><\/figure>\n
\"\"
This picture depicts the rainbow PHI coloring scheme; PHI objects are depicted with rainbow colors similar to radar.<\/figcaption><\/figure>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n

The findings discussed here only briefly touch on a few of the concepts that were tested and evaluated over the years of the EM Experiment.\u00a0 Taken together, our EM sample reported relatively high numeracy and a preference for receiving numerical (over textual) information.\u00a0 This preference was expressed in their responses to PHI; they found PHI provided important information and improved their decision-making.\u00a0 Further, they had an overwhelming preference for the PHI visualization that made it easy to distinguish hazards and probabilities of occurrence, i.e., the monochromatic color scheme.\u00a0 Unfortunately, our sample <\/span>is<\/span><\/i> small.\u00a0 We don\u2019t have the variability in our EMs numeracy scores to speak to whether these preferences change as a function of self-reported numeracy, but that is a great long-term goal for future research!\u00a0 Stay tuned for more updates and information about other aspects of the Experiment!<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

The Emergency Manager (EM) experiment has hosted over 40 emergency managers since its inception in 2016, including 19 EMs in 2019!\u00a0 Before coming to the NOAA HWT housed in the… <\/p>\n

Read more \u00bb<\/a><\/p>\n","protected":false},"author":127,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-393","post","type-post","status-publish","format-standard","hentry","category-research"],"_links":{"self":[{"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/posts\/393","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/users\/127"}],"replies":[{"embeddable":true,"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/comments?post=393"}],"version-history":[{"count":1,"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/posts\/393\/revisions"}],"predecessor-version":[{"id":400,"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/posts\/393\/revisions\/400"}],"wp:attachment":[{"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/media?parent=393"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/categories?post=393"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/inside.nssl.noaa.gov\/phi-em\/wp-json\/wp\/v2\/tags?post=393"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}