WoFS in the virtual NOAA Hazardous Weather Testbed


The first week of April concluded the end of the 2021 Warn-on-Forecast Testbed Experiment as part of the NOAA Hazardous Weather Testbed. In this experiment, a total of 16 forecasters from nine southern regions National Weather Service Forecast Offices (WFOs), the NOAA NWS Storm Prediction Center (SPC), and the NOAA NWS Weather Prediction Center (WPC) came together over four weeks to explore the use of Warn-on-Forecast System (WoFS) guidance in the watch-to-warning time frame.

Like many other scientific activities, this experiment was delayed and then moved virtually due to the ongoing COVID-19 pandemic. Despite the many challenges this unique situation presented, our research team is pleased to report the experiment was very successful. This success is attributable to the significant efforts of numerous University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies (CIMMS) and NOAA National Severe Storms Laboratory scientists, as well as participants and collaborators in the NWS

Together, Pat Skinner (with OU CIMMS/NSSL), Patrick Burke (with NSSL), Burkely Gallo (with OU CIMMS/SPC), and I (Katie Wilson with OU CIMMS/NSSL) designed and executed this experiment to examine how forecasters envision WoFS guidance fitting into both their existing and visionary forecast processes, and to explore the ways that WoFS guidance can be used most effectively given national center and local office forecasting responsibilities.

A screenshot showing an experimental National Weather Service severe weather graphic. The black graphic shows storm areas highlighted.
An example of an experimental decision support graphic influenced by WoFS and constructed during one of the 2021 WoFS experiment case studies. (Screenshot)

A Collaborative Undertaking

During the experiment, Skinner delivered an overview presentation to build familiarity with WoFS guidance prior to participants’ completion of two case studies. These cases formed the first of two major activities during the experiment, which was for participants to immerse themselves in simulated real-time events and use WoFS guidance to make forecast and communication decisions. 

To prepare for the case study activity, each week Burke provided participants with a hands-on AWIPS-2 demo. Furthermore, his work to design AWIPS-2 perspectives and procedures, which was conducted jointly with Gallo, enabled a simulation setup that was more familiar to participants, especially to those from national centers who do not use AWIPS-2 in the way local office forecasters do.

A screenshot of the experimental WoFS guidance in the AWIPS-2 viewer.
For the first time, experimental WoFS guidance was viewable in the AWIPS-2 interface.

The preparation of these case studies was a major task undertaken by Jonny Madden (OU CIMMS/NSSL), Justin Monroe (OU CIMMS/NSSL), Jorge Guerra (OU CIMMS/NSSL), and Dale Morris (OU CIMMS/NWS Warning Decision Training Division).

The case studies presented two notable firsts:

  • Running AWIPS-2 in-the-cloud such that participants could complete the case studies from their own homes, and;
  • Presenting WoFS guidance in AWIPS-2, including the development of a tool to visualize paintball plots. 

Madden, Monroe, Guerra, and Morris worked together to accomplish numerous tasks, including: aggregating and processing a full suite of observational and model datasets for both cases, setting up the WES- 2 Bridge and AWIPS-2 interfaces, and collaborating with federal partners to get datasets onto the cloud framework. Much of what was accomplished for the case study portion of this experiment has laid the AWIPS-2 in-the-cloud groundwork for future virtual experiments. 

The second major activity during the experiment was focus groups. Together, Wilson and Gallo led three semi-structured discussions each week to explore forecasters’ visions for how WoFS will impact the current and future forecast process. Additionally, the presence of both national center and local office forecasters meant that much was learned about each others’ workflows, how one another makes decisions, and where there is an opportunity to strengthen collaboration. In a post-experiment questionnaire, participants rated the focus groups as a highly effective activity for sharing thoughts and ideas, and was the most enjoyed activity of the week.

A graphic showing how the team used Google Meet Jam Board to spur discussion. The graphic has two circles, with forecast offices in one area and SPC and WPC in another.
The team used the Google Meet jam board to spur discussion in focus groups.

In addition to the efforts of scientists at OU CIMMS and NOAA NSSL, we were grateful for input from our collaborators at NWS Southern Region, including Chad Gravelle (SR HQ), Todd Lindley (OUN SOO), Stephen Bieda (AMA SOO), and Randy Bowers (OUN). Gravelle and Lindleyalso joined the experiment for multiple weeks, and Randy created two excellent weather briefing videos to prepare forecasters for the case studies. A big thank you also goes to our pilot participants, Laren Reynolds (El Paso, Texas) and Joseph Merchant (Lubbock, Texas), for volunteering their time to fulfill an important support role throughout the whole experiment. This support role emerged following findings from the pilot week, and made for a much stronger experiment.

We are extremely appreciative to the 16 NWS forecasters who participated in this experiment. We realize the stressful conditions many people continue to live and work with, and have done so over the past year. We also realize the disappointment from not being able to attend this experiment in the NOAA Hazardous Weather Testbed, in Norman, Oklahoma, as originally planned. However, participants showed up to our virtual experiment with enthusiasm and made meaningful contributions to the experiment. We collected an enormous amount of data, and we can’t wait to analyze it and share what was learned.

For questions on this or other WoFS-related research please contact WoFS Program Lead, Patrick Burke, nssl.outreach@noaa.gov.

Share this:

NOAA HWT Hosts Annual Spring Forecasting Experiment

The NOAA HWT Experimental Forecast Program (EFP) is focused on the use of computer models of the atmosphere to improve predictions of hazardous and convective weather events from a few hours to a week in advance, and over several counties to the continental U.S. The EFP supports the NWS goal to increase lead-time and accuracy for weather and water warnings and forecasts. The NOAA HWT EFP Spring Experiment is a yearly project that investigates the use of convection-allowing model forecasts as guidance for the prediction of hazardous convective weather.

For for the next five weeks NOAA’s Hazardous Weather Testbed Experimental Forecast Program is focused on one topic — using cutting-edge computer models to improve predictions of hazardous weather from one hour to several hours to several days in advance.

The aim of such an experiment is to support NOAA’s goal of more accurate severe weather warnings and forecasts. From April 30 to June 1, researchers and forecasters will work together to examine and evaluate a variety of model output daily in the HWT, exploring a variety of different models and forecast guidance throughout the week. Forecast and research meteorologists create experimental forecasts, interact with new technologies, provide feedback to the researchers, and decipher the best use of tools in simulated real-time forecasting environments.

“Driving what we do is being able to work with forecasters to get their input,” said Adam Clark, project lead and researcher with the NOAA National Severe Storms Laboratory. “To design a system that is useful, you have to get feedback from the end user, which is forecasters.”

Better Tools for Forecasting Dangerous Weather

The experimental forecast models used in the NOAA Hazardous Weather Testbed become more refined each year as technology improves. Ultimately, the NOAA National Weather Service incorporates insights and improvements from the HWT into their operations.

One of the tools being tested is NSSL’s experimental high-resolution, on-demand and frequently updating ensemble forecast system product called NSSL Experimental Warn-on-Forecast System for ensembles, or NEWS-e. Each day, researchers focus their attention on the area with the highest risk for severe weather based on the forecast from NOAA’s Storm Prediction Center. NEWS-e runs in that area and frequently assimilates radar and satellite data to provide specific forecasts of when and where severe weather is most likely on a short time scale — up to three hours in advance.

Another tool being tested is the High-Resolution Rapid Refresh Ensemble, or HRRRE, a prediction system developed by the Global Systems Division of NOAA’s Earth Systems Research Laboratory. HRRRE brings in a variety of weather data to produce multiple forecasts, each with a slightly different starting point, to generate hourly snapshots of possible hazardous weather. Each forecast outcome includes the degree of certainty or uncertainty of the hazard occurring. For example, numerous HRRRE snapshot forecasts predicting that a severe storm will form at a specific place and time will provide users more confidence in that forecast. HRRRE predictions provide the starting point for NSSL’s NEWS-e as well.

Utilizing both products in NOAA’s HWT allows forecasters to evaluate the ability of such tools to highlight severe weather hazards, including low-level rotation resulting in tornadoes.

Collaboration is Key

A panoramic view of NOAA's Hazardous Weather Testbed during the Spring Forecasting Experiment in April and May.
The NOAA Hazardous Weather Testbed is a joint project of the National Weather Service and the National Severe Storms Laboratory. The HWT provides a conceptual framework and a physical space to foster collaboration between research and operations to test and evaluate emerging technologies and science for NWS operations. The HWT was borne from the “Spring Program” which, for the last decade, has been used to test and evaluate new forecast models, techniques, and products to support NWS Storm Prediction Center forecast operations.

During its 20 year history, researchers have evaluated a variety of weather models during the annual Spring Forecasting Experiment.

To better compare different models created by various developers, NSSL researchers created the experimental Community Leveraged Unified Ensemble, or CLUE, a large collaborative effort providing common guidelines to optimize high resolution models. Before, each model, including the HRRRE, operated using different measurements and standards. CLUE provides one set of guidelines for all to operate in the HWT.

The guidelines include using the same parameters to run models, operating on the same resolution, and providing the same display. Such a framework allows each model to be more easily understood on a level field.

“That way we are able to control as many variables as we can so we can say more about why the different systems work they way they do,” Clark said. “It’s like comparing apples to apples rather than apples to oranges.”

The testbed is operated by the NOAA National Severe Storms Laboratory and the NOAA National Weather Service.


Share this: