Researchers leverage machine learning to improve forecasting tools

Weather models are the basic building blocks of any forecast. NOAA National Weather Service forecasters utilize a variety of models to provide accurate weather information for the public when severe weather threatens. 

NOAA and cooperative institute researchers are leveraging machine learning techniques and high resolution weather models in an effort to improve these tools.

“We hope our research will provide forecasters with more information on when they should, or shouldn’t, rely on certain forecast models,” said Burkely Gallo, a researcher at the University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies, whose work supports the NOAA NWS Storm Prediction Center.

Burkely Gallo presenting a powerpoint of her research in front of people.
Burkely Gallo presenting on her and the team’s machine learning techniques research at the NOAA booth at the American Meteorological Society 100th Annual Meeting in January 2020. Burkely is a University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies research whose work supports the NOAA NWS Storm Prediction Center. (Photo by Emily Summars-Jeffries/OU CIMMS/NOAA NSSL)

Computer weather models continue to improve, providing accurate forecasts as much as a week in advance. Yet, each has strengths and weaknesses and must be interpreted by knowledgeable human forecasters. It could take decades of experience for forecasters to gain expertise on what forecasting models are the most accurate for specific weather events.

A series of preliminary research aims to allow automatic flagging of problematic forecasts, provide quality control for the development of new atmospheric models and allow model developers to learn why a model is or is not valid.

“To achieve the latest and greatest forecasting models, people developing the models need to know how they are performing in certain scenarios,” Gallo said. “We hope this can help them identify priorities for future model development.”

Alex Anderson-Frey is a co-researcher on the project, which began as an internal funding proposal in a competition organized by the NOAA Central Regional Collaboration Team. Anderson-Frey and Gallo won funding for their project and their work was supported by NOAA’s National Severe Storms Laboratory when it began.

Burkely Gallo and Alex Anderson-Frey stand in front of their powerpoint presentation.
The OAR/NWS Shark Tank, Season 2 where Anderson-Frey and Gallo presented their research idea. The OAR/NWS Shark Tank was coordinated by the NOAA Central Region Collaboration Team was held in February 2018 at the National Weather Center in Norman, Oklahoma. (Photo by James Murnan/NOAA)

“Alex and I have wanted to work together since college,” Gallo said. “We decided this internal program could be the spark for collaboration.”

Gallo and Anderson-Frey used the competition funding to hire a graduate student part-time and the result of his efforts allowed them to have a complete dataset to begin their work, which leverages machine learning. Machine learning sorts storms based on different environmental factors surrounding the storms. Environmental factors include fields, like dew point and temperature. Found patterns can then be matched to a current model forecast. This  provides forecasters an idea of how the model they are using performed in similar past scenarios. 

“We want to provide tools that allow forecasters to quickly learn, so they can know if a model has statistically performed very well for tornado detection in this type of model environment,” Gallo said. “Forecasters manage a fire hose of data and we hope to make the fire hose manageable.”

Gallo said she expects the project to continue for several years, with the team’s goal of testing the products in NOAA’s Hazardous Weather Testbed.

Share this:

Information analysis: social science adds needed piece to the weather puzzle

Research Scientist Jack Friedman with University of Oklahoma Center for Applied Social Research observes and works in the National Weather Forecast Office in Huntsville, Alabama. Friedman is one of the several social science researchers involved in the VORTEX-Southeast project spring 2017 experiment. (NOAA NSSL)

Increasing our knowledge of severe storms and improving the tools used to forecast them has been the singular mission of the NOAA National Severe Storms Laboratory since it was formed more than 50 years ago — until recently. Now NSSL researchers are expanding their focus to include people — how they receive, understand and interact with weather information.

A new report released this month by the National Academies of Sciences, Engineering and Medicine concludes that realizing the greatest return on investment from significant improvements in weather information will require a better understanding of how individuals, households and communities respond to weather forecasts, watches and warnings.

NSSL is already doing many of the recommendations mentioned in the report, said Kim Klockow, a research associate working at NSSL with the University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies. Dozens of researchers are integrating disciplines such as communication, psychology and education into the traditional meteorological research at NSSL.

“Meteorologists care about saving lives and property, and ultimately those goals depend on the actions people choose to take,” Klockow said. “Information is just one piece of the puzzle. Providing the public with information about possible dangers doesn’t stop the threats from having an impact, and it alone doesn’t motivate people to take action.

“In our research at NSSL, we have to account for the ways people understand what we’re saying, the things they’re able to do, and the things that motivate them.”

Klockow leads a new societal impacts group at NSSL created to ensure new technologies are useful and usable by the public, emergency managers and public broadcasters. Several recent projects are highlighted below.

Research in the Hazardous Weather Testbed

Each year, NSSL invites broadcast meteorologists, emergency managers and National Weather Service forecasters to the NOAA Hazardous Weather Testbed in Norman, Oklahoma, to test new technology developed at NSSL and within NOAA.

“Our research needs to engage those who will be using it,” Klockow said. “We have them test what our researchers have developed to see if they can use it, or will use it.”

Next year, the researchers plan to invite larger private sector companies to participate in testbed experiments. These forecasters may provide new insights, Klockow said.

The NOAA Hazardous Weather Testbed during the Spring 2017 experiment about the Geostationary Lighting Mapper. (Photo by James Murnan/ NOAA NSSL)

People’s responses to warnings
Recently, NSSL teamed up with OU’s Center for Risk and Crisis Management to analyze how the public receives and acts on weather warnings. This project, part of the broader Probability of What project, is to study the effectiveness of the current warning infrastructure. This information will help NSSL measure the impacts new technologies might have on the public.

“We are looking at providing more information between a watch and warning to fill the information gap and provide up to an hour of advanced notice for all kinds of severe weather,” Klockow said. “We need to know if it will be beneficial to people — if they will use that information — or if giving a slew of probabilities may be more difficult to understand.”

The POW research team is conducting nationwide surveys and small experiments to measure the public’s understanding of weather information.

Social science integral part of tornado study
How emergency managers and forecasters handle information during hazardous weather events is an important part of VORTEX-Southeast, a research program studying storms and tornadoes in the southeastern United States.

“VORTEX-SE is the first time social science has been integrated into a weather field campaign,” Klockow said. “When the physical science researchers deploy to the field, so do the social science researchers.”

Klockow said social scientists have embedded with local emergency managers and National Weather Service forecasters, studying how they receive information, process that information, and relay it to the public.

“We see if there are any information gaps, points of confusion, or breaks in the communication channels and how the process may be improved,” Klockow said.

Studying the latest technology
Part of informing the public about weather affecting them includes staying apprised of the latest and greatest technology. Klockow is researching the ATSC 3.0, a new television broadcast system offering more options, including advanced emergency alerts.

“It will fundamentally change the way TV works, so someone can point to the TV with their remote and get more detailed or local information during severe weather coverage,” Klockow said. “The viewer could pull up radar, probability plumes defined by NSSL research or timelines. This offers an amazing opportunity to get more information to the user. We have to make sure we are aware of this new technology and get it in sync with our research designs.”

Whether studying the structure of a thunderstorm, developing a new radar algorithm, improving a weather forecasting model, or analyzing the ways people receive weather information — every project at NSSL has at its heart the goal of minimizing the impacts of hazardous weather on society.

Share this:

Weather Reports from Citizens Provide Research Input

Is it raining, snowing or hailing where you are? Tell us about it! Report the weather at your location any time on the mPING app to help NOAA researchers and forecasters, and join citizen scientists all around the world participating in Citizen Science Days through May 20.

Downloadable to your smartphone, mPING (Meteorological Phenomena Identification Near the Ground) is a free application that allows users to submit information about the weather to NOAA’s National Severe Storms Laboratory. Reports are immediately archived into a database at NSSL, and are displayed on a map accessible to anyone.

An mPING report. See more at

To use the app, reporters select the type of weather that is occurring, and tap “submit.” The anonymous reports can be submitted as often as every 30 seconds.

The main goal of mPING is to provide more information to researchers and forecasters about the weather affecting the public. As a bonus, that very same public can see these reports! Weather radars cannot “see” at the ground, so mPING reports are used by the NOAA National Weather Service to fine-tune their forecasts.

Reports from mPING are also helping NOAA researchers in a variety of ways, including to develop new radar and forecasting technologies and techniques, said Kim Elmore, research scientist with The University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies working at the NOAA NSSL.

The first goal of mPING was verifying the type of precipitation detected by new radar technology. But, data from mPING proved useful for things not originally envisioned. While checking the accuracy of reports, Elmore and his team learned more about the scale and variability of freezing rain.

One mistake people could reasonably make is calling freezing rain just rain, since freezing rain is rain until it freezes onto something, Elmore explained. But with disagreeing observations, Elmore said they found only about 17 percent of the observations were rain; but about 60 percent of the observations that disagreed with freezing rain were really ice pellets.

“What that tells us is people can clearly discriminate between freezing rain and rain and freezing rain and ice pellets, since they all agree on what ice pellets are,” Elmore said. This shows that people know what they are seeing and mPING reports are mostly accurate.

Digging a little deeper, we know that freezing rain can exist in only a very narrow set of environmental conditions. If the air near the ground gets only a little colder, the raindrops will freeze before they reach the surface and be reported as ice pellets. If conditions warm up only a degree or two near the ground, the temperature is no longer below freezing and there can be no freezing rain.

Reports from mPING also helped the team learn about one of the newest numerical weather prediction models called the Rapid Update model, or RAP, which is used for short term weather prediction. Data from mPING showed the old version of RAP did not properly identify ice pellets. Once the model developers learned about this, they immediately made changes to better forecast ice pellets.

For one winter season, NOAA’s Earth System Research Lab and the

 operated the old version (but with the ice pellet fix) and new RAP models at the same time. This was a perfect perfect opportunity to see how much better the new RAP system handled ice pellets. To test this, mPING reports were compared to RAP model forecasts.

“It turns out, ice pellets are reported far more often than the model forecasts them,” he said. “We reported on the fact the new version of RAP helps some — it is a small but statistically significant improvement.”

The mPING app was developed through a partnership between NSSL, the University of Oklahoma and the Cooperative Institute for Mesoscale Meteorological Studies and was included in Scientific American’s list of 8 Apps That Turn Citizens into Scientists. For more information on the application, or to watch a short video about it, visit

Scientists will continue to look for new ways to use the mPING data in their research, Elmore said. So keep those reports coming!

Share this:

April 27 Reddit AMA: Tornado! Severe Weather Research & Prediction with NOAA

Spring has arrived and with it come efforts to study and learn to better predict severe weather like tornadoes. Join NOAA for a Reddit Ask Me Anything (AMA) on severe weather research and prediction on April 27, 2017.

Patrick Marsh, Adam Clark, Kim Klockow and Harold Brooks will take your questions during Thursday’s #Reddit AMA.

Severe weather touches every state in the U.S. Tornadoes, severe thunderstorms, hail, strong winds, and floods are real threats to our property and our lives. The NOAA Hazardous Weather Testbed and VORTEX-SE (Verification of the Origins of Rotation in Tornadoes EXperiment-Southeast) are designed to learn more about storms, helping to improve our prediction abilities and bring you better forecasts.

At the National Weather Center, which houses NOAA’s National Severe Storm Laboratory (NSSL) and Storm Prediction Center, as well as the University of Oklahoma Cooperative Institute for Mesoscale Meteorological Studies (CIMMS), our scientists work to better understand and predict severe weather to help everyone be prepared.

Reddit AMA Details


     Harold Brooks, NOAA NSSL research meteorologist

     Kim Klockow, UCAR scientist at CIMMS

     Adam Clark, NOAA NSSL research meteorologist

     Patrick Marsh, NOAA SPC warning coordination meteorologist

When: Thursday, April 27, 2017, from 9:00 a.m. to 11:00 a.m. CT

Where: Reddit Science AMA series

About the Scientists

Harold Brooks, a senior scientist in the Forecast Research and Development Division of NOAA NSSL, is originally from St. Louis, Missouri. He received a Ph.D. in atmospheric science in 1990 from the University of Illinois at Urbana-Champaign. He joined NSSL in 1991 as a research meteorologist specializing in tornado climatology.

Adam Clark is a meteorologist with NOAA NSSL and a 2014 Presidential Early Career Award for Scientists and Engineers (PECASE) winner. Originally from Des Moine, Iowa, Clark received his Ph.D. in meteorology and started working at NSSL in 2009. Clark is active in the NOAA Hazardous Weather Testbed, which conducts experiments mainly late March and April.

Kim Klockow is a University Corporation for Atmospheric Research (UCAR) project scientist at NOAA’s Cooperative Institute for Mesoscale Meteorological Studies at The University of Oklahoma who earned her Ph.D. in Human Geography. Working with the NOAA National Severe Storms Laboratory, her research involves behavioral science focused on weather and climate risk, and explores the effects of risk visualization on judgment and perceptions of severe weather risk from a combination of place-based and cognitive perspectives.

Patrick Marsh is a warning coordination meteorologist at the NOAA National Weather Service’s Storm Prediction Center, which provides forecasts and watches for severe thunderstorms and tornadoes over the contiguous United States. He was born in Georgia but grew up in Arkansas and received his Ph.D. at the University of Oklahoma.

Share this:

Linda McGuckin Named NOAA Employee of the Month

2015_McGuckin-1NSSL Administrative Assistant Linda McGuckin was named NOAA Employee of the Month for December 2015. She is one of two individuals selected for the honor by VADM Michael S. Devany, NOAA Deputy Under Secretary for Operations.

In September 2014, Linda volunteered to serve as temporary property custodian when the previous property manager retired. She went above and beyond the call of duty, not only sustaining property management activities at NSSL, but improving their overall quality and efficiency. Under her leadership, the inventory team met all deadlines with an exceptionally high inventory accuracy rate of 99.9 percent.

Additionally, Linda supported the Plains Elevated Convection At Night project. This multi-agency scientific field experiment involved numerous mobile scientific instruments (many belonging to NSSL) and included foreign-national personnel working with government equipment and technology. Linda documented property and personnel returning from PECAN. She filed complex travel vouchers for participants in both PECAN and NOAA’s Hazardous Weather Testbed. She accommodated an unusually large number of equipment procurement requests, reconciled accounts associated with NSSL divisions and field experiments, and received, inventoried, and delivered new equipment. Linda’s invaluable contributions bolstered the success of the projects with which she was involved.

“This recognition was unquestionably deserved, and we are all indebted to Linda for her outstanding service in a time of dire need,”said NSSL Director Steve Koch. “She exemplifies the meaning of being a team member and serving where there is a need.”

Congratulations to Linda and thank you for all of your hard work!

Share this:

2015 PARISE Experiment

This week, researchers from NOAA’s National Severe Storms Laboratory will launch the 2015 Phased Array Radar Innovative Sensing Experiment to assess the impacts of rapidly updating radar data on forecasters’ warning decision performance. The project will be carried out over the course of six weeks, and will conclude on September 25.

As in previous years, NSSL Research Scientist Dr. Pam Heinselman and CIMMS Researcher Katie Bowden will take the lead on the experiment. They will be working with NOAA National Weather Service forecasters to produce timelines of the warning decision process. Later they will analyze these timelines to determine the situational awareness attained from phased array radar data and how that information was used in warning decisions. The experiment will be conducted in three parts.

The first segment of 2015 PARISE will be conducted like a traditional experiment, according to Heinselman. Thirty National Weather Service forecasters from across the Great Plains region will be assembled to study nine archived cases. These cases will be worked in simulated real-time, using one-, two-, or five- minute phased array radar updates. The forecasters will determine whether or not to warn, based on the situational awareness gained from the radar data. Upon completion of each study, they will provide a detailed account of their warning decision process and overall workload. With more participants and additional case studies this year, the results are expected to be an improvement over previous experiments.

New this year will be the use of eye-tracking technology to better understand the decision-making processes of the forecasters. Eye-tracking technology has been successfully used for analysis in healthcare, air traffic control, and other human-computer interactions. Data pertaining to eye gaze will be gathered from each of the 30 forecasters while they are working on PAR case studies. Analysis of this data is expected to illustrate how update timelines impact forecasters’ decisions.

Eye-tracking technology used in PARISE will help NSSL researchers determine how forecasters use phased array radar data to make decisions.

On the final day of PARISE, researchers will conduct a focus group aimed at generating insightful feedback. Forecasters will have the opportunity to share new ideas that will help shape the future of the PAR network. As radar continues to develop and forecasting resources are enhanced, National Weather Service meteorologists will be better equipped to warn the public of impending severe weather. This, in turn, will support the NWS objective to protect life and property and will help to build a Weather Ready Nation.

Share this:

Multi-Radar Multi-Sensor Team To Receive NOAA Silver Medal

US_Dept_of_Commerce_Silver_MedalOne of the highest awards presented within NOAA will be awarded to the NSSL team that developed Multi-Radar Multi-Sensor, a system that helps forecasters manage the flood of weather data available to them. Under Secretary of Commerce Kathryn D. Sullivan announced the award of a NOAA silver medal for science/engineering achievement. Their work was a collaborative effort with the University of Oklahoma’s Cooperative Institute for Mesoscale Meteorological Studies.

The MRMS system, which became operational throughout the National Weather Service in October 2014, quickly harnesses the tremendous amount of weather data from multiple sources, intelligently integrates the information, and provides a detailed picture of the current weather. MRMS uses a holistic approach to merging multiple data sources, allowing forecasters to better analyze data and potentially make better predictions.

The new MRMS products, generated every two minutes, combine multiple radars, along with satellites, surface observations, upper air observations, lightning reports, rain gauges, and numerical weather prediction models. With this data, forecasters are able to better visualize high-impact weather threats like heavy rain, snow, hail, and tornadoes. This, in turn, leads to better forecasting techniques and improves lead time.

Congratulations to the team!



Share this:

NOAA Scientists tackle mystery of nighttime thunderstorms

PECAN researchers will deploy an armada of instruments after dark, including weather balloons. (Credit: NOAA).
PECAN researchers will deploy an armada of instruments after dark, including weather balloons. (Credit: NOAA).

This summer, more than 20 NOAA scientists will stay up late to learn why some thunderstorms form and grow at night, without the energy from the sun’s heat. They will be participating in the Plains Elevated Convection At Night (PECAN), a large, intensive field campaign to collect data before and during nighttime thunderstorms in the western Great Plains from June 1 to July 15.

PECAN researchers will deploy instrumented aircraft, ground-based instruments, mobile radars, and weather balloons to learn what triggers these storms, how the atmosphere supports their lifecycle, and how they impact lives, property, agriculture and the water budget in the region. Meteorologists believe these targeted observations will build understanding and ultimately improve forecasts of these sometimes damaging storms.

A nighttime thunderstorm near Scottsbluff, NE. Photo credit: Chris Spannagle.
A nighttime thunderstorm near Scottsbluff, NE. Photo credit: Chris Spannagle.

“Large nighttime thunderstorms are an essential source of summer rain for crops, but also produce widespread and potentially hazardous severe weather, excessive rainfall, flash flooding, and unusually frequent cloud-to-ground lightning,” said Conrad Ziegler, a research meteorologist at the NOAA National Severe Storms Laboratory and principal scientist for PECAN.  “Weather forecast models often struggle to accurately account for these. The PECAN field campaign will provide us with valuable insights—and improve our ability to save lives and property through more accurate forecasts.”

The PECAN field campaign will involve scientists, students, and support staff from eight research laboratories and 14 universities. The $13.5 million project is largely funded by the National Science Foundation (NSF), which contributed $10.6 million. Additional support is provided by NOAA, NASA, and the U.S. Department of Energy.

Nighttime storm triggers
Once the sun goes down, the Earth and its lower atmosphere usually loses heat and becomes more stable, an environment not so favorable for supporting thunderstorms.  In the Great Plains, however, many summer storms form after sunset, and sometimes without an obvious trigger.

PECAN scientists are interested in large complexes of thunderstorms called Mesoscale Convective Systems that can grow overnight, last for hours and often produce severe and hazardous weather. They will investigate how a low-level river of air triggers thunderstorms and supports storm evolution, what causes storms to grow into MCSs, and how MCSs respond to the surrounding environment.

In addition, PECAN researchers will test their hypotheses about how deep waves in the atmosphere form and ripple across the plains, like what happens with water when a stone is thrown in a pond, causing new storms to form after sunset. One type of atmospheric ripple is called a “bore.” Thunderstorms can create bores, but bores can also cause a thunderstorm to suddenly intensify. PECAN is the first modern campaign to study the role of bores and how they trigger and support Mesoscale Convective Systems.

Armada of instruments
More than 20 NOAA researchers and students will be responsible for gathering data with multiple instruments including the NOAA-X-Pol, a dual-pol mobile radar, two mobile balloon launch vehicles, and two “mobile mesonet” vehicles equipped with weather instruments. New to the fleet is the Collaborative Lower Atmosphere Mobile Profiling System (CLAMPS) designed by NSSL researchers to meet many of NOAA’s and its National Weather Service’s needs for lower atmosphere temperature, humidity and wind profiles. Additionally, one of the three aircraft participating in PECAN will be a NOAA Lockheed WP-3D Orion aircraft, best known for its hurricane hunting missions.

Unique to the experiment is an observation strategy that uses PECAN Integrated Sounding Array (PISA) stations to provide temperature, humidity, and wind profiles about every five minutes. The Department of Energy will provide six out of the eight ground-based upward-looking infrared spectrometer instruments. Dave Turner, NSSL scientist and PECAN steering committee member, will coordinate their operation.

Deploying in the dark
The campaign is based in Hays, Kansas, and will begin each day at 8 a.m. CDT.  A team of meteorologists, including retired forecasters from NOAA’s Storm Prediction Center, will work on a forecast for the upcoming night. At 3 p.m., scientists will use the forecast to determine where across northern Oklahoma, central Kansas, or south-central Nebraska to deploy their mobile resources. Teams will then ferry the instruments to the target area, set up, and collect data from dusk until after midnight. When the observation period is complete, they will ferry the instruments back to the base in Hays. A better understanding of these storms will have relevance for areas beyond the Great Plains, because clustered nighttime thunderstorms are common in various regions scattered across the globe.

Share this:

2015 Spring Warning Project will look at new severe weather warning guidance

Researchers and forecasters work side by side in the Hazardous Weather Testbed.

Several experiments to improve National Weather Service severe weather warnings will be conducted this spring in the NOAA Hazardous Weather Testbed (HWT) as part of the annual Experimental Warning Program, a joint project of the National Weather Service and NSSL/CIMMS to support NOAA’s goal to evolve the National Weather Service and build a Weather-Ready Nation. The EWP’s Spring Warning Project will run from May 4 through June 12, and provides a conceptual framework and a physical space to foster collaboration between research and operations to test and evaluate emerging technologies and science.

Forecasters will evaluate an updated Lightning Jump Algorithm (LJA), based on the GOES-R Geostationary Lightning Mapper, that was enhanced based on feedback from forecasters participating in the 2014 program. In severe storms, rapid increases in lightning flash rate, or “lightning jumps,” typically precede severe weather such as tornadoes, hail, and straight line winds at the surface by tens of minutes.  These evaluations will help prepare for possible operational implementation in 2016 following the launch of GOES-R.

Earth Networks’ total lightning and total lightning derived products, including storm-based flash rates tracks, time-series, and three levels of thunderstorm alerts will be evaluated in real time, building upon the initial evaluation in 2014. The 2015 evaluation will test the feasibility of use and performance under the stress of real-time warning operations.

A new set of high-resolution Weather Research and Forecasting (WRF) models will serve as a prototype for developing the “Warn-on-Forecast” warning paradigm. Feedback from this project will go into developing new model tools capable of managing the large amounts of model information associated with future forecast systems.

During three weeks of the experiment, forecasters will assess a new tool using rapidly-updating high-resolution gridded Probabilistic Hazard Information (PHI) as the basis for next-generation severe weather warnings. This experiment is part of a broad effort to revitalize the NWS watch/warning paradigm known as Forecasting a Continuum of Environmental Threats (FACETs). The major emphasis of the HWT PHI experiment will be on initial testing of concepts related to human-computer interaction while generating short-fused high-impact Probabilistic Hazard Information for severe weather. The long-term goal of this effort is to migrate the refined concepts and methodologies that result from this experiment into Hazard Services, the next generation warning tool for the NWS, for further testing and evaluation in the HWT prior to operational deployment.

This year will mark the inaugural HWT Experiment with Emergency Managers. The EMs will be provide feedback on their interpretation of experimental probabilistic forecasts generated in the HWT from the PHI experiment and the Experimental Forecast Program (EFP). This feedback will be used in conjunction with feedback from forecasters to refine how the uncertainty information is generated and disseminated.

Share this: