We are really happy when public safety agencies use the imagery from http://ondemand.nssl.noaa.gov/ to show the impact of the recent tornadoes in Illinois. Hey, that’s our stuff, we want to shout. It reminds us of why we do what we do.
But there’s a lot of noise on those accumulation products, noise that can be removed by the use of Multiple Hypothesis Tracking (MHT). We couldn’t do MHT on-demand or in real-time because it is so slow, but just a few weeks ago, we figured out a way to do it faster with not much of a tradeoff in noise removal.
We repeated the analysis of the Illinois outbreak using the faster method and boy, is it faster! We can process an hour of data in 5 minutes! (Here are the cleaned-up rotation tracks for the Illinois tornadoes. It is a KML file, so view it in Google Earth.) MHT will be implemented on the ondemand website in a few days. So, the next time you see folks sharing Rotation Tracks images, they will be cleaner and crisper.
One of the best ways to improve the quality of the Rotation Tracks products is to apply spatial QC using hysteresis and temporal QC using Multiple Hypothesis Tracking.
Unfortunately, this used to be quite slow. An hour of azimuthal shear data covering the CONUS could take as much as two hours to process. Therefore, it was used only in research studies and off-line, but not to produce the post-event rotation tracks that you can download from http://ondemand.nssl.noaa.gov/
w2accumulator’s -Q option now supports two vastly more efficient optimizations. You can specify that the number of hypothesis is 1 (meaning to only keep the best track, and not bother about second-best, third-best, etc.) or that the algorithm should retain all potential tracks (specifying -1 for number of hypotheses). These are the most likely values that you will want to specify and with these, the algorithm runs 20x faster. Yup, you can now process an hour of data in about 6 minutes.
CPU Time (microseconds)
You used to have only the first two options for -Q available. Now, you have two more, and these two “special” values are highly optimized.
What’s the impact of these options? (Open the images in different tabs in your browser and switch between them so that you can see the differences between the last two images more readily)
For more details about MHT-QC and its application to rotation tracks products, please see these scientific articles:
w2qcnndp is a WDSS-II algorithm that employs polarimetric moments to do quality-control of weather radar reflectivity data. Single-pol QC (via w2qcnn) has lots of problems distinguishing bioscatter from light precipitation, but polarimetric moments (variance of Zdr, especially) help w2qcnndp outperform w2qcnn in terms of removing bioscatter. If you are using w2qcnn on US weather radar data from after the polarimetric upgrade, you should definitely start using w2qcnndp instead.
w2qcnndp used to have lots of problems with electronic interference and with sun spikes. Recently, we added two modules to w2qcnndp (they are turned on by default; use -m “-sunstrobe -electronicinterference” to turn these off). Here are a couple of examples to show the impact of these modules.
First, electronic interference:
w2qcnndp used to have problems with sun spikes (sun strobes) that were connected to valid echoes, but this has been improved:
For more details about w2qcnndp, please refer to the following two scientific articles:
Users of WDSS-II range from students and researchers who download WDSS-II from the wdssii.org website, engineers at private companies that license WDSS-II from the University to forecasters at government agencies and meteorological departments.
A question that I often get from users of WDSS-II is: “What’s new? What have you done in the past X months?”. This blog is an attempt to answer that. We will add examples of changes we have made to small parts of the system, the sort of changes that can go unnoticed in a system where the names of the tools and algorithms tend to remain the same but the tools get more and more powerful and the algorithms more and more skillful.
Another purpose of this blog is to point WDSS-II users to tips and tricks. If you have a cool way to use or customize a WDSS-II application, please consider contributing a guest post.
Questions about use of the software or products, however, should be posted on the WDSS-II forum.