Earlier entries (to be updated) have gone over a few of the technical issues we’ve found during these first couple of events. Managed to find a new one on Friday.
Our flash sorting algorithm (i.e., w2lmaflash) seems to be having difficulty dealing with timestamps across multiple minutes from multiple domains coming in at the same time. For example, lets say the current time is 2012 UTC… the OK-LMA is sending 2009 UTC data (the more activity the slower the data transfer gets, up to about 4 min latency), but meanwhile there is no activity in the NA-LMA domain so the timestamp on the LDM feed the most recent data is 2012 UTC. This seems to be leading the w2lmaflash algorithm to be ‘prune’ (or drop) flashes that are more than a minute older (our current processing interval) than the most recent time from an LMA network…
At least that’s what I believe the problem is right now since it’s not reproducible in post-processing when it pulls from all the networks at the same time.
—-
I pulled all the data from Friday off the realtime feed and re-ran it over the weekend over the Oklahoma domain. The Norman tornadic storm is labeled as Cell 1 in all the figures and tables below using scale 1 from the w2segmotionll (kmeans) storm tracking.
The total flash rate (TFR) rate for this storm peaked at 282 flashes per min at 2244 UTC, with jumps occurring at 1835, 1844, 1853, 1940, 1952, 2012, and 2027 UTC (Fig. 1). Hail times from Local storm reports associated with this storm were recorded at 2041 (1 in), 2055 (1 in), and 2142 (1.5 in). The EF1 tornado moved through Norman, OK between 2102 and 2113 UTC (estimates from TDWR).
I’m discounting the lightning jump at 2012 UTC since there was no real-time radar merger between 2000 and 2008 UTC, hence nothing for the algorithm to track. Oddly, during this same time period [2002-2008 UTC] four stations in the OKLMA were also not communicating, leading to too few stations to do any quality control for flash sorting. So it is unknown whether the storm would have produced a lightning jump at this time had the merger/tracking been working correctly.
What is really interesting with this storm is how the character of the lightning changed prior to 2000 UTC and after 2010 UTC as the storm began it’s deviant motion right (towards Norman). The animation below (5 min intervals) catches this character as lightning initiations grow not only in number but also across the storm. The flash rate continues to increase with initiations spreading into the anvil region by the time the core of the storm is over Norman. This increase in flash rate and expanded coverage of lightning is reflective of change in the strength and size of the main updraft core over the same time period (linked via storm electrification processes in around the updraft region).
20120413_animation (quicktime movie, looks a bit better than gif animation below. Either way, you’ll need to click on the link above or image below to see the animation)
There was also a tornado reported in Mustang at 12:45 am CDT, but I haven’t had time to evaluate that storm yet.
Feel free to leave any questions or comments below or send them directly to me at kristin.kuhlman -at- noaa.gov
Regarding the timing of the lightning data ingest: it sounds as if this could seriously compromise real-time results, which will affect SHAVE performance and the guidance the SHAVE folks receive from the Lightning Jump analysis, but post-processing would not be affected. Is this correct?
Also – in the animation, check out the rogue \”left-mover\” (?) that zips in between the other cells (moving from south to north). Interesting . . .
Tom, in regards to timing of data.. this shouldn\’t really be a problem for SHAVE data collection. The visualization of a \’jump\’ may run about 4-5 min post actual jump, but that will be enough for them to flag a storm and begin collecting data in that area. Since they try to find the \’start-point\’ of severe weather it shouldn\’t be a big deal.
As far as post-processing goes the actual time is tied to the segmotion algorithm time. The lightning may be 5-min old, but it will be tied to radar-merger time as it is writing out every minute with the newest available data.
In post-processing we could re-run segmotion/storm tracking and then all the times will match. But I\’m not sure if that is the current plan.
Just got access to the blog (thanks Greg).
Kristin et al.
A few issues concern me:
1. What\’s the plan to insure that sufficient number of LMA sensors are operating/communicating in the storm region of interest to provide an accurate flash count? In research mode, this is obviously easier to police but I\’m wondering how we have confidence in the implementation here? I brought up this issue a while back. I\’m not sure it was ever resolved.
2. The latency issue sounds like a potential problem to me for real time LJ calculation (under at least some domain-wide high flash rate circumstances) unless you are going to post-process for actual verification.
3. Seems like quite a few jumps early on in this case. Maybe that\’s just the way it is but I\’m still not fully confident that the LJ algorithm (logic implemented in code) being run here is sufficiently close to S09/S11, if that\’s the intent of this test. I know Chris has had some sidebar communications with you over last few months but I\’d like to resolve the issue. I\’m open to suggestions on how to do that. Looking at code can be difficult if not everyone is familiar with computer language. Maybe you and Chris can each map it out (e.g., flow chart) with data flow shown so we can compare implementations? Just a suggestion.
4. Regardless of issues, I\’m guessing that this is a tornadic supercell case that LJA would have difficulties adding value to radar. No doubt a challenge. Of course, not all storms are supercells and I\’ve always personally hypothesized the LJA to have more potential value in other types of storms (e.g., pulse severe etc). Hopefully, we\’ll get to see some of that too.
Thanks