Relative skill

During the Thursday CI forecast we decided to forecast for a late show down in western Texas where the models were indicating storms would develop. The models had a complex evolution of the dryline and moisture return from SW OK all the way down past Midland, TX. There was a dryline that would be slowly moving southeast and what we thought could be either a moisture return surge or bore of some kind moving northwest. CI was being indicated by most models with large spatial spread (from Childress down to Midland) and some timing spread (from 03 to 07 UTC) depending on the model. More on this later.

Mind you, none of these models had correctly predicted the evolution of convection earlier in the day along the TX panhandle-western OK border. In fact the dryline in the model was well into OK and snaked back southwest and remained there until after 23 UTC. The dryline actually made it to the border area, but then retreated after the storms formed southwest of SW OK in TX. This was probably because of the outflow from these storms propagating westward. This signal was not at all apparent in the models, maybe because of the dryline position. The storms that did form had some very distinct behavior with storms that formed to the north side of the 1st initiation episode moving North, not east like in the models. The southern storms were big HP supercells, slowly moving east northeast, and continually developing in SW OK and points further SW into TX (though only really the first few storms were big; the others were small in close proximity to the big storms – a scale enigma). We had highlighted the areas to the south in our morning forecast, along with an area in KS to the North but left a sight risk of CI in between. So while our distinct moderate risk areas would sort of verify in principle (being two counties further to the east than observed) we still did not have the overall scenario correct.

That scenario being, storms developing in our region and moving away, with the possibility for secondary development along the dryline a bit later. Furthermore we expected the storms to the north to develop in our moderate risk area and move east. When in fact the OK storms moved into our KS region just prior to our northern KS moderate area verifying well with an unanticipated arcing line of convection. This was a sequence of events that we simply could not have anticipated. We have discussed many times having the need to “draw what the radar will look like in 3 hours”. This was one of those days where we could not have had any skill whatsoever in accomplishing that task.

Drawing the radar in 1 or 2 or 3 hours is exactly what we have avoided doing given our 3 hour forecast product. We and the models, simply do not have that kind of skill at the scales where it will be required to have added value. This is not so much a model failure or even human failure. It is an operational reality that we simply don’t have enough time to efficiently and quickly mine the model data to extract enough information to make a forecast product. More on this later.

Back to the overnight convection. So once these SW OK supercells had been established we sought other model guidance, notably the HRRR from 16 or 17 UTC. By then it had picked upon the current signal and was showing a similar enough to observations evolution. This forecast would end up being the closest solution, but to be honest was still not that different way down in TX to the ensemble which was a 24-30 hour forecast. They all said the same thing: dryline boundary and moisture surge would collide, CI would ensue within 2 hours into a big line of convective storms that would last all night and make Fridays forecast very difficult.

Sure enough these boundary collisions did happen. From the surface stations point of view, winds in the dry air had been blowing SW all day with temps in the upper 80’s low 90s. After 02 UTC, the winds to the NW had backed and were now blowing W with some blowing W-NW. While at the dryline, winds were still SW but weakening. Ahead of the dryline, they were SE and weak. By 0400 UTC, the moisture surge intensified from weak SE winds to strong SE winds, with the dew point at CDS increasing from 34 to 63 in that hour. On radar the boundaries could be seen down in Midland, as very distinct with a clear separation:


You can see CI already ongoing to the north where the boundaries have already collided and the zipper effect was in progress further southwest but it took nearly 2 more hours.


Also note the “gravity waves” that formed in the upper level frontal zone within a region of 100 knots of vertical shear back in NM. Quite a spectacular event. Let me also note that the 00 UTC ensemble and other models DID NOT pick up this event, until 3 hours later than shown by the last radar image. Spin up may have played a significant role in this part of the forecast. As you can see, the issues we face are impressive on a number of levels, spatial, and temporal scales. We verified our forecast of this event with the help of the ensemble and the HRRR and the NSSL WRF. To reiterate the point of the previous post: It is difficult to know when to trust the models. But in this case we put our faith in the models and it worked out, whereas in the previous forecast, we put our faith in the models and we had some relative skill, but not enough to add value.

Not so fast

What can I say. We verified our Tuesday forecasts and felt pretty good capturing the elevated convection over KS and OK at about the right time (1st lightning strike was 45 minutes before our period start time) for an elevated convection event and we rocked it out in CO during the day.

Which brings us to Wednesday. A slight risk in OK with a triple point from a warm front, stationary front and dryline in place. The dryline was forecast to be nice and strong, with horizontal convective rolls (HCRs) present on the north side interacting with the dryline circulation, and more HCRs in the dry air behind the dry line further south. The end result was a series of CI failures (indicated exclusively by accumulated precipitation) along the dryline, but eventually classic right moving supercells (1 or 2 dominant and long lived) were present in the ensemble. The HCRs were detected via some of the unique output variables we are experimenting with, particularly vertical velocity at 1.1 km AGL.

The complicating factor was that not all members had supercells, perhaps 1 in 3. We also had a situation where CI was very sensitive to the distribution of mixing ratio in the warm sector. It appeared we had two different behaviors along the dryline southward, but a definite moisture pool was dominant to the north. This pool was in the area where the dryline bent back westward and where the HCRs directly interacted with the dryline/warm front boundary. There was very little vertical velocity in the warm sector. Not sure why this was the case, assuming the PBL heights were not much lower than 1.1 km.

But lets be serious. There was not a whole lot of over-forecasting by the models. Storms did attempt to form. They just didn’t last very long or get very strong. Nor could we really call them storms (having met no single definition of CI other than some coherent weak reflectivity). In this case it appears the strongest forcing (the dryline and HCRs) was separated from where the realizable instability was present. We analyzed where, when, and how the instability could be realized (in great detail) in the model. We could not verify these models with observations because we don’t have very many sounding sites nor do we have frequent launches.  What we could not do is pinpoint where this forecast was going wrong.

The KOUN sounding is presented below:


Note that this sounding has nearly 3000 J/kg CAPE with 43 knot deep layer shear, and strong (31 knot) 0-1km shear. An ideal sounding for supercells, and possibly tornadoes. But Norman is well away from where the dryline was setup in western OK. There are no sounding sites in SW OK or NW OK. If we look at LMN which is north of OUN and happens to be north of the warm front:


We see very little instability because of cooler surface temperature and dew point and an elevated capped layer. Modifying this sounding for surface conditions at OUN would indicate strong instability and no cap using the virtual parcel. It is unlikely to be this easy as there is probably some mesoscale variability that has not been sampled in this area.

What is clear is that the forecasters were very much willing to buy into a reasonable solution. What we lacked was a solid reason to not believe the models. I am assuming we should first believe the models and that is perhaps not the best starting point. So lets reverse that thought: what reasons did we use to believe the models? I won’t speak for the group, but we should address this question when we review this event tomorrow.

Another point: Assume the model is not a poor representation of observations. What if it was very close? How could we recognize the potentially small errors which could lead to the development of storms or the lack there of? These are really fundamental questions that still need to be addressed.

On a day like today it would have been highly valuable to have soundings in key storm-scale locations near the dryline in the warm sector to the east, and to the immediate north between the dryline and warm front. At the very least we can ascertain if the model was depicting the stratification correctly and also the moisture content, instability and inhibition. It would have been great to have a fine scale radar to measure the winds around the dryline to look for HCRs.