CMC pulls ahead of the GFS? We're #4?
Update on 20200130: CMC better in Summer Hemisphere?
Mike Fiorino
30 January 2020
Recap of the November Blog
In my first post: https://wxmapstertc.blogspot.com/2019/11/cmc-ncep.html I discussed how the CMC GDPS (Global Deterministic Prediction System -- the Canadian model) was outperforming the NCEP GFS (the American model) in a very basic measure of global model forecast quality -- the 5-day northern Hemisphere (NHEM) height Anomaly Correlation (5DNACC).
I acknowledged that a few points do not make a trend, but at the time hypothesized that the improvements in Canadian model came from new/better physics ...
I asked two questions:
- when/what were the apparent changes in the CMC GDPS (Global Deterministic Prediction System)?
- was the gap between the NCEP.GFS and the CMC.GDPS a blip or a trend?
And he provided these links describing the changes:
- science: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019MS001781
- tech: http://collaboration.cmc.ec.gc.ca/cmc/cmoi/product_guide/docs/changes_e.html
Recent Trends...
Pete Kaplan's EMC Stat Page (the long-running web page used at all global model meetings at EMC during my time there in the 1990s) has been down for 'technical reasons'(?), but here are the latest stats from the EMC VSDB (https://www.emc.ncep.noaa.gov/gmb/STATS_vsdb/):
 |
The new CMC GDPS model has been running since July 2019 so we have 7 months to compare in the text table below:
NHEM SHEM
CMC GFS CMC-GFS CMC GFS CMC-GFS
201906: .866 .876 -0.010 N/A
201907: .863 .871 -0.008 .883 .889 -0.006
201908: .897 .890 +0.007 .882 .891 -0.009
201909: .876 .874 +0.002 .885 .891 -0.006
201910: .898 .894 +0.004 .908 .899 +0.009
201911: .907 .911 -0.003 .914 .911 +0.003
201912: .904 .911 -0.007 .903 .903 +0.000
202001: .913 .907 +0.004 .887 .875 +0.012
What's most interesting is how the Canadian model does better in the summer hemisphere.
We're not quite at the peak of austral TC activity (around February), but it looks like the convection changes are really improving the CMC height scores and confirms (IMHO) how the tropics impacts the quality of the midlatitude forecasts.
Some final thoughts...
Cliff Mass' recent blog post on US NWP (https://cliffmass.blogspot.com/2020/01/the-future-of-us-weather-prediction.html) makes a compelling argument that American operational NWP is fundamently broken with perhaps the greatest dysfunction (unstated there) in the US Navy 😟. In the NHEM plot above, the Navy global model is barely competitive with the CFSR! This is very distressing to me as a retired Naval Oceanography officer that implemented the first operational two-way interactive, moving nested-grid TC model in 1981...
I'm seeing a real trend here and at the rate we're going, American NWP will be 4th rate for the foreseeable...
While Cliff correctly identifies our problem mostly as one of leadership (I agree), the (much?) bigger in my mind is the data handling systems at NCEP NCO. Using the unix filesystem to 'manage' gridded fields is appalling primitive and at least 30 years behind the rest of the world! Although the Navy global model is clearly in last place, the data systems at FNMOC have always been top notch.
Data handling is certainly not a sufficient condition, as demonstrated by the lack of performance by the Navy global model, but it is a necessary condition. I cannot see American NWP at NCEP progressing without a fundamental change in the nitty gritty of data, a fully-funded and outside-DC EPIC notwithstanding.
