CMC > NCEP?

CMC pulls ahead of the GFS? We're #4?

CMC Anomaly Correlation solidly #3 since August 2019...Waz Up Canada? Eh?

Mike Fiorino

20191125

The NWP league table

I casually follow the NWP scores at: EMC Stat Page (Pete Kaplan's long-running web page used at all global model meetings at EMC during my time there in the 1990s).

Here are the latest 5-d NHEM stats:

The  usual pecking order, or what I consider the 'league table' is:

1. ECMWF
2. UKMO
3. NCEP (GFS)
4. CMC
5. NAVY 

i.e., we (the American Model) are typically #3. Since August CMC has pulled ahead of the GFS and as of today is 1 point higher.  My big question is how the CMC global model/data assimilation system has changed...

Taking a longer term view:

The color scheme: ECMWF; UKMO; CMC; Navy

ECMWF has been ichi ban for over 20 y...the MetOffice #2 and CMC almost always #4.

 

Is CMC the new #3?  How good is the ACC?

I appreciate that the anomaly correlation (ACC) does not measure the entire quality of an NWP modeling system...  However, I hold that the 5-d 500 mb NHEM ACC (5NACC) is a kind of 'magic number' -- highly correlated with skill in other areas/forecast times/variables as seen in the 'scorecard.'

For example, from the latest ECMWF implementation in June 2019:

https://www.ecmwf.int/sites/default/files/elibrary/2019/19156-newsletter-no-160-summer-2019.pdf

Again, while this single score only tells part of the story, general model skill does follow the 5NACC -- it's a necessary but not sufficient condition for model improvement.

The importance of this score in NWP was made very clear to me during my 1.5 year secondment to ECMWF 1998-99 to work on the ERA-40 reanalysis.  I developed a scheme to assimilate the tropical cyclone (TC) 'vitals' (working best track data on TC position, movement, intensity and other structure parameters).  I tested the scheme in the full ERA-40 version of the IFS and a 1point degradation in the 5NACC was the reason why TCs were not assimilated.  At the time, there was zero tolerance for any model change lowering the 5NACC (still true today?)...

 

Are TC forecasts consistent with the 5NACC?

I was curious if the apparent improvement implied by the 5NACC was reflected in TC track prediction...

The short answer: maybe in the atLANTic but not in Western north PACific.

Historically the CMC global model has not been a very good TC forecast aid and is known to have hyper-active tropical convection that causes excessive TC genesis especially in WPAC.  The same is true in EPAC.  The CMC model has shown some track prediction skill in the LANT relative to the GFS.

My standard score for TCs is the 72-h (3-d) mean position error (3MPE)because roughly 2/3 of all official forecasts will have a verifying 72-h position and beacause 3-d is about 1/2 of the mean life cycle of a TC.

In 2018 the LANT 3MPE:

• CMC: 122 nmi (85 cases)
• GFS: 105 nmi (85 cases) --  GFS lower (better) by 17 nmi

For the period 2019081500-2019112500 (today)

• CMC: 128 nmi (73 cases)
• GFS: 143 nmi (73 cases)  -- GFS higher (worse) by 15 nmi

 Here are the full position error plots (i.e., where the numbers above come from) :

2018 CMC v GFS LANT

2019081500-2019112500 CMC v GFS LANT

In WPAC the 3MPE for the same periods as in the LANT

In 2018 WPAC 3MPE:

• CMC: 142 nmi (179 cases)
• GFS: 119 nmi (179 cases) -- GFS lower (better) by 23 nmi

For the period 2019081500-2019112500 (today) in WPAC:

• CMC: 171 nmi (89 cases)
• GFS: 130 nmi (89 cases)  -- GFS lower (better) by 41 nmi

Some Bottom Lines:

I also read the area forecast discussion put out by WFO Denver/Boulder and have recently found references to the Canadian model as part of their prognostic reasoning.  Are the forecasters seeing the improved CMC model?

Why is the new GFS struggling to keep up with the NWP leaders in the UK; and now our friends to the North?

I could offer a few reasons, but to me the big one is that modern (> 2000) NWP is almost entirely a problem of physics, not dynamics (spatial resolution at the hydostatic limit is only significant in its interaction with physics).  Furthermore, observations and data assimilation only matter when the innovations are small (i.e., a good model that 'looks' like the obs).

At the end of the day it's (still) the model and that means physics.

HAGIBIS > FAXAI @ Tokyo?

HAGIBIS (20W.2019) > FAXAI (14W.2019)?

intensity-wise no...strength-wise YES!

Mike Fiorino 

21 October 2019

0330 UTC

In my September FAXIA-BLOG-POST I concluded this was the strongest typhoon to hit Tokyo, but now I wonder if typhoon HAGIBIS that struck Tokyo was stronger?

The first Japan Times article I read (shortly after landfall) drew comparisons to typhoon IDA of 1958, but the bigger difference vis-a-vis FAXAI was the horrendous rain and severe flooding.

My choice of 'strongest' typhoon to hit Tokyo was rather poor in hindsight...I should have said 'most intense' as in the TC's maximum surface wind speed.  Let's compare the JTWC working best track for both storms around landfall (using 35.7N 139.7E as the latitude/longitude of Tokyo)

FAXAI 14W.2019:

2019090812      14W.2019 100  952  34.0N  139.1E  129  65  350.5 12.7 B  lf: 0.00     
2019090818      14W.2019 090  958  35.3N  139.7E   94  49   14.1 11.9 B  lf: 0.69     
2019090900      14W.2019 075  982  36.3N  141.0E   78  36   34.0 13.9 B  lf: 0.18     

HAGABIS 20W.2019:

2019101206      20W.2019 085  951  33.7N  138.2E  241 141   19.3 15.4 B  lf: 0.00   
2019101212      20W.2019 075  960  35.6N  139.4E  289 125   24.8 19.8 B  lf: 0.69  
2019101218      20W.2019 065  970  38.4N  142.1E  289 115   33.8 28.3 B  lf: 0.04 

The closest 6-h best track position for HAGIBIS has an intensity of 75 kts whereas FAXAI was at 90 kts as highlighted above...  Thus, FAXAI was more intensity...  However, the mean radius of 34 kt winds (sometimes used as a measure of TC strength) or R34 for HAGAIS was 289 nmi but for FAXAI was 94 nmi.  Thus, HAGIBIS was much stronger!!!

The plot below gives a histogram of mean R34 in WESTAC using JTWC best track data 2009-2019:

Focus on the green bars -- the distribution for analysis time (or 'tau 0') for all verifiable positions.  The mean is 107 nm and the median 101 nmi.  R34 > 250 nmi is very rare and HAGIBIS at this time was almost record setting!!!

In terms of Integrated Kinetic Energy (IKE - Powell, M. D. and T. A. Reinhold, 2007: Tropical cyclone destructive potential by integrated kinetic energy. Bull. Amer. Meteor. Soc., 87, 513-526) based on the symmetric wind profile in Fiorino and Elsberry 1989: Some aspects of vortex structure related to tropical cyclone motion, J. Atmos. Sci, 46, 976-990 that only requires intensity Vmax and R34 we find:

FAXAI: Vmax: 90.0 kts; R34: 94 nmi -- IKE: 50 Tj
HAGIBIS: Vmax 75.0 kts; R34: 289 nmi -- IKE: 273 Tj

or HAGIBIS was around 5.5X stronger than FAXAI

The only (small  😢) saving grace for Japan was that HAGIBIS was moving quickly at 20 kts at landfall and accelerating during extra-tropical transition whereas FAXAI was slower at 12 kts and more tropical

The much larger rainfall impact of HAGIBIS is partly explained by the massive size of the circulation.  Another reason is that the storm was becoming extra-tropical as it moved over Honshu.

2019 will certainly be historical for Japan:  the year of the most intense typhoon (FAXAI) to strike Tokyo and the strongest ever storm HAGIBIS to impact Tokyo, the Kanto plain the all of northern Honshu.  All I can say is WOW and thoughts and prayers for Japan.

FAXAI Tokyo

FAXAI (14W.2019) -- strongest typhoon to hit Tokyo? -- YES

Mike Fiorino 

09 September 2019

0330 UTC

I've done this search before...and now that the excellent JTWC forecasts -- that FAXAI would hit Tokyo -- have verified...I've taken another look...

here is the latest working JTWC best track:

Using the lat/lon of Tokyo as 35.7N 139.7E  we find from the 18Z 08 SEP 19 position:

2019090818      14W.2019 090  958  35.3N  139.6E   94  49    0.0 13.0 C  TY WN  RCB  42/43  lf: 0.69 FAXAI

Breaking down the position:

2019090818      date-time-group YYYYMMDDHH

14W.2019        NNB.YYYY -- storm number NN subbasin B

090             max 10-m 1-min surface wind [knots] 

958             central pressure [mb]

35.3N  139.6E   latitude  longitude

94              mean radius of 34 kt winds [nmi]

49              mean radius of 50 kt winds [nmi]

C  TY WN        C: source is the CARQ card

RCB  42/43  

lf: 0.69 FAXAI  lf: is %land on a 0.5 deg grid

this position 35.3N 139.6E is almost a direct hit...  However, this is the JTWC working best track and so the intensity (90 kt) and position will likely change but it is highly unlikely to be more than 10 kts and 0.5 deg..

Is 90 kts the strongest ever?  Consulting the digital typhoon and searching a 1.0 deg box around the Tokyo lat/lon there were 11 typhoons:

1       195202       DINAH      W. N. Pacific   1952-06-20 06:00        1952-06-24 18:00        4 Days 12 Hours         960
2       195405       GRACE      W. N. Pacific   1954-08-12 00:00        1954-08-20 12:00        8 Days 12 Hours         940
3       195822       IDA        W. N. Pacific   1958-09-20 18:00        1958-09-27 00:00        6 Days 6 Hours          877
4       195906       ELLEN      W. N. Pacific   1959-08-02 18:00        1959-08-10 00:00        7 Days 6 Hours          965
5       196517       LUCY       W. N. Pacific   1965-08-15 06:00        1965-08-23 00:00        7 Days 18 Hours         940
6       197113       IVY        W. N. Pacific   1971-07-05 00:00        1971-07-08 18:00        3 Days 12 Hours         990
7       197129       CARMEN     W. N. Pacific   1971-09-25 18:00        1971-09-26 18:00        1 Days 0 Hours          990
8       198514       RUBY       W. N. Pacific   1985-08-28 00:00        1985-09-01 00:00        4 Days 0 Hours          985
9       198615       NO-NAME    W. N. Pacific   1986-09-02 06:00        1986-09-03 00:00        0 Days 18 Hours         992
10      200115       DANAS      W. N. Pacific   2001-09-04 00:00        2001-09-12 06:00        8 Days 6 Hours          945
11      201506       NOUL       W. N. Pacific   2015-05-03 18:00        2015-05-12 06:00        8 Days 12 Hours         920

which are shown below:

finding the positions closest to Tokyo from the JMA and JTWC best tracks here's a listing of the intensities for the 11 typhoons before FAXAI:

1       195202       DINAH      55
2       195405       GRACE      50? NOT IN JTWC BT
3       195822       IDA        70
4       195906       ELLEN      50
5       196517       LUCY       55 interp
6       197113       IVY        40
7       197129       CARMEN     45
8       198514       RUBY       45
9       198615       NO-NAME    40 NOT in JTWC BT
10      200115       DANAS      65 interp
11      201506       NOUL       40

IDA (15W.1958) at 70 kts and DANAS (06W.2001) 65 kts were the previous strongest typhoons near Tokyo. 

Conclusion: FAXAI (14W.2019) at 90 kts is the strongest typhoon to hit Tokyo!

a full listing of the data is show below:

1       195202  DINAH      W. N. Pacific        1952-06-20 06:00        1952-06-24 18:00        4 Days 12 Hours         960
1952    6       23      18      35.8    139.9   984

1952062312      02W.1952 055 ----  33.9N  136.4E  --- ---   51.7 34.7 b  NT NW  ---  19/27  lf: 0.32          
1952062318      02W.1952 050 ----  35.3N  140.1E  --- ---   64.6 35.0 b  NT NW  ---  20/27  lf: 0.67 


2       195405  GRACE      W. N. Pacific        1954-08-12 00:00        1954-08-20 12:00        8 Days 12 Hours         940
1954    8       19      12      35.4    139.4   991   # not in JTWC BT -- XT?


3       195822  IDA        W. N. Pacific        1958-09-20 18:00        1958-09-27 00:00        6 Days 6 Hours          877
1958    9       26      12      34.4    139.0   955  
1958    9       26      18      36.6    140.4   980

1958092612      15W.1958 070 ----  34.5N  139.5E  --- ---   31.7 20.0 b  NT NW  ---  25/30  lf: 0.07          
1958092618      15W.1958 070 ----  36.8N  140.5E  --- ---   28.1 22.7 b  NT NW  ---  26/30  lf: 0.86  


4       195906  ELLEN      W. N. Pacific        1959-08-02 18:00        1959-08-10 00:00        7 Days 6 Hours          965
1959    8       9       06      35.2    138.8   980
1959    8       9       12      36.0    141.0   984

1959080906      06W.1959 050 ----  35.4N  138.9E  --- ---   67.9 16.0 b  NT NW  ---  29/42  lf: 0.84          
1959080912      06W.1959 050 ----  36.0N  141.3E  --- ---   72.2 18.0 b  NT NW  ---  30/42  lf: 0.00          


5       196517  LUCY       W. N. Pacific        1965-08-15 06:00        1965-08-23 00:00        7 Days 18 Hours         940
1965    8       22      18      35.5    139.6   990

1965082212      18W.1965 070 ----  34.9N  138.9E  --- ---   35.6  9.2 b  NT NW  ---  31/34  lf: 0.47          
1965082218      18W.1965 045 ----  35.8N  139.7E  --- ---   39.3 11.0 b  NT NW  ---  32/34  lf: 0.94  


6       197113  IVY        W. N. Pacific        1971-07-05 00:00        1971-07-08 18:00        3 Days 12 Hours         990
1971    7       7       12      34.9    138.0   992
1971    7       7       18      36.0    140.4   994


1971070712      13W.1971 040 ----  35.0N  138.1E  --- ---   45.5 19.3 b  TW NW  ---  10/11  lf: 0.77          
1971070718      13W.1971 030 ----  36.1N  140.4E  --- ---   54.0 21.3 b  TW NW  ---  11/11  lf: 0.5


7       197129  CARMEN     W. N. Pacific        1971-09-25 18:00        1971-09-26 18:00        1 Days 0 Hours          990
1971    9       26      12      35.5    139.2   994

1971092606      28W.1971 045 ----  34.4N  136.2E  --- ---   25.8 33.3 b  TW NW  ---   9/10  lf: 0.94          
1971092612      28W.1971 040 ----  35.8N  139.2E  --- ---   46.6 29.1 b  TW NW  ---  10/10  lf: 0.99        


8       198514  RUBY       W. N. Pacific        1985-08-28 00:00        1985-09-01 00:00        4 Days 0 Hours          985
1985    8       30      12      34.8    139.5   992     50

1985083012      14W.1985 050 ----  34.7N  139.7E  --- ---  354.3 12.6 b  TW WN  ---  15/21  lf: 0.07          
1985083018      14W.1985 040 ----  36.0N  139.9E  --- ---    5.6 12.6 c  TW WN  ---  16/21  lf: 0.87         

9       198615  NO-NAME    W. N. Pacific        1986-09-02 06:00        1986-09-03 00:00        0 Days 18 Hours         992  # not in jtwc BT !!!
1986    9       3       00      34.5    139.1   996
1986    9       3       06      36.1    139.9   997


10      200115  DANAS      W. N. Pacific        2001-09-04 00:00        2001-09-12 06:00        8 Days 6 Hours          945
2001    9       11      03      35.8    139.9   974     55

2001091100      19W.2001 070  972  35.1N  139.5E  170  70   39.0  9.0 C  TY WN  ---  35/40  lf: 0.29          
2001091106      19W.2001 060  980  36.4N  140.4E  165  65   37.0 12.0 C  TY WN  ---  36/40  lf: 0.66         


11      201506  NOUL       W. N. Pacific        2015-05-03 18:00        2015-05-12 06:00        8 Days 12 Hours         920
2015    5       12      12      34.3    138.0   996
2015    5       12      18      38.2    142.2   996

2015051212      06W.2015 040  993  34.4N  138.4E  --- ---   50.0 46.0 C  TS NW  ---  50/51  lf: 0.20 NOUL     
2015051218      06W.2015 040 ----  37.6N  144.1E  --- ---   54.1 55.5 c  TW NW  ---  51/51  lf: 0.00  


12      14W.2019
2019090818      14W.2019 090  958  35.3N  139.6E   94  49    0.0 13.0 C  TY WN  RCB  42/43  lf: 0.69 FAXAI

python implementation of CLIPER

A Python Implementation of CLIPER

Mike Fiorino

NOAA ESRL Boulder CO

14 November 2016

michael.fiorino@noaa.gov

1.0 CLIPER model

The CLImatology and PERsistence statistical model of tropical motion is often used as a baseline 'no-skill' aid.  Percent improvement (smaller) of mean position error over CLIPER defines 'skill.'  The classic implementation by Charlie Neumann makes 12-72 h position forecasts in all TC basins globally.  Although the classic CLIPER regression model has been updated with more best track data and extended to 120-h forecasts, the Neumann scheme is still used as a baseline...which begs the (Bill Gray) question -- "why are you doing this Mike?"  The answer is that I'm reviewing a paper and want to generate CLIP forecasts. It would also be good to run CLIPER with the same initialization as for the model trackers I run using fields from ECMWF/NCEP/CMC/UKMO/FNMOC...

2.0 .py implementation

Using .f code in the ATCF at both JTWC and NHC circa 2008, I put a .py wrapper on the subroutines (using 'f2py') and ran the model for the 2012-16 seasons using both CARQ (operational positions and motion) and best track data, and then compared to the operational runs from the JTWC/NHC adecks (the CLIP aid).

Of course I did not get the same answer as in operations even though I used the operational CARQ positions.  The results using the best track suggests that part of the difference may come from a different initialization as the .py implementation forecasts were better at tau 0 to 36 h...

The JTWC code: jtwc.cliper.lib.f and the NHC code: nhc.cliper.lib.f.

The question to Buck Sampson (and JTWC/NHC) is this the current code? 

3.0 2012-16 results 

Here are the mean position errors from CLIPER:  1) local .py; 2) operational JTWC/NHC adecks; and 3) local .py with the best track.

3.1 NIO (both Arabian Sea and Bay of Bengal)

 

2012-2016 NIO CLIPER mean position error [nm].  the first bar comes from the local .py with operational initial position and motion; the next from the JTWC adecks and the third and darkest from the local .py using the best track.  Note that the initial position error with the best track is 0 nmi.

 In the NIO, my local .py has lower error than in operations, because ??? (run incorrectly)

 3.2 WPAC (western North Pacific)

2012-2016 WPAC CLIPER mean position error [nm].  the first bar comes from the local .py with operational initial position and motion; the next from the JTWC adecks and the third and darkest from the local .py using the best track.  Note that the initial position error with the best track is 0 nmi.

 

The best track version is clearly the best at 0-36 h, but my local .py is about 5% larger that the operational JTWC CLIP.

3.3 EPAC (eastern North Pacific (140W-60W))

2012-2016 EPAC CLIPER mean position error [nm].  the first bar comes from the local .py with operational initial position and motion; the next from the NHC adecks and the third and darkest from the local .py using the best track.  Note that the initial position error with the best track is 0 nmi.

The improvement using the best track in EPAC is confined to 0-24 h.  the 48-h and 72-h with the best track are actually higher that the NHC operational runs!

3.4 LANT (North atLANTic) 

2012-2016 LANT CLIPER mean position error [nm].  the first bar comes from the local .py with operational initial position and motion; the next from the NHC adecks and the third and darkest from the local .py using the best track.  Note that the initial position error with the best track is 0 nmi.

 

 

 

 

 

 

 

 

 

Similar results as in EPAC...

4.0 Comments and Questions

As Charlie Neumann discovered many years ago, the LANT is the 'hardest' basin to forecast because the climatology and persistence have the highest position error.  In contrast, EPAC is the 'easiest' with mean errors 30-40% lower than in the LANT.  WPAC is the next hardest basin and the NIO is between EPAC and WPAC.

The NIO errors are puzzling in that my local implementation does better than in operations and note the strong sensitivity to the initial position and motion..

The code I'm using in EPAC/LANT must be slightly different than at NHC since I cannot reproduce their CLIP even using the best track...  

I need