the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 01 Oct 2024
Improving Accuracy and providing Uncertainty Estimations: Ensemble Methodologies for Ocean Forecasting
Abstract. Ensemble forecasting has emerged as an essential approach for addressing the uncertainties inherent in ocean prediction, offering a probabilistic framework that enhances accuracy of both short-term and long-range forecasts. By more effectively addressing the intrinsic chaotic nature of mesoscale and sub-mesoscale variability, ensemble methods offer critical insights into forecast errors and improve the reliability of predictions. This paper reviews the ensemble methodologies currently used in ocean forecasting, including techniques borrowed from weather prediction like virtual ensembles and Monte Carlo methods. It also explores the latest advancements in ensemble data assimilation, which have been successfully integrated into both ocean general circulation models and operational forecasting systems. These advancements enable more accurate representation of forecast uncertainties (error-of-the-day) by sampling perturbations conditioned on available observations. Despite the progress made, challenges remain in fully realizing the potential of ensemble forecasting, particularly in developing tools for analyzing results and incorporating them into decision-making processes. This paper highlights the crucial role of ensemble forecasting in improving ocean predictions and advocates for its wider adoption in operational systems.
- Preprint
(768 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on sp-2024-10', Anonymous Referee #1, 07 Oct 2024
The paper provides a brief description of the motivation for producing ensemble ocean forecasts together with an overview of the different methods for generating them. A brief description of how ensembles are assessed is also provided. A table is included which lists some existing operational ensemble ocean forecasting systems. The paper is a useful introduction to the methods for ocean ensemble forecasting and the status of the field.
Main comments:
- The structure of the text in section 1.1 could be improved. The text doesn’t seem to clearly follow the different options for ensemble production as shown in Fig. 2.
- More references could be included to give the reader more information about particular advances, as suggested in the minor comments below.
- Table 1 is quite an ad hoc selection of different systems and not all up to date. Some suggestions for improving it are included in the minor comments below.
Minor comments:
- The section numbering seems quite ad hoc. Perhaps all sub (and sub-sub) sections could be made into new main sections.
- Line 31. Suggest rewording to “…to communicate forecast confidence to end users for better decision making.”
- Figure 1 caption. Not sure why SST is specified as the observation type – the rest of the Fig and caption are more general.
- Line 58. Fig. label is incorrect.
- Line 74. “Alternatively, …”. The initial condition uncertainty is an additional source of uncertainty rather than an alternative to those mentioned previously.
- Line 70. Some references for ocean model stochastic model schemes would be useful here, e.g. from Storto et al., 2021, Brankart et al., 2015.
- Line 76. It wasn’t clear to me how EDA schemes fit into these options.
- Line 79. Lateral boundary and surface forcing perturbation schemes might still be needed for some applications where the available atmospheric and global ocean ensembles may not be appropriate in a given operational setting. Perhaps some references could be included on these, e.g. Storto et al., 2023. You could also mention about the uncertainty in other inputs such as the rivers, e.g. Zedler et al., 2023.
- Line 100. You don’t mention CRPS here which is often used to assess ensemble forecasts.
- Line 110. Do you mean global systems here? Some regional operational forecasting systems have been running ensembles for a long time, e.g. TOPAZ (Bertino et al., 2008).
- Line 115. Missing comma between “horizons” and “the”.
- Line 119. Perhaps it could be stated that table 1 is a selection of systems, rather than a comprehensive list (which is difficult to provide).
- Table 1:
- The FOAM ensemble includes internal physics perturbations according to Lea et al. 2022.
- The Bluelink system now runs an operational 1/10° global ensemble using the EnKF (Brassington et al., 2023).
- The ECMWF ocean system is an ensemble system (Zuo et al., 2019) and could be included.
- A couple of surface wave systems are listed, but others also run ensemble wave forecasts, e.g. MeteoFrance, UK Met Office.
- Seasonal forecast systems are mostly ensemble-based systems but only the BoM system is listed. See https://climate.copernicus.eu/seasonal-forecasts.
- A regional system that could also be included is described by Röhrs et al., 2023.
- Line 130. SWOT is now flying so this sentence should be amended.
- Line 133. Ocean forecasting systems have been produced by DA system for a long time so I wasn’t quite sure why this sentence was included.
Possible additional references:
Bertino, L. & K A Lisæter (2008) The TOPAZ monitoring and prediction system for the Atlantic and Arctic Oceans, Journal of Operational Oceanography, 1:2, 15-18, DOI: 10.1080/1755876X.2008.11020098
Brankart, J.-M., Candille, G., Garnier, F., Calone, C., Melet, A., Bouttier, P.-A., Brasseur, P., and Verron, J.: A generic approach to explicit simulation of uncertainty in the NEMO ocean model, Geosci. Model Dev., 8, 1285–1297, https://doi.org/10.5194/gmd-8-1285-2015, 2015.
Brassington, G. B., Sakov, P., Divakaran, P., Aijaz, S., Sweeney-Van Kinderen, J., Huang, X., and Allen, S.: OceanMAPS v4. 0i: a global eddy resolving EnKF ocean forecasting system, in: OCEANS 2023-Limerick, IEEE, 1–8, https://doi.org/10.1109/OCEANSLimerick52467.2023.10244383, 2023.
Röhrs, J., Gusdal, Y., Rikardsen, E. S. U., Durán Moro, M., Brændshøi, J., Kristensen, N. M., Fritzner, S., Wang, K., Sperrevik, A. K., Idžanović, M., Lavergne, T., Debernard, J. B., and Christensen, K. H.: Barents-2.5km v2.0: an operational data-assimilative coupled ocean and sea ice ensemble prediction model for the Barents Sea and Svalbard, Geosci. Model Dev., 16, 5401–5426, https://doi.org/10.5194/gmd-16-5401-2023, 2023.
Storto A, Andriopoulos P. A new stochastic ocean physics package and its application to hybrid-covariance data assimilation. Q J R Meteorol Soc. 2021; 1691–1725. https://doi.org/10.1002/qj.3990
Storto A and Yang C (2023) Stochastic schemes for the perturbation of the atmospheric boundary conditions in ocean general circulation models. Front. Mar. Sci. 10:1155803. doi: 10.3389/fmars.2023.1155803.
Zedler S.E., Polton J.A., King R.R., Wakelin S.L., 2023. The effect of uncertain river forcing on the thermohaline properties of the north west European shelf seas. Ocean Model., 183 (2023), Article 102196
Zuo, H., Balmaseda, M. A., Tietsche, S., Mogensen, K., and Mayer, M.: The ECMWF operational ensemble reanalysis–analysis system for ocean and sea ice: a description of the system and assessment, Ocean Sci., 15, 779–808, https://doi.org/10.5194/os-15-779-2019, 2019.
Citation: https://doi.org/10.5194/sp-2024-10-RC1 -
RC2: 'Comment on sp-2024-10', Anonymous Referee #2, 28 Oct 2024
This is a short article reviewing the today use and benefit of ensemble forecasts and data assimilation in the context of ocean operational real time forecasting.
The abstract clearly presents the focus of the paper.
The introduction highlights the motivation and briefly describe the benefit of ensemble approaches compared to the deterministic ones for ocean operational forecasting.
The different methods are illustrated along with probabilistic assessment methods.
The two last sections of the article present the status of ensemble forecasts in OOFSs, with the list of them and perspectives.
In this section 1.2 on probabilistic assessment, the explanation of the different diagnostics is not so easy to understand for someone who is not familiar with ensemble approaches. Few additional sentences illustrating the metrics would help to understand their meaning for non experts. I would also suggest to add references for readers interested in more details (for example: Section 12.B Statistical Concepts - Probabilistic Data - Forecast User Guide - ECMWF Confluence Wiki).
Some additional information on the limitations, challenges for ensemble approaches and the use of hybrid methods in OOFSs would be useful to describe in this article to give a more complete and objective point of view. For example, does the “inflation” methods still need to be used or regularisation / localisation in case to small ensemble, …?
The numbering of the different sections of the article needs to be revised: 1, 1.1, 1.2, 1.1.1, 2. The section 1.1.1 may be changed to 1.3.
Line by line comments
Figure 2:
Virtual ensemble: for point 2. “Source” may be to vague: replace by model/system as in the text?
“Adhoc selection”: I do not clearly understand what it means.
DA ensemble forecast: quantity -> quantify ?
l.89: can you give few more details/references on the present limitations and method to overcome them to illustrate today limitations and challenges that still need to be addressed or recognized as a limitation for ensemble forecasts and ensemble DA analysis.
l.94: Hybrid methods or multi-scale analysis, with the use of lower resolution ensemble covariance than the HR analysis, should at least be mentioned even if not discussed here.
l.109: OOFS acronym is not defined.
l.112: The increased/high resolution “tendency” is also to answer the user requests for higher resolution analysis and forecasts.
l.117: ever-increasing availability of computational power? Any more recent references than 2010 and 2011 to support it?
l.119: table 1: As it is the view at a given date you may need to write it explicitly and mention that the table does not show the ensemble analysis and forecasting systems are under development in OOF centers.
Table 1: It would be nice to add the spatial resolution. In some OOF centers, the ensemble “system” is complementarity to a deterministic higher resolution system.
l.123: Lack of physical HR data: There is today high-resolution satellite observations that are not fully exploited with fine scale observed as some SST, Ocean Colour products and now SWOT observations also going very close to the shore. The lack of HR data is true for the ocean interior.
l.124: “poorly constrained” and not “poorly unconstrained”?
l.130: Since the SWOT data are now available, the text must be updated.
Citation: https://doi.org/10.5194/sp-2024-10-RC2 -
CC1: 'Comment on sp-2024-10', Johannes Röhrs, 26 Nov 2024
We would like to thank the authors for opening a much-needed discussion on ocean ensemble forecasting. In particular, I find their classification of types of ensemble initialisations very useful. If I may, I would like to add a point here about a type of ensemble that could be coined as identity-retaining ensembles. The idea is the following: While the Monte-Carlo ensemble initialization relies on explicit perturbations, the data assimilation informed ensemble derives ensemble increments for each member and thereby maintains spread. Ensemble forecast cycles are initialized such that each ensemble member is initialized by a forecast of the same member from the previous cycle, removing the need for explicit perturbation. This is the case in EnKF applications, but even without any data assimilation an identity-retaining ensemble is perceivable that describes flow-dependent uncertainty with an error of the day, potentially differentiating predictable regimes from unpredictable circulation patterns.
To provide more context, I would like to suggest that the need for initialization in ocean prediction is discussed in more detail. A comparison with atmospheric prediction that lies the scientific foundation for ensemble forecasting could be useful here. An important difference is that weather prediction benefits from very accurate analysis, and the scope of the EPS is to model the uncertainty due to unstable growing modes which can be adequately described using Monte-Carlo type perturbations. Ocean ensembles, on the other hand, need to deal with large uncertainties in the initial conditions, driving the need for approaches that are different to what is needed in weather prediction, as e.g. identity-retaining ensembles.
With regard to the listed ocean ensembles, I’d like to point to the comments of reviewer #1, adding to the overview list of operational ocean ensemble systems. Perhaps a sorting of this list with respect to forecast variables could be useful, e.g. waves, storm surge, ocean circulation/ hydrodynamics, and sea ice, and a remark on how these systems fit into your classification of ensemble initializations.
Please note that the Topaz 4 model is not operational any more and has been replaced by Topaz5; https://data.marine.copernicus.eu/product/ARCTIC_ANALYSISFORECAST_PHY_002_001/description
Kind Regards
Johannes Röhrs, Victor de AguiarCitation: https://doi.org/10.5194/sp-2024-10-CC1
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 101 | 42 | 153 | 296 | 3 | 3 |
- HTML: 101
- PDF: 42
- XML: 153
- Total: 296
- BibTeX: 3
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1