the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
A description of Validation Processes and Techniques for Ocean Forecasting
Abstract. Operational forecasting systems architecture requires clear identification of best practices for assessing the quality of ocean products: it plays a key role not only for the qualification of predictions skill, but also for advancing in the scientific understanding of the ocean dynamics from global to coastal scales. The Authors discusses on the role of observing network for performing validation of ocean model outputs, identifying current gaps (i.e., different capacity in assessing physical essential ocean variables versus biogeochemical ones), but also emphasizing the need of new metrics (tailored for end-users comprehension and usages). An analysis on the level of maturity of validation processes from global to regional systems is provided. A rich variety of approaches exist, and the most we move towards the coast the higher is the complexity in calculating such metrics, due to increased resolution but also somehow limited by the lack of coastal observatories worldwide. It is provided as example how the Copernicus Marine Service currently organizes the product quality information from producers (with dedicated scientific documentation, properly planned and designed) to end-users (with publication of targeted estimated accuracy numbers for its whole product catalogue).
- Preprint
(616 KB) - Metadata XML
- BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on sp-2024-33', Anonymous Referee #1, 02 Oct 2024
The paper is a review of the different model validation carried out by different groups. Some model has more detailed information than others.
It is striking to note the amount of details given to the Canadian system. Similar details should be done on all the different systems listed in the present work as it will enrich the readers understanding of each system as well as provide useful information which will help other groups develop their system.In term of usage of satellite data, (Line 173): are satellite data not used to validate model in the coastal region due to lack of remotely sensed data product or also due to higher uncertainty of remotely sensed coastal data?
Most important, I also suggest the authors include conclusion of their findings.
The following are minor correction:
Line 27: ETOOFS is not defined.
Line 60: Line EOV is not defined.
Line 104. I don't understand the sentence (may be missing something in the sentence)
Line 119: I suggest: "It aims to deliver a consistent, reliable, and state-of-the-art information derived from space..."
Line 132: PQ is not defined
Line 200: DA is not defined
Citation: https://doi.org/10.5194/sp-2024-33-RC1 -
RC2: 'Comment on sp-2024-33', Anonymous Referee #2, 08 Nov 2024
A description of Validation Processes and Techniques for Ocean Forecasting, Marcos G. Sotillo, Marie Drevillon, and Fabrice Hernandez
Summary
The manuscript reviews briefly the sources of observations used in operational ocean model systems before identifying some of the key variables used to perform different model validation. The last sections present the general evaluation framework in use for Copernicus Marine Service before presenting succinctly work done internationally outside of Copernicus.
General comments
The manuscript in its current form lacks clarity on multiple aspects.
Firstly, it is not obvious what are the specific objectives of the manuscript. Adding the typical paragraph(s) at the end of the introduction stating the main goals of the manuscript and the sections to achieve these objectives would significantly help orient the reader.
More importantly, the absence of a discussion / conclusions leaves the reader without a clear take-home message.
In its present status, the novelty and contribution of the manuscript is not clear to me. Review and strategic papers are generally useful to the community when they convey a clear message. The section on the Copernicus Marine Service, to my opinion, does not reflects the quality of the work done. CMEMS is often cited as the reference in terms of operational oceanography, and this is not clearly conveyed. A more detailed description of the procedures with examples and references would help illustrate this. The paper is too general and lacks references in its first sections while providing more references to previous work in the last section. I suggest adding to the Canadian experience a more comprehensive review of the validation work done in other groups internationally.
The syntax is overly complex and confusing with too many explanations between parentheses that disrupt the pace and obscure the message. An effort to simplify the writing would also help clarify the points brought up in the manuscript.
The use of specific jargon combined with missing definitions for multiple acronyms make the paper harder to read. I suggest reviewing the manuscript to define all acronyms at their first reference and propose the use of defined acronyms systematically.
Specific comments
L19. Define CMEMS as an acronym early on and use throughout the manuscript.
L27. ETOOFS not defined
L31. Mention clearly the overall objectives and short descriptions of the manuscripts sections and how it helps achieve it.
L37. What is mostly leaving out? Please clarify
L38. Unclear please rephrase the beginning of the sentence.
L47. Define MFC NRT
L52. I would make the sentence between (…) a separate sentence.
L60. EOV not defined
L63. PQ not defined
L63. strategic plan points to what? Defining what the following points are would clarify, limitations, gaps, etc.
L67. Separate sentences at ;
L71. Explicitly mention the limitation for clarity
L82. This section should present more clearly temperature in general before addressing the SST and deeper temperature evaluation. Currently the section is presented as SST, making the Argo discussion look out of place.
L94. Sea level should be a separate point
L97-101. Make these two sentences for clarity.
L109-115. Revise and clarify. Avoid using too many parentheses and verify all opened parentheses are closed.
L110-111 avoid unnecessary repetitions, in this case three OOFS in two sentences. Applies elsewhere in the manuscript.
L117. See previous comment on systematically using CMEMS.
L118. Rephrase complex system of systems framework
L137. Provide examples and/or references here
L144. consistency across what? the different OOFS? Please clarify.
L154. OOFS repetition
L176. Broken link
L188. No opening parentheses
L190. Should be … in house…
L195-200 Any references for this part of the discussion, a project or website documenting those short-lived models? This application is similar to some discussed relocatable coastal ocean system. More details on the methodology would also be interesting as such applications would increase in the coming years.
L210. Reference to Paquin et al. should be corrected in the text & the references. Please use :
Paquin, JP., Roy, F., Smith, G.C., MacDermid, S., Lei, J., Dupont, F., Lu, Y., Taylor, S., St-Onge-Drouin, S., Blanken, H., Dunphy, M., Soontiens, N. A new high-resolution Coastal Ice-Ocean Prediction System for the East Coast of Canada. Ocean Dynamics 74, 799–826 (2024). https://doi.org/10.1007/s10236-024-01634-7
L210. Paquin et al. 2019 (see reference below) presented the prototype of the mentioned port models. Nudds et al. (2020) presents the initial intercomparison projects that took place to compare the NEMO model implementation described in Paquin et al. (2019) with an unstructured model implementation using FVCOM. Please consider adding.
Paquin J-P, Lu Y, Taylor S, Blanken H, Marcotte G, Hu X, Zhai L, Higginson S, Nudds S, Chanut J, Smith GC, Bernier N, Dupont F (2019) High-resolution modelling of a coastal harbour in the presence of strong tides and significant river runoff. Ocean Dyn 70:365–385. https://doi.org/10.1007/s10236-019-01334-7
Nudds S, Lu Y, Higginson S, Haigh SP, Paquin J-P, O’Flaherty-Sproul M, Taylor S, Blanken H, Marcotte G, Smith GC, Bernier NB, MacAulay P, Wu Y, Zhai L, Hu X, Chanut J, Dunphy M, Dupont F, Greenberg D, Davidson FJM, Page F (2020) Evaluation of structured and unstructured models for application in operational ocean forecasting in Nearshore Waters. J Mar Sci Eng 8(7):484. https://doi.org/10.3390/jmse8070484
L217. please replace “…” by “etc.”
L224. Here again references or website would increase impact of argument. Otherwise the discussion remains to vague to really be helpful.
L233. The manuscript needs a conclusion to summarize the discussion and remind the reader of the relevance of the discussion.
References - Review carefully references and standardize format
L284. Inconsistency Mourre et al. 2021 in text, 2018 in reference
L291. Correct Paquin et al. to provided reference
L318. Inconsistency Sotillo 2019 in reference, 2020 in text.
Citation: https://doi.org/10.5194/sp-2024-33-RC2
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
79 | 56 | 105 | 240 | 2 | 1 |
- HTML: 79
- PDF: 56
- XML: 105
- Total: 240
- BibTeX: 2
- EndNote: 1
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1