[REVIEW]: AIBECS.jl: The ideal tool for exploring global marine biogeochemical cycles.

[whedon]

Submitting author: @briochemc (Benoit Pasquier)
Repository: https://github.com/JuliaOcean/AIBECS.jl
Version: v0.11.2
Editor: @kthyng
Reviewers: @dankelley, @zhenwu0728
Archive: 10.5281/zenodo.5787531

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/df0ebed76e349224389195ebd8b7c4a4"><img src="https://joss.theoj.org/papers/df0ebed76e349224389195ebd8b7c4a4/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/df0ebed76e349224389195ebd8b7c4a4/status.svg)](https://joss.theoj.org/papers/df0ebed76e349224389195ebd8b7c4a4)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dankelley, @zhenwu0728 please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @kthyng know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @dankelley

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@briochemc) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @zhenwu0728

✨ Important: Please do not use the Convert to issue functionality when working through this checklist, instead, please open any new issues associated with your review in the software repository associated with the submission. ✨

Conflict of interest

• I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

• I confirm that I read and will adhere to the JOSS code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Contribution and authorship: Has the submitting author (@briochemc) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
• Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

• Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
• A statement of need: Does the paper have a section titled 'Statement of Need' that clearly states what problems the software is designed to solve and who the target audience is?
• State of the field: Do the authors describe how this software compares to other commonly-used packages?
• Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
• References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @dankelley it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.05 s (1386.9 files/s, 139568.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Julia                           45           1080            856           3503
Markdown                         7            105              0            291
YAML                             9             12              0            288
TeX                              1             35              0            254
TOML                             2              5              0             77
JSON                             1              0              0             35
-------------------------------------------------------------------------------
SUM:                            65           1237            856           4448
-------------------------------------------------------------------------------


Statistical information for the repository 'ceb9abfad540c058d4e7a0b6' was
gathered on 2021/10/11.
No commited files with the specified extensions were found.

[whedon]

PDF failed to compile for issue #3814 with the following error:

 Can't find any papers to compile :-(

[kthyng]

@whedon generate pdf from branch JOSSpaper

[whedon]

Attempting PDF compilation from custom branch JOSSpaper. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[dankelley]

@kthyng I filled in the boxes, and wrote up some notes as I was reading. I am putting those notes below. I don't quite see how to submit a detailed review otherwise, so I hope this is useful. If you're looking for a Boolean answer, mine is "publish".

I will be able to look at this again tomorrow if desired but after that I won't have much time. I enjoyed reading this and trying out the software, and I thank you for considering me. Dan.

Recommendation

I recommend accepting this submission. It is a welcome addition to the field,
and it would be to the credit of JOSS to publicize it to a wide range of
educators/researchers.

I have made some comments below, in the major/minor format that I normally use
for scientific journals. I am not sure whether this is useful for a JOSS
manuscript, but I didn't see any other way to provide more detailed comments,
after I finished the checklist. As can be seen, I have a few suggestions that
might improve the clarity of the writing, but I leave it up to the authors to
decide whether to follow them (except for the minor comment about a broken
citation).

Were this a conventional scientific paper, I would have appreciated a bit more
about the class of modelling frameworks supported by this project, and about
any related limitations. However, for JOSS, I think what the authors have said
on this matter is wholly sufficient.

As for JOSS, I do think it would be helpful to add a few sentences about why
Julia is chosen for the project, because their addition might give the work
a broader impact. (Basically, Julia opens up doors that are closed to other
systems, and folks who have not used it might not be aware just how important
this is.) However, I leave the choice up to the authors.

I am always happy to see software aimed at more than just a single project.
It's even better when developers take the time to educate others about how
their creations might help others in the work. I think JOSS is designed
specifically for this sort of thing, and I think the authors deserved to be
thanked not just for their authorship of the software, but also for considering
JOSS as a way to expose the work to a wider audience.

Major Comments

The manuscript outlines in reasonable detail the need for the package, although
I would have appreciated more commentary on the focus on steady-state models,
which is a limitation that is certainly acceptable in many applications,
although it is somewhat swept under the rug here. On the technical side, the
paper might have been strengthened by discussing in more detail why Julia is
well-suited to this application.

As for the github site and the code, both are of high quality. I worked
through the 'age' tutorial, and it is straightforward and informative --
a model of how to do this sort of thing, in my opinion.

Minor comments

• Overall, the citations are overly heavy, in my opinion. I prefer writing that
  cites only papers that a reader might reasonably want to consult, to learn
  more about a particular point raised in the text. As an example, L103
  consists entirely of citations, and all readers are told about them is that
  they relate to parameters relating to the carbon cycle. This seems to be
  insufficient guidance, at least for readers like me. I am not objecting to
  this style, because I know it to be common in some fields.
• I suggest removing "ideal" from the title. Let readers decide that.
• Para 1 (L2:18) I think it might be useful to add a few sentences on why Julia
  is a good choice for this project. Otherwise, the paper might only be read by
  folks who are already familiar with the language.
• L27: This suggestion by Box about all models being wrong was essentially
  a discussion point, not a scientific finding or a mathematical conclusion. It
  might help to rewrite that sentence in a form like "Box (1979) has remarked
  that ", to make things a bit clearer. In some existential sense,
  perhaps everything is wrong.
• L28: Readers might take the authors to mean that parameters only arise when
  assumptions are made. This is not entirely true. Another use of parameters
  is to use a model to make predictions for a range of cases. While I do not
  object to the phrasing as it is, I might suggest L28 s/give rise/can give rise/. (Here I am using a sort of sed instruction to indicate a suggested
  edit, although the 28 refers to a line number as printed, not in the file.
  I'll use that notation throughout this review.)
• L31: It's not clear that minimizing model-observation misfit always improves
  the skill of a model, if the latter is (I think, reasonably) taken to
  indicate prediction of a yet-unobserved state. Overfitting can yield poor
  predictions, sometimes. Perhaps L32 s/which also improves/which may also improve/ might help a bit, if the authors agree that there is a slight
  chance of confusion here.
• L73: the Taburet_etal_2019 citation is broken.

[kthyng]

@dankelley Thank you so much for your quick review! It is fine to put comments in the review issue like this. As there become more, we ask that they go to the software repository under review itself as issues that link back to this review issue (no need to change anything in this case if there is nothing to add).

I just want to verify: most of your comments are about the paper, while most of the JOSS review itself tends to be focused on the software. I see you said that you worked through a tutorial so it sounds like it went well. Can I just verify you were able to install the software and use it then? Thanks.

[dankelley]

Oh, yes, installation was trivial (as is usually the case with Julia) and the sample code that I tried all worked. I went through one of the tutorials in detail, and it was not just correct, but also educational -- the authors have gone beyond merely documenting the software as such, and have also explained the gist of the relevant calculations. This is a useful, working, and splendidly documented package.

[dankelley]

Sorry, that last comment was in reply to @kthyng.