Build and deployment documentation (#280)

* Build and deployment first draft

* Minor updates

Added information on required repositories and building methods for ACCESS-ESM1.6.

* (#279) - Clarify the prerequisites and the recommended build workflow

* (#279) - Revisiting the page completely.

* (#279) - updates after reading the rendered version.

* (#279) - Add information on existing deployments

* (#279) - Reorganise info between Building and Deployment pages.

* (#279) - Integrated Aidan's feedback

* (#279) - add deployment page to navigation and move navigation entry higher up in mkdocs.yml

Got tired of having to scroll down or search for the nav entry when it's the bit we modify the most.

* (#279) - Re-read

* (#279) - Edits from review

* (#279) - More edits.

* (#279) - Edits from review

* (#279) - links check

* (#279) - Link check

* (#279) - use lowercase for all documentation files

Consistency makes linking between pages easier.

* (#279) - Update links to lowercase filenames

Author: ccarouge

documentation/docs/pages/infrastructure/Git-practices.md

@@ -0,0 +1,52 @@
+# Build infrastructure
+
+Most users of ACCESS-ESM1.6 will not need to build the model. Released versions of the model are deployed by ACCESS-NRI to Gadi and available to users. Check the [Deployment page](/infrastructure/deployment/) to learn about available deployments. If you require your own build of the model, typically because you need to modify one or several model components, you will find the required information on this Building page.  
+
+!!! info
+
+    To understand the difference between a release of the model and its deployment, refer to [this website](https://www.geeksforgeeks.org/software-engineering/software-deployment-in-software-development/#software-deployment-vs-software-release).
+
+ACCESS-NRI is using [Spack](https://spack.io/), a build from source package manager designed for use with high performance computing. 
+
+## Software structure
+
+ACCESS-ESM1.6 being a Earth System model, it is constituted of several model components, each residing in its own repository on GitHub. In addition, some model components need other utilities at compilation time that are stored in separate repositories. This results in a complex software structure involving a number of repositories, including:
+
+- deployment repository: 
+    - [ACCESS-NRI/ACCESS-ESM1.6](https://github.com/ACCESS-NRI/ACCESS-ESM1.6)
+- model components:
+    - [ACCESS-NRI/cice5](https://github.com/ACCESS-NRI/cice5)
+    - [ACCESS-NRI/GFDL-generic-tracers](https://github.com/ACCESS-NRI/GFDL-generic-tracers)
+    - [ACCESS-NRI/MOM5](https://github.com/ACCESS-NRI/MOM5)
+    - [ACCESS-NRI/UM7](https://github.com/ACCESS-NRI/UM7)
+    - [CABLE-LSM/CABLE](https://github.com/CABLE-LSM/CABLE)
+- other software:
+    - [ACCESS-NRI/FMS](https://github.com/ACCESS-NRI/FMS)
+    - [ACCESS-NRI/mocsy](https://github.com/ACCESS-NRI/mocsy)
+    - [ACCESS-NRI/GCOM4](https://github.com/ACCESS-NRI/GCOM4)
+
+!!! note
+
+    The _UM7_ and _GCOM4_ repositories are private because these software are not open-source. They are covered by the UKMO's Momentum Partnership licence. To gain access to these repositories, please request an invitation via [this ACCESS-Hive forum topic](https://forum.access-hive.org.au/t/request-access-to-access-esm1-6-component-repositories/5709).
+
+!!! note
+
+    ACCESS-ESM1.6 has other dependencies such as Fortran compilers, netCDF and more. These dependencies are considered model infrastructure and are rarely changed by end-users. The build infrastructure we provide will find these dependencies for you.  
+
+## Building ACCESS-ESM1.6 using the deployment infrastructure
+
+ACCESS-NRI has developed a build and deployment infrastructure for ACCESS-ESM1.6 on GitHub. With minimal setup, it allows all users to build the model and make their build available on _Gadi_ under the `vk83` project. It is the recommended method because it does not require users to have access to all the repositories, it handles the build automatically and it provides traceability and shareability of the deployment.
+
+This method is explained on the ACCESS-Hive page for [creating pre-releases and releases](https://docs.access-hive.org.au/models/build_a_model/create_a_prerelease/) of ACCESS models. For ACCESS-ESM1.6, you will need _write permissions_ on the [ESM1.6 deployment repository](https://github.com/ACCESS-NRI/ACCESS-ESM1.6) and on any model component you need to modify. Request these permissions via this [ACCESS-Hive forum topic](https://forum.access-hive.org.au/t/request-access-to-access-esm1-6-component-repositories/5709).
+
+!!! warning
+
+    This method to build the model only works if your modifications are committed in branches of the model repositories on GitHub listed in the [software structure](#software-structure). This method can not be used if your modifications sit in a fork of one of the repositories or a local clone.
+
+## Local build of ACCESS-ESM1.6
+
+The Spack infrastructure allows you to create your own local build on _Gadi_ for ACCESS-ESM1.6. Once setup, this might be faster than using the deployment infrastructure since it can build from your local copy of the source code. This is especially useful when you want a lot of small incremental builds during development.
+
+However, this method has several drawbacks. It requires users to install the Spack software which can make it harder for ACCESS-NRI to provide support. It requires users to have at least _read permissions_ on all the repositories required by ACCESS-ESM1.6 listed in the [Software structure](#software-structure) section. It builds the model in the user space on _Gadi_ thus potentially exposing licensed source code to unlicensed users, limiting the shareability of the build and consuming a potentially non-insignificant amount of inodes. Finally, contrary to the deployment infrastructure, it does not provide an easy way to access several test builds from the same model component branch at the same time.
+
+This method is explained on the ACCESS-Hive page for [modifying an ACCESS model's source code](https://docs.access-hive.org.au/models/build_a_model/build_source_code/). 

…ocs/pages/infrastructure/Architecture.md …ocs/pages/infrastructure/architecture.mddocumentation/docs/pages/infrastructure/Architecture.md renamed to documentation/docs/pages/infrastructure/architecture.md documentation/docs/pages/infrastructure/Architecture.md renamed to documentation/docs/pages/infrastructure/architecture.md

@@ -0,0 +1,27 @@
+# Deployment
+
+All released versions of ACCESS-ESM1.6 configurations come with a pre-deployed build on _Gadi_. Deployed builds are stored under the project `vk83`. You can see all deployed builds via:
+
+```bash
+module use /g/data/vk83/modules
+module avail access-esm1p6
+```
+
+There are builds deployed for development only (identified with a `dev_` prefix to the date tag) and released deployments (identified with only a date tag). Up-to-date model configurations use the latest full released build. 
+
+These deployments are optimised for the Sapphire-Rapid hardware from Intel (i.e., the *normalsr* queue), and will run slower on Cascade-Lake (i.e., the *normal* queue). In general, these deployments won't run on other NCI hardware.  
+
+
+## Release information
+
+The deployment process creates a GitHub release with the same tag as the deployed module. All releases are available under the [Releases page](https://github.com/ACCESS-NRI/ACCESS-ESM1.6/releases). Each release has a changelog and meta-data with detailed information about the build and deployment, including:
+
+- paths on Gadi to all executables built in the deployment process (_spack.location_ file)
+- a _spack.lock_ file, which is a complete build provenance document, listing all the components that were built and their dependencies, versions, compiler version, build flags and build architecture
+- the environment file _spack.yaml_ used for the deployment
+
+## Deployment process
+
+The deployment is managed via GitHub Actions that are triggered when a new version of the spack.yaml file is committed to the `main` branch or a dedicated `backport/VERSION` branch of the [ACCESS-NRI/ACCESS-ESM1.6](https://github.com/ACCESS-NRI/ACCESS-ESM1.6) repository. Refer to the [Building page](/infrastructure/building) for information on creating new model build versions.
+
+No manual deployments will be accepted.

documentation/docs/pages/infrastructure/building.md

@@ -10,7 +10,39 @@ site_url: !ENV READTHEDOCS_CANONICAL_URL
 repo_url: https://github.com/ACCESS-NRI/access-esm1.6-configs
 repo_name: ACCESS-NRI/access-esm1.6-configs
 edit_uri: blob/main/documentation/docs/ #hopefully will fix the edit button from 404'ing, https://github.com/mkdocs/mkdocs/issues/2416
- 
+
+# Navigation
+nav:
+  - Home:                            pages/index.md
+  - Contributing:          
+        - Contributing overview:     pages/contributing/index.md   
+  - Model Components:
+        - MOM - Ocean:      pages/model_components/mom.md
+        - WOMBAT - Ocean Biogeochem.:   pages/model_components/wombat.md
+        - CICE - Sea Ice:     pages/model_components/cice.md
+        - UM - Atmosphere:       pages/model_components/um.md
+        - CABLE - Land:    pages/model_components/cable.md
+        - CASA - Land Biogeochem.:     pages/model_components/casa.md
+  - Inputs:
+        - Ocean:    pages/inputs/ocean.md
+        - Sea ice:    pages/inputs/seaice.md
+        - Atmosphere:    pages/inputs/atmosphere.md
+        - Land:    pages/inputs/land.md
+        - Restarts: pages/inputs/restarts.md
+
+  - Configurations and Experiments:
+        - Configuration:    pages/configs_experiments/overviewconfiguration.md
+        - Experiments:    pages/configs_experiments/experiments.md
+
+  - Diagnostics:
+        - Sea Ice: pages/diagnostics/sea_ice.md #do not edit directly, change 
+          # sea_ice_source.ipynb then export as HTML and rename that file to sea_ice.md
+  - Infrastructure:    
+        - Architecture:          pages/infrastructure/architecture.md
+        - Deployment:            pages/infrastructure/deployment.md
+        - Building:              pages/infrastructure/building.md
+        - Git practices:         pages/infrastructure/git-practices.md
+
 # Theme
 theme:
   name: material
@@ -92,37 +124,6 @@ markdown_extensions:
       title: On this page
       permalink: true
 
-# Navigation
-nav:
-  - Home:                            pages/index.md
-  - Contributing:          
-        - Contributing overview:     pages/contributing/index.md   
-  - Model Components:
-        - MOM - Ocean:      pages/model_components/mom.md
-        - WOMBAT - Ocean Biogeochem.:   pages/model_components/wombat.md
-        - CICE - Sea Ice:     pages/model_components/cice.md
-        - UM - Atmosphere:       pages/model_components/um.md
-        - CABLE - Land:    pages/model_components/cable.md
-        - CASA - Land Biogeochem.:     pages/model_components/casa.md
-  - Inputs:
-        - Ocean:    pages/inputs/ocean.md
-        - Sea Ice: pages/inputs/sea_ice.md 
-        - Atmosphere:    pages/inputs/atmosphere.md
-        - Land:    pages/inputs/land.md
-        - Restarts: pages/inputs/restarts.md
-
-  - Configurations and Experiments:
-        - Configuration:    pages/configs_experiments/overviewconfiguration.md
-        - Experiments:    pages/configs_experiments/experiments.md
-
-  - Diagnostics:        
-        - Sea Ice: pages/diagnostics/sea_ice.md #do not edit directly, change 
-          # sea_ice_source.ipynb then export as HTML and rename that file to sea_ice.md
-  - Infrastructure:    
-        - Architecture:          pages/infrastructure/Architecture.md
-        - Building:              pages/infrastructure/Building.md
-        - Git practices:         pages/infrastructure/Git-practices.md
-
 # Footer
 extra:
   generator: false