Added spack module

[atteggiani]

Added a first instance of a spack module for deployment on Gadi.

This PR is mainly to keep track of the modulefile for spack and allow versioned modification. It will be kept as a draft until we finalise the location within the repo where to place it and are happy to consider it merged into the infrastructure.

[CodeGat]

Just some comments

[atteggiani]

A comment on this file:

This is an example.
We can decide to set the default version like this, or have another way for setting it.

If we choose this method, this file in the repo should probably be dynamic, with the version set at deployment time.

Other solutions (which would not require a dynamic file) could be:

• set the default version to the latest one deployed (if we're sure we always want the latest-deployed version to be the default one)
• default version automatically set based on version name (in this case we need a stable version naming scheme, and I think we do)

[CodeGat]

Just noting that we will need to incorporate #375 into the final design of the modulefile

[atteggiani]

• Change python version to the one currently (3.6)
• Set base_dir dynamically based on the parent directory of the modulefile
• Separate bash script

[CodeGat]

After a meeting with @harshula and @atteggiani :

• Rather than separating modules via Pre/Release based on the module load (for example, module load spack/1.1/prerelease/module load spack/1.1/release), we can encode that in the module use, like so:

  module use /g/data/vk83/modules
  module load spack/1.1
  # Loads the release instance of spack at /g/data/vk83/apps/spack/1.1 by virtue of the module location
  module use /g/data/vk83/prerelease/modules
  module load spack/1.1
  # Loads the prerelease instance of spack at /g/data/vk83/prerelease/apps/spack/1.1 by virtue of the module location

  • Furthermore, we should direct regular users to the module use /g/data/vk83/modules version. The prerelease version might be useful for getting alpha testers of new versions of spack-config or something.

• In .common, if we can get the path to the modulefile itself, we can do some path magics to construct the base_dir - it'd probably look something like $path_to_module_location/../../apps/spack/$module_version

• If we could export SPACK_USER_CACHE_PATH in the module, that would be great. However, there are two cases to take into account:

  • The service user (has SPACK_ADMIN_CONFIG exported), which needs SPACK_USER_CACHE_PATH=$base_dir/spack-admin-cache
  • The regular user (does NOT have SPACK_ADMIN_CONFIG exported), which needs SPACK_USER_CACHE_PATH=/g/data/$PROJECT/$USER/spack/$module_version/spack-user-cache

Did I miss anything? 😆

[atteggiani]

Updates

The discussed updates have been implemented:

• Spack python changed to be using the default python3 executable on Gadi (/bin/python3)
  The modulefile has been renamed to modulefile
  6b893ee

• The base directory and spack_root are now set dynamically based on the modulefile parent directory
  9d9aabe

• The bash script has been moved into separate file (bash_file.sh) for better clarity and maintainability.
  a2be738

• The spack user cache environment variable (SPACK_USER_CACHE_PATH) is exported based on the existence of SPACK_ADMIN_CONFIG env variable.
  d9c0b08

Info for the deployment

For simplicity, the modulefile and bash script in this repo have been named modulefile and bash_script.sh, respectively.
At deployment time (within the build-cd deployment CD) the following should happen to match the current implementation:

• The deployed modulefile name should match the spack version deployed (e.g., if we're deploying spack version 1.1, the modulefile should be named 1.1).
  An example modulefile path could be /g/data/vk83/modules/spack/1.1.
• The bash file should be deployed in the same directory as the modulefile, named as .<version>.sh (e.g., if we're deploying spack version 1.1, the bash script should be named .1.1.sh).
  An example bash script path could be /g/data/vk83/modules/spack/.1.1.sh.

Testing

I copied the files (properly named) in /g/data/vk83/prerelease/modules/spack for testing, so this updated spack module can be used with:

module use /g/data/vk83/prerelease/modules/spack
module load spack/1.1

[CodeGat]

Looks good! I might put an update to the create-deployment-spack.yml workflow that also deploys this - thanks!!

[atteggiani]

@CodeGat The workflow changes for the deployment look good!
Just had a minor curiosity.

[atteggiani]

Updates n.2

The discussed updates have been implemented:

• Spack python changed to be using /bin/python3

• Added information on where the user spack cache is stored when the module gets loaded

• Automatic detection of aliases (similarly to what is done for env vars and functions)

• Restructure of the bash_script to be clearer and added logic to be able to reinstate vars, functions or aliases that get removed after sourcing the setup-env.sh (might never happen, but like this it's complete and we might not need to worry about updates of the setup-env.sh breaking the logic).

The updated files have been copied to /g/data/vk83/prerelease/modules/spack