Skip to content

Add -j flag to linux-build.bash so that builds are done in parallel#272

Merged
uwagura merged 2 commits into
NOAA-GFDL:mainfrom
uwagura:feature/parallel-builds
May 11, 2026
Merged

Add -j flag to linux-build.bash so that builds are done in parallel#272
uwagura merged 2 commits into
NOAA-GFDL:mainfrom
uwagura:feature/parallel-builds

Conversation

@uwagura

@uwagura uwagura commented May 11, 2026

Copy link
Copy Markdown
Collaborator

This PR adds the -j flag to $makeflags, so that all builds are now done in parallel. In my tests on gaea, this cut the model build time by more than half:
Without the -j flag:

135759991                                   MOM6SIS2_gaea_build   gfdl_med        128  COMPLETED   00:17:20 

With the -j flag:

135761241                          MOM6SIS2_gaea_parallel_build   gfdl_med        128  COMPLETED   00:06:58

@uwagura uwagura requested a review from theresa-cordero May 11, 2026 14:28
@uwagura

uwagura commented May 11, 2026

Copy link
Copy Markdown
Collaborator Author

The effect is more subtle on the CI than on gaea. For this build with the -j flag, the build times were

build_mom6sis2_debug: 3 min 16s
build_mom6solo_repro: 7 min 30s

Whereas for the last build:

build_mom6sis2_debug: 3 min 54s
build_mom6solo_repro: 9 min 30s

will look into this before merging @theresa-cordero

@uwagura

uwagura commented May 11, 2026

Copy link
Copy Markdown
Collaborator Author

Latest run with parallel build of libyaml is even faster:

build_mom6sis2_debug: 2 min 39s
build_mom6solo_repro: 6 min 34s

I think the reason why I'm not seeing the > 50% speed up I saw with the full repro build on gaea is 1.) that second build may not have included libyaml ( which does not seem to be rebuilt if the .libs directory is present ) and 2.) building cobalt serially in repro mode is a bigger bottleneck during that build process than serialization is to the mom6sis2_debug build or the mom6solo_repro build.

Likewise, the time differences in the mom6sis2 and mom6solo builds can likely be explained by the fact that one is built in debug mode, and one is built in repro.

@theresa-cordero

Copy link
Copy Markdown
Contributor

Is it safe to do a squash merge here or should we keep two commits?

@uwagura

uwagura commented May 11, 2026

Copy link
Copy Markdown
Collaborator Author

I think a squash merge should be fine

@uwagura uwagura merged commit 1542729 into NOAA-GFDL:main May 11, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants