Continuous deployment Python script by mira-miracoli · Pull Request #80 · usegalaxy-eu/vgcn

mira-miracoli · 2023-12-14T16:05:57Z

Script that

builds the image with packer
converts the image with qemu-img convert
(optional) uploads the image to OpenStack
(optional) copies the image to sn06 static/vgcn directory

domgz

This is an extremely thorough review, but I wanted to invest the time given that this script is critical. We have seen the effect of neglected image builds numerous times (e.g. BRAKER3, probably cgroups, and many other examples from the past).

To illustrate the importance of the script, I want to draw an analogy between it and the technical revision of ambulances, if the screws on the wheels are loose, they fall away and you can't even send the doctor when somebody is dying (or you send him/her walking and other people die at the hospital meanwhile). That's exactly what is happening to us right now with numerous bugs.

Please look at the comments from a distance, do not get lost in them. Despite the fact that you see a comment every few lines, I am commenting on minor details or cases where the spec does not match what we agreed. Overall this is script is solving the problem.

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

mira-miracoli · 2023-12-19T19:05:28Z

Thank you for the thorough review, I think I learned a lot :)

Compute commit time in `assemble_timestamp` instead of build time as agreed in usegalaxy-eu#78.

Prevents "SyntaxError: f-string expression part cannot include a backslash". ``` In [1]: f"[{','.join('\"' + x + '\"' for x in self.provisioning)}]" Cell In[1], line 1 f"[{','.join('\"' + x + '\"' for x in self.provisioning)}]" ^ SyntaxError: f-string expression part cannot include a backslash ```

domgz · 2023-12-20T15:37:54Z

@mira-miracoli Consider this approved, I just want to run it before hitting approve.

A good reason to run it may be https://github.com/usegalaxy-eu/issues/issues/505 if it is caused by something that needs to be fixed in the images. I'll look into the issue and if the images need to be fixed, I'll build and upload them using this, then hit approve.

mira-miracoli · 2023-12-21T15:28:23Z

was it intentional that 8b363fa leaves the generic tag in the name?
I'm fine with that, I just wanted to note that because, I think in #78 the idea was to only mention !generic if it is missing
But I am fine with both ways

bgruening · 2024-01-02T18:27:49Z

Thanks, everyone! And a happy new year!

mira-miracoli added 8 commits November 28, 2023 15:53

initial commit

3cf0da1

dry run working

57f0871

update gitignore

7f612a5

dependency handling

fb4fd86

use conda command directly, remove wrapper

6918ad8

add publish method

4686df8

auth error handling

ef590cd

use subprocess

9199d44

domgz linked an issue Dec 15, 2023 that may be closed by this pull request

Release management for VGCN images #78

Closed

6 tasks

mira-miracoli added 6 commits December 15, 2023 13:35

mostly documentation

a64c3b6

use sys.exit instead of exception

6d344b8

add conda env file

8cfaaa9

add openstack to env file

04f2a40

allow build.py to set ansible extra vars

39c1d1b

no fail if vault password is missing

78843e2

mira-miracoli marked this pull request as ready for review December 15, 2023 14:59

mira-miracoli added 2 commits December 15, 2023 16:00

nl

ad8085e

better binary path management

dd6b84f

bgruening reviewed Dec 16, 2023

View reviewed changes

Comment thread build.py Outdated

Comment thread build.py Outdated

Comment thread build.py Outdated

Comment thread build.py Outdated

Comment thread build.py Outdated

mira-miracoli added 3 commits December 19, 2023 11:42

sort imports

b437945

use black, use fstrings, remove unnecessary function

4fc5aa6

comment

d1d6b46

domgz reviewed Dec 19, 2023

View reviewed changes

mira-miracoli and others added 7 commits December 19, 2023 17:45

remove unnecessary linebreaks

d953c66

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

Update .gitignore

b898deb

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

make parser as function

2d3a60c

add license to spinner class

d6107de

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

remove unnecessary linebreaks

3c5ce69

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

remove unnecessary linebreaks

2227162

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

Update doc

6b97b23

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

mira-miracoli and others added 16 commits December 19, 2023 18:24

no append

1616455

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

remove choices

07cd13a

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

wrap argparse in function

c071715

pathlib.path constants

1cedd6d

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

remove anaconda from docs

39be50e

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

pathlib objects look better

45468b4

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

another simple path join

0392ffc

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

and one more

505ef7c

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

that should be a variable

36f041c

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

fix order

9c8c903

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

more paths to join

60ee6d6

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

use pathlib join

3b6a4b0

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

copy list

8b363fa

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

add architecture

1f153c5

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

reorder assemble name

e1c6d93

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

use pathlibs exist method

ef2e646

Co-authored-by: José Manuel Domínguez <43052541+kysrpex@users.noreply.github.com>

domgz changed the title ~~Python script~~ Continuous deployment Python script Dec 20, 2023

domgz and others added 2 commits December 20, 2023 16:29

Compute commit time in assemble_timestamp

56b9e2f

Compute commit time in `assemble_timestamp` instead of build time as agreed in usegalaxy-eu#78.

domgz mentioned this pull request Dec 20, 2023

Release management for VGCN images #78

Closed

6 tasks

mira-miracoli added 3 commits December 21, 2023 15:43

special var is called group_names

522cd03

not hostname but internal

c54ea61

and the quotes

9975044

mira-miracoli mentioned this pull request Dec 21, 2023

Miner block #81

Draft

bgruening merged commit 6f78a5f into usegalaxy-eu:dev Jan 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Continuous deployment Python script#80

Continuous deployment Python script#80
bgruening merged 47 commits intousegalaxy-eu:devfrom
mira-miracoli:python-script

mira-miracoli commented Dec 14, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

domgz left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mira-miracoli commented Dec 19, 2023

Uh oh!

domgz commented Dec 20, 2023 •

edited

Loading

Uh oh!

mira-miracoli commented Dec 21, 2023

Uh oh!

bgruening commented Jan 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

mira-miracoli commented Dec 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

domgz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mira-miracoli commented Dec 19, 2023

Uh oh!

domgz commented Dec 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mira-miracoli commented Dec 21, 2023

Uh oh!

bgruening commented Jan 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mira-miracoli commented Dec 14, 2023 •

edited

Loading

domgz left a comment •

edited

Loading

domgz commented Dec 20, 2023 •

edited

Loading