Skip to content

add generic CargoPythonBundle easyblock#2964

Merged
boegel merged 12 commits intoeasybuilders:developfrom
Micket:20230718191807_new_pr_cargopythonbundle
Dec 22, 2023
Merged

add generic CargoPythonBundle easyblock#2964
boegel merged 12 commits intoeasybuilders:developfrom
Micket:20230718191807_new_pr_cargopythonbundle

Conversation

@Micket
Copy link
Copy Markdown
Contributor

@Micket Micket commented Jul 18, 2023

(created using eb --new-pr)

I haven't tested it yet but i think it might be just this simple. I don't think each individual extension even needs to specify the cargopythonpackage easyblock, this.. should just work i think

Edit: I tested it with timm and it does work, except for one part: the Bundle easyblock hardcodes a check to ensure noone tries to include any sources...

        # list of sources for bundle itself *must* be empty
        if self.cfg['sources']:
            raise EasyBuildError("List of sources for bundle itself must be empty, found %s", self.cfg['sources'])

Ugh.. commenting this out and it just works.

I tried to be clever and changed the init order in my constructor here, but since EB likes to re-initialize easyblocks even when just building stuff (just like the issues with the init step) this still fails on the second pass. Ugh again.

@Micket Micket added the new label Jul 18, 2023
@Micket Micket added this to the next release (4.8.1?) milestone Jul 18, 2023
@Micket
Copy link
Copy Markdown
Contributor Author

Micket commented Jul 18, 2023

so i wanted to control the order, in my

class CargoPythonBundle(PythonBundle, Cargo):1
    def __init__(self, *args, **kwargs):
        """Constructor for CargoPythonBundle easyblock."""
        print("XXXXXXXXXXXXXXXXXX initializing")
        PythonBundle.__init__(self, *args, **kwargs)
        print("################## Now running cargo init next")
        Cargo.__init__(self, *args, **kwargs)

but somehow the order was still wrong. I added a stack printout to know how in hell my Cargo init was called before PythonBundle's init:

  File "/dev/shm/eb-er0u2cce/included-easyblocks-v8j7l7r9/easybuild/easyblocks/generic/cargopythonbundle.py", line 54, in __init__
    PythonBundle.__init__(self, *args, **kwargs)
  File "/apps/Common/software/EasyBuild/4.8.0/lib/python3.6/site-packages/easybuild/easyblocks/generic/pythonbundle.py", line 59, in __init__
    super(PythonBundle, self).__init__(*args, **kwargs)
  File "/dev/shm/eb-er0u2cce/included-easyblocks-v8j7l7r9/easybuild/easyblocks/generic/bundle.py", line 70, in __init__
    super(Bundle, self).__init__(*args, **kwargs)
  File "/dev/shm/eb-er0u2cce/included-easyblocks-v8j7l7r9/easybuild/easyblocks/generic/cargo.py", line 131, in __init__
    traceback.print_stack()

eh, why does super(Bundle, self) == Cargo... this doesn't fit my worldview of multiple inheritance and python

@Micket
Copy link
Copy Markdown
Contributor Author

Micket commented Jul 18, 2023

I was wrong, it's not called multiple times, it's just the MRO and use of super() here which gets very confusing.

I have no choice but to end up with the following CargoPythonBundle.__mro__:

<class 'easybuild.easyblocks.generic.cargopythonbundle.CargoPythonBundle'>
<class 'easybuild.easyblocks.generic.pythonbundle.PythonBundle'>
<class 'easybuild.easyblocks.generic.bundle.Bundle'>
<class 'easybuild.easyblocks.generic.cargo.Cargo'>
<class 'easybuild.framework.extensioneasyblock.ExtensionEasyBlock'>
<class 'easybuild.framework.easyblock.EasyBlock'>
<class 'easybuild.framework.extension.Extension'>
<class 'object'>

and since init methods always just start off calling the super-init method, this thus becomes the strange situation where Cargo must be before Bundle.

I think we really want init methods to just be calling their superclass all the time directly, regardless of diamond inheritance on self.

I wonder if Bundle, and the other base level packages should be calling super() all the way.. or just TheirBaseClass.init... instead. That would force classes with multiple inheritance to do it twice, which would trigger the base class EasyBlock.init to trigger twice. Meh. It all kind of sucks.

Comment thread easybuild/easyblocks/generic/bundle.py Outdated
@boegelbot

This comment was marked as resolved.

@boegel boegel changed the title new easyblock for cargopythonbundle add generic CargoPythonBundle easyblock Aug 2, 2023
Comment thread easybuild/easyblocks/generic/bundle.py Outdated
@easybuilders easybuilders deleted a comment from boegelbot Sep 5, 2023
Comment thread easybuild/easyblocks/generic/cargopythonbundle.py Outdated
@easybuilders easybuilders deleted a comment from boegelbot Oct 17, 2023
@easybuilders easybuilders deleted a comment from boegelbot Dec 18, 2023
@Micket
Copy link
Copy Markdown
Contributor Author

Micket commented Dec 19, 2023

Test report by @Micket

Overview of tested easyconfigs (in order)

  • SUCCESS GTK4-4.13.1-GCC-12.3.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
vera-skylake-build - Linux Rocky Linux 8.8, x86_64, Intel Xeon Processor (Skylake, IBRS, no TSX), Python 3.6.8
See https://gist.github.com/Micket/2f9ec4acf87ea890557bd5c5050a9e0f for a full test report.

@Micket
Copy link
Copy Markdown
Contributor Author

Micket commented Dec 19, 2023

Uploaded test report of something using Bundle to ensure that it still works. New test report for CargoPythonBundle itself is in easybuilders/easybuild-easyconfigs#18332

Comment thread easybuild/easyblocks/generic/cargopythonbundle.py Outdated
@boegel boegel merged commit 56edc96 into easybuilders:develop Dec 22, 2023
@Micket Micket deleted the 20230718191807_new_pr_cargopythonbundle branch December 26, 2023 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants