Skip to content

Add list-dependencies as a sub-command to checkpoint.#88

Open
havardf wants to merge 2 commits intomainfrom
87-list-out-python-package-dependencies-from-a-checkpoint
Open

Add list-dependencies as a sub-command to checkpoint.#88
havardf wants to merge 2 commits intomainfrom
87-list-out-python-package-dependencies-from-a-checkpoint

Conversation

@havardf
Copy link
Copy Markdown
Collaborator

@havardf havardf commented Dec 10, 2025

No description provided.

@havardf havardf linked an issue Dec 10, 2025 that may be closed by this pull request
@havardf havardf requested a review from vegardb December 10, 2025 14:03
click.echo('Anemoi module versions in checkpoint:')
for name, version in provenance["module_versions"].items():
if name.startswith('anemoi'):
click.echo(f' {name}: {version}')
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe write this in the format {name}=={version}, so it can be used as part of some installation process at some point?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, ok, if the output should prepare for automation, I guess I should just remove the validate_environment thing also? Or maybe add it as a separate command?

What do you think of limit this to just anemoi packages? Not sure whats best here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe validate-environment as a command-line switch?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yupp, will do. Maybe the output should be as a pylock file? Then it can be directly used I think. https://packaging.python.org/en/latest/specifications/pylock-toml/

@vegardb
Copy link
Copy Markdown
Member

vegardb commented Dec 10, 2025

Are we sure these are always correct?

@havardf
Copy link
Copy Markdown
Collaborator Author

havardf commented Dec 11, 2025

Are we sure these are always correct?

What do you mean? That the listing can fail because it can't find the correct metadata? Yes, I think this can happen. The path anemoi-metadata/ai-models.json is the default, but I think it can be overrridden. But if it is, I am not sure how to handle that in a safe way. I am thinking its best to just assume default path, and then fail if its not there. I can add some sensible error message for the case though

@vegardb
Copy link
Copy Markdown
Member

vegardb commented Dec 11, 2025

Are we sure these are always correct?

What do you mean? That the listing can fail because it can't find the correct metadata? Yes, I think this can happen. The path anemoi-metadata/ai-models.json is the default, but I think it can be overrridden. But if it is, I am not sure how to handle that in a safe way. I am thinking its best to just assume default path, and then fail if its not there. I can add some sensible error message for the case though

My concern is more that I often see references to non-existing package versions, and things like that. But in that case, the problem may very well be with the checkpoint creators, rather than this PR.

@havardf
Copy link
Copy Markdown
Collaborator Author

havardf commented Dec 11, 2025

Are we sure these are always correct?

What do you mean? That the listing can fail because it can't find the correct metadata? Yes, I think this can happen. The path anemoi-metadata/ai-models.json is the default, but I think it can be overrridden. But if it is, I am not sure how to handle that in a safe way. I am thinking its best to just assume default path, and then fail if its not there. I can add some sensible error message for the case though

My concern is more that I often see references to non-existing package versions, and things like that. But in that case, the problem may very well be with the checkpoint creators, rather than this PR.

We could try to be smart, and try to find the "closest released version" of the versions listed in the checkpoint, and then list these official versions instead?

@vegardb
Copy link
Copy Markdown
Member

vegardb commented Dec 11, 2025

Are we sure these are always correct?

What do you mean? That the listing can fail because it can't find the correct metadata? Yes, I think this can happen. The path anemoi-metadata/ai-models.json is the default, but I think it can be overrridden. But if it is, I am not sure how to handle that in a safe way. I am thinking its best to just assume default path, and then fail if its not there. I can add some sensible error message for the case though

My concern is more that I often see references to non-existing package versions, and things like that. But in that case, the problem may very well be with the checkpoint creators, rather than this PR.

We could try to be smart, and try to find the "closest released version" of the versions listed in the checkpoint, and then list these official versions instead?

...or just accept that the versions may be wrong, and make it a manual task to fix them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

List out python package dependencies from a checkpoint

2 participants