Skip to content

Upload 2.0 support for data library creation.#89

Merged
bgruening merged 5 commits intogalaxyproject:masterfrom
jmchilton:library_upload_2.0
May 8, 2018
Merged

Upload 2.0 support for data library creation.#89
bgruening merged 5 commits intogalaxyproject:masterfrom
jmchilton:library_upload_2.0

Conversation

@jmchilton
Copy link
Copy Markdown
Member

Will convert older style library YAML format from ephemeris to modern version, will attempt to implement a subset of the modern format with legacy APIs as needed.

Some of the defaults that only made sense for training data have been hidden behind a new --training flag.

@jmchilton jmchilton force-pushed the library_upload_2.0 branch from aed3eb3 to 350a9ba Compare May 7, 2018 20:34
@galaxyproject galaxyproject deleted a comment May 7, 2018
@galaxyproject galaxyproject deleted a comment May 7, 2018
@galaxyproject galaxyproject deleted a comment May 7, 2018
@galaxyproject galaxyproject deleted a comment May 7, 2018
@galaxyproject galaxyproject deleted a comment May 7, 2018
@bgruening
Copy link
Copy Markdown
Member

If we really want to support the old format as well we should add a test with the old file to https://github.com/galaxyproject/ephemeris/blob/master/tests/test_workflow_and_data.sh#L20

@bgruening
Copy link
Copy Markdown
Member

  • legacy mode of new and old yaml file puts file into a lib with names like https://raw.githubusercontent.com/yhoogstrate/segmentation-fold/55d0bb28b01e613844ca35cf21fa41379fd72770/scripts/energy-estimation-utility/tests/test-data/workflow-test_cd-box_kturns.xml --> can we use here just the filename?

@jmchilton
Copy link
Copy Markdown
Member Author

If we really want to support the old format as well we should add a test with the old file to https://github.com/galaxyproject/ephemeris/blob/master/tests/test_workflow_and_data.sh#L20

Yeah - I added a file to do that but forgot to update the test. I'll do that.

Comment thread setup.cfg Outdated
[flake8]
max-line-length = 150
max-complexity = 30
ignore = E501
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Long URLs are ignored by standard. So I cannot see a reason why line length should be ignored.

@rhpvorderman
Copy link
Copy Markdown
Contributor

rhpvorderman commented May 8, 2018

Pull request looks very nice. Travis only fails on a few linting errors.

jmchilton added 2 commits May 8, 2018 05:52
Will convert older style library YAML format to modern version, will attempt to implement a subset of the modern format with legacy APIs as needed.
@jmchilton jmchilton force-pushed the library_upload_2.0 branch from aade4be to a37e4c8 Compare May 8, 2018 09:56
@galaxyproject galaxyproject deleted a comment May 8, 2018
We essentially don't want to change the Galaxy API to loose the URL (galaxyproject/galaxy#3695, galaxyproject/galaxy#3996), we tried at some point and had to revert it. The new upload API is smart though and can be used to set a name at upload time and embed the URL we may want to have around later in the info of the dataset. I'd prefer to make this explicit in format if that is okay, so it remains a proper superset of the real API format - I've added an example of doing that here. Unfortunately for the legacy API, this will not work because we aren't exposing the name option in bioblend's library upload options.
@jmchilton
Copy link
Copy Markdown
Member Author

legacy mode of new and old yaml file puts file into a lib with names like https://raw.githubusercontent.com/yhoogstrate/segmentation-fold/55d0bb28b01e613844ca35cf21fa41379fd72770/scripts/energy-estimation-utility/tests/test-data/workflow-test_cd-box_kturns.xml --> can we use here just the filename?

Added an example of how to do that and explained why I'd prefer not to make it automatic in this commit's comment:

dea1ff2

@galaxyproject galaxyproject deleted a comment May 8, 2018
@bgruening bgruening merged commit 1d13a9f into galaxyproject:master May 8, 2018
@bgruening
Copy link
Copy Markdown
Member

Thanks @jmchilton!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants