Planemo fetches old database

[hexylena]

relevant line

Instead of fetching a specific revision that will slowly age out, how about just symlinking it (or providing a 'latest')?

[dannon]

I actually tried to implement this in the galaxy_downloads repo but botched it, let me review what went wrong.

[dannon]

See https://github.com/dannon/galaxy-downloads/blob/master/latest.sqlite

[hexylena]

Ah, so not as simple it could be. Someone just needs to publish an ftp directory of them... (or for the code to understand github's view of a symlink)

[dannon]

Minor progress figuring this out (maybe -- you can get the contents anyway):
curl --header "Content-Type: application/vnd.github.VERSION.raw" https://api.github.com/repos/jmchilton/galaxy-downloads/contents/latest.sqlite

[hexylena]

Huh. I would've figured that a request to https://raw.githubusercontent.com/dannon/galaxy-downloads/master/latest.sqlite with that header would've responded correctly, but it looks like it's JUST the API that responds that way. That's bloody annoying.

Okay, my vote is for changing

planemo/planemo/galaxy_config.py

Line 71 in 4250874

DATABASE_TEMPLATE_URL = DOWNLOADS_URL + "db_gx_rev_0120.sqlite"

from

DATABASE_TEMPLATE_URL = DOWNLOADS_URL + "db_gx_rev_0120.sqlite"

to

import urllib
LATEST_URL = "https://raw.githubusercontent.com/jmchilton/galaxy-downloads/master/latest.sqlite"
DATABASE_TEMPLATE_URL = DOWNLOADS_URL + urllib.urlopen(LATEST_URL).read()

It's too easy...even if it isn't as clean as some solution which would redirect properly.

[dannon]

Annoying that github won't just serve the file, but I think this solution is fine. I can mirror the change in galaxy's test framework, since it has the same issue.

[jmchilton]

I may roll this back or tweak it somehow - I don't like that planemo is now incompatible with the latest stable. I feel like too old is better than too new.

Wonder if it is possible to just force Galaxy to use the newer database - in many cases it would probably just work.

[hexylena]

I noticed that too in my testing. I agree, that's unpleasant behaviour.

maybe it would make sense to:

• offer a "latest" db
• offer an "old" db
• default to one but provide a flag for the other

Or something like that.

[jmchilton]

I think I want to move this functionality into Galaxy's run_tests.sh - that way API and casper tests could potentially leverage it and run_tests.sh can know what version it is on.

[hexylena]

Sure, that'd make sense too.