GIE Hardening

[natefoo]

After a few attempts to teach workshops (mainly overseas) we've discovered some robustness problems with GIEs on usegalaxy.org. Here's a list of identified issues and potential resolutions, as well as planning for enhancements, in order of priority:

• Re-fix autoloading launch dataset @natefoo (broken by don't uppercase envoverride #4458, fix in [17.09] GIEs: Grandfather automatic uppercasing of some env_override variables #4760)
• Client alive timeouts don't work for slow connections: client side functions (the readiness test and keepalive) use a 500ms timeout which is easily hit. Also these functions can be updated to handle general failures and timeouts differently. The timeout count should be reset when the connection succeeds @natefoo [17.09] Client side fixes for GIEs #4680
• Increase the in-container timeout (this is trivially easynot supported with the containers lib =/) @natefoo [17.09] Fix missing support for command_inject when using containers lib in GIEs #4740
• Monitoring: ensure usegalaxy.org GIEs are working, probably w/ Jenkins/Selenium @jmchilton Selenium Testing of Jupyter Notebooks #4694
• Reconnecting to an existing running container @martenson
  • Reconnection UI
  • Indicator of whether you have a running container
• More feedback on progress while waiting for launch, remove spinner and display permanent error message on failure @natefoo [17.09] Client side fixes for GIEs #4680
• Asynchronous dataset get() on launch
• Missing green "save" button in 17.05.1 Jupyter image
  • Fix and build new image @bgruening fix the magix button bgruening/docker-jupyter-notebook#23
  • Deploy new image on Test/Main @natefoo
• Prevent launch on configurably large datasets
• uWSGI proxy @natefoo
• Keep terminated containers for a configurable period and allow them to be relaunched
  • Most recent container only
  • Older containers
• Multiple containers per user/session
• swarmscale bug fixes @natefoo galaxyproject/usegalaxy-playbook@dd9ab85
• Implement Docker API containers interface @natefoo Implement the API version of the Docker containers interface #5861
• Improve the launch UI

[natefoo]

ping @nekrut

[nekrut]

@erasche @bgruening - we need to solve the mystery of the green button. @davebx volunteered to help so can you guide him?

[hexylena]

We're happy to help, but this isn't a good week for us. Next week will be better for me at least, I think @bgruening will still be at conferences/meetings then though.

[blankenberg]

Would be neat to enable checkpointing and restoring, including filesystem changes.

[martenson]

This should be the green button fix: bgruening/docker-jupyter-notebook#23

[natefoo]

Client/timeout fixes are now on Test/Main.

[natefoo]

The container timeout is now 5 minutes on Test/Main, and both now use the 17.09 image, which includes the fixed magic button.

[martenson]

mockup of adjusted UI, this would allow non-uniform approach based on what environment you select (some allow selecting datasets, some whole histories, some nothing etc.). You could also initialize the dialogue based on what is passed as params and validate it against the already available IE configs

[martenson]

vector: ie_launcher.pdf