Skip to content

Toolshed tool installation error due to socket issue (release 20.01 branch) #9756

@pcm32

Description

@pcm32

In the middle of toolshed installations with Ephemeris I get in a particular HPC installation the following error:

Server side:

galaxy.tool_util.deps DEBUG 2020-05-11 05:27:10,561 [p:32473,w:4,m:0] [uWSGIWorker4Core0] Using dependency monocle3-cli version 0.0.7 of type conda
10.25.100.188 - - [11/May/2020:05:18:38 +0100] "POST /api/tool_shed_repositories/new/install_repository_revision?key=ff1fadcdba102650fb96c7f486b45d6b HTTP/1.1" 200 - "-" "python-requests/2.23.0"
Mon May 11 05:27:10 2020 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /api/tool_shed_repositories/new/install_repository_revision?key=ff1fadcdba102650fb96c7f486b45d6b (ip 10.25.100.188) !!!
Mon May 11 05:27:10 2020 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /api/tool_shed_repositories/new/install_repository_revision?key=ff1fadcdba102650fb96c7f486b45d6b (10.25.100.188)
OSError: write error
[pid: 32473|app: 0|req: 291/1167] 10.25.100.188 () {42 vars in 751 bytes} [Mon May 11 05:18:38 2020] POST /api/tool_shed_repositories/new/install_repository_revision?key=ff1fadcdba102650fb96c7f486b45d6b => generated 0 bytes in 511584 msecs (HTTP/1.1 200) 3 headers in 0 bytes (0 switches on core 0)

at the client side you see:

05:23:39 Traceback (most recent call last):
05:23:39   File "/var/lib/jenkins/miniconda3/envs/galaxy-ephemeris/lib/python3.6/site-packages/ephemeris/shed_tools.py", line 609, in install_repositories
05:23:39     install_repository_revision(repository, self.tsc)
05:23:39   File "/var/lib/jenkins/miniconda3/envs/galaxy-ephemeris/lib/python3.6/site-packages/ephemeris/shed_tools.py", line 428, in install_repository_revision
05:23:39     response = tool_shed_client.install_repository_revision(**repository)
05:23:39   File "/var/lib/jenkins/miniconda3/envs/galaxy-ephemeris/lib/python3.6/site-packages/bioblend/galaxy/toolshed/__init__.py", line 146, in install_repository_revision
05:23:39     return self._post(url=url, payload=payload)
05:23:39   File "/var/lib/jenkins/miniconda3/envs/galaxy-ephemeris/lib/python3.6/site-packages/bioblend/galaxy/client.py", line 152, in _post
05:23:39     files_attached=files_attached)
05:23:39   File "/var/lib/jenkins/miniconda3/envs/galaxy-ephemeris/lib/python3.6/site-packages/bioblend/galaxyclient.py", line 159, in make_post_request
05:23:39     body=r.text, status_code=r.status_code)
05:23:39 bioblend.ConnectionError: Unexpected HTTP status code: 504: <!DOCTYPE html>

Currently my socket setup is as follows:

uwsgi:
    buffer-size: 16384
    die-on-term: true
    enable-threads: true
    hook-master-start: unix_signal:2 gracefully_kill_them_all
    hook-master-start: unix_signal:15 gracefully_kill_them_all
    manage-script-name: false
    master: true
    module: galaxy.webapps.galaxy.buildapp:uwsgi_app()
    offload-threads: 1
    processes: 4
    py-call-osafterfork: true
    pythonpath: lib
    socket: /tmp/galaxy_gxa_uwsgi.sock
    static-map: /static/style=static/style/blue
    static-map: /static=static
    static-map: /favicon.ico=static/favicon.ico
    threads: 1
    thunder-lock: true
    virtualenv: .venv

the call for the web handler is:

/*****/server/venv_20.01/bin/uwsgi --virtualenv /*****/server/venv_20.01 --yaml /*****/server/config_20.01/galaxy.yml --logdate --thunder-lock --master --processes 4 --threads 1 --logto /*****/server/galaxy-20.01/galaxy_web_%(host_node_name)s.log --socket /tmp/galaxy_gxa_uwsgi.sock --pythonpath lib --pidfile2 /*****/server/galaxy-20.01/galaxy_web_%(host_node_name)s.pid --stats 127.0.0.1:9191 -b 16384

any suggestions on how I should change the comm between UWSGI and Galaxy? Any other config bit that I should show for helping in diagnosis? nginx doesn't show any errors. Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions