Skip to content

Run-data-managers now outputs stderr of failed jobs. Run-data-managers to pytest.#110

Merged
mvdbeek merged 33 commits intomasterfrom
issue107
Aug 27, 2018
Merged

Run-data-managers now outputs stderr of failed jobs. Run-data-managers to pytest.#110
mvdbeek merged 33 commits intomasterfrom
issue107

Conversation

@rhpvorderman
Copy link
Copy Markdown
Contributor

Added the code to log a stderr when a data manager job is failing.

I also ported run-data-managers to the new pytest framework. So this extra logging stuff is also tested.

This should fix #107 .

After a lot of trail and error due to various issues, codacy and coverage should work on this branch.

Comment thread tox.ini Outdated
bash -c 'if [ "$TRAVIS" = "true" ] ; then python-codacy-coverage -r coverage.xml; fi'

# This is only the case when the pull request is run from the ephemeris repo. So this is checked.
bash -c 'if [ "$TRAVIS" = "true" ] && [ "$TRAVIS_REPO_SLUG" = "galaxyproject/ephemeris" ] ; then python-codacy-coverage -r coverage.xml; fi'
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this belongs in the deploy stage of the travis.yml. We shouldn't adapt our testing to travis, we should adapt travis to our testing. Same with the sed command.

Copy link
Copy Markdown
Contributor Author

@rhpvorderman rhpvorderman Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 will change this.
Also there is some problems with codacy coverage still... Working on it.
EDIT: By the way:
I can only get the envsitepackagesdir inside tox. So I have to run sed inside tox unfortunately.

Comment thread tests/test_run_data_managers.py Outdated

import pytest
import yaml
from docker_for_galaxy import start_container
Copy link
Copy Markdown
Member

@mvdbeek mvdbeek Aug 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can add # noqa: F401 at the end of this line and drop lines 17 and 18

from docker_for_galaxy import start_container # noqa: F401

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need the two spaces before the comment as usual, github formatting strips this here

Comment thread tox.ini
bash
commands =
pytest --cov={envsitepackagesdir}/ephemeris --cov-report xml {[tox]test_dir}
pytest -v --cov={envsitepackagesdir}/ephemeris --cov-report xml {[tox]test_dir}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't this work ?

pytest -v --cov src --cov-report xml

it does work locally for me, and I think you won't have to sed the xml file

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it work in a tox environment? Because when I tried that in a tox environment it did not work.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems to:

$ tox -e py36-pytest
GLOB sdist-make: /Users/mvandenb/src/ephemeris/setup.py
py36-pytest create: /Users/mvandenb/src/ephemeris/.tox/py36-pytest
py36-pytest installdeps: -rrequirements.txt, pytest, pytest-cov, coverage, codacy-coverage, docker
py36-pytest inst: /Users/mvandenb/src/ephemeris/.tox/dist/ephemeris-0.9.1.dev0.zip
py36-pytest installed: atomicwrites==1.1.5,attrs==18.1.0,bioblend==0.11.0,boto==2.49.0,certifi==2018.8.13,chardet==3.0.4,codacy-coverage==1.3.11,coverage==4.5.1,docker==3.5.0,docker-pycreds==0.3.0,docutils==0.14,ephemeris==0.9.1.dev0,galaxy-lib==18.5.13,idna==2.7,Jinja2==2.10,MarkupSafe==1.0,more-itertools==4.3.0,packaging==17.1,pluggy==0.7.1,py==1.5.4,pyparsing==2.2.0,pytest==3.7.2,pytest-cov==2.5.1,PyYAML==3.13,requests==2.19.1,requests-toolbelt==0.8.0,six==1.11.0,urllib3==1.23,websocket-client==0.51.0
py36-pytest runtests: PYTHONHASHSEED='1055593875'
py36-pytest runtests: commands[0] | pytest -v --cov=ephemeris --cov-report xml
========================================================================== test session starts ===========================================================================
platform darwin -- Python 3.6.5, pytest-3.7.2, py-1.5.4, pluggy-0.7.1 -- /Users/mvandenb/src/ephemeris/.tox/py36-pytest/bin/python3.6
cachedir: .pytest_cache
rootdir: /Users/mvandenb/src/ephemeris, inifile:
plugins: cov-2.5.1
collected 7 items

tests/test_run_data_managers.py::TestRunDataManagers::test_install_data_managers PASSED                                                                            [ 14%]
tests/test_run_data_managers.py::TestRunDataManagers::test_run_data_managers

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take that back:

ERROR: Failed to generate report: No data to report.


ERROR: InvocationError for command '/Users/mvandenb/src/ephemeris/.tox/py36-pytest/bin/pytest -v --cov=ephemeris --cov-report xml' (exited with code 4)

@rhpvorderman
Copy link
Copy Markdown
Contributor Author

@mvdbeek are all issues solved satisfactorily?

Copy link
Copy Markdown
Member

@mvdbeek mvdbeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, looks good!

@mvdbeek mvdbeek merged commit 8c9dadd into master Aug 27, 2018
@bgruening bgruening deleted the issue107 branch December 24, 2020 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Difficult to debug run_data_managers

2 participants