refactor(test): Run tests with pytest#1629
Closed
joelspadin wants to merge 1 commit intozmkfirmware:mainfrom
Closed
refactor(test): Run tests with pytest#1629joelspadin wants to merge 1 commit intozmkfirmware:mainfrom
joelspadin wants to merge 1 commit intozmkfirmware:mainfrom
Conversation
Contributor
|
Tried to install the deps within our dev containers, and it had issues, and couldn't run the tests after: |
Contributor
|
Similar error on my local host: |
Collaborator
Author
|
I don't think this conflict is related to my changes. Remarshal requires PyYAML ^5.3 while cmsis-pack-manager requires PyYAML >=6.0,<7. To resolve that, we need to get rid of the dependency on remarshal, which appears to be unmaintained since October 2020. |
Collaborator
Author
|
|
joelspadin
force-pushed
the
pytest
branch
5 times, most recently
from
55473fd to
53a8d96
Compare
petejohanson
requested changes
Jan 24, 2023
Contributor
petejohanson
left a comment
petejohanson
left a comment
There was a problem hiding this comment.
We should update https://zmk.dev/docs/development/tests with info on installing the now required Python deps in order to run the tests.
Comment on lines
-65
to
+66
| run: west test tests/${{ matrix.test }} | ||
| run: west test ${{ matrix.test }} |
Contributor
There was a problem hiding this comment.
We'll need to install the new Python deps before we run out tests here too. See the current CI failures.
Collaborator
Author
There was a problem hiding this comment.
Yeah, I tried that, but the container doesn't have pip, so it will also fail if I try to install them.
Contributor
There was a problem hiding this comment.
Yeah, we install pip to our build image then remove it again, to keep that image size small:
RUN \
apt-get -y update \
&& if [ "$(uname -m)" = "x86_64" ]; then gcc_multilib="gcc-multilib"; else gcc_multilib=""; fi \
&& apt-get -y install --no-install-recommends \
ccache \
file \
gcc \
"${gcc_multilib}" \
git \
gperf \
make \
ninja-build \
python3 \
python3-dev \
python3-pip \
python3-setuptools \
python3-wheel \
&& pip3 install \
-r https://raw.githubusercontent.com/zephyrproject-rtos/zephyr/v${ZEPHYR_VERSION}/scripts/requirements-base.txt \
&& pip3 install cmake \
&& apt-get remove -y --purge \
python3-dev \
python3-pip \
python3-setuptools \
python3-wheel \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
We could either:
- Install pip again, then install the deps, or
- Add pytest to our Docker build images (I'm curious the change in image size, I can check this), or
- Switch to the
devimage just for this.
I'm tested to say #2 makes the most sense, if it doesn't bloat things a ton.
Collaborator
Author
There was a problem hiding this comment.
Note that the tests also use the pythonsed package, not just pytest and pytest-xdist
Contributor
There was a problem hiding this comment.
Ok, have a draft PR up: zmkfirmware/zmk-docker#136 will inspect the images sizes to compare, the local builds w/ buildah here seemed just a tiny bit larger.
Replaced the run-test.sh script with a Python script that runs our unit tests through pytest. Tests are now run in parallel to speed up running the entire test suite, and it allows for integration with other tools that support pytest, such as IDEs. Also removed a dependency on remarshal, because it depends on an old version of PyYAML that conflicts with other Python packages in our Docker image. Replaced it with yq.
|
This PR has been automatically marked as stale because it has not had activity in 10 months. It will be closed in 14 days if no further activity occurs. Feel free to give a status update or re-open when it has been rebased and is ready for review (again). Thanks! |
|
This PR was closed because it had no activity for over 10 months. Feel free to give a status update or re-open when it has been rebased and is ready for review (again). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaced the run-test.sh script with a Python script that runs our unit tests through pytest. Tests are now run in parallel to speed up running the entire test suite, and it allows for integration with other tools that support pytest, such as IDEs.
Time to build and run the entire test suite on my i5-13600k running in WSL: