You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+64Lines changed: 64 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -60,6 +60,70 @@ GitHub provides additional document on [forking a repository](https://help.githu
60
60
[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
61
61
62
62
63
+
## Integration Test Guidelines
64
+
65
+
Integration tests run in CI via `.github/workflows/integration-tests.yml`. All jobs have AWS credentials. Tests in `build` and `local` jobs that require credentials are separated using a pytest marker.
66
+
67
+
### Tests that require AWS credentials
68
+
69
+
Some tests in `tests/integration/buildcmd/` and `tests/integration/local/` need AWS credentials (e.g. STS calls, Lambda layer publishing, SAR template resolution). These are:
70
+
71
+
-**Excluded** from build/local jobs via `-m "not requires_credential"`
72
+
-**Collected and run** in the dedicated `cloud-based-tests` CI job via `-m requires_credential`
73
+
74
+
To mark a test that requires AWS credentials, add the marker:
75
+
76
+
```python
77
+
import pytest
78
+
79
+
@pytest.mark.requires_credential
80
+
classTestMyCloudFeature(SomeBaseClass):
81
+
...
82
+
```
83
+
84
+
The marker is registered in `tests/conftest.py`. Build and local jobs automatically exclude these tests; `cloud-based-tests` automatically includes them.
85
+
86
+
### Docker container cleanup in parallel tests
87
+
88
+
`start-api` and `start-lambda` tests run in parallel (`-n 2`). Each test class snapshots existing Docker container IDs before starting its local server, and on teardown only removes containers created after the snapshot. This prevents one worker from killing another worker's containers.
89
+
90
+
If you write a new base class that manages Docker containers, follow the same pattern in `start_lambda_api_integ_base.py` and `start_api_integ_base.py`: snapshot container IDs in `setUpClass`, and scope removal to only new containers in `tearDownClass`.
91
+
92
+
### Tier 1 cross-platform smoke tests
93
+
94
+
A curated subset of ~50 tests marked with `@pytest.mark.tier1` runs on every OS/container-runtime combination (e.g. Linux+Finch). These validate platform-specific code paths: file system operations, container runtime interaction, process spawning, and network operations.
95
+
96
+
There are two markers:
97
+
98
+
-`tier1` — standalone tests that don't duplicate any parameterized entry (e.g. a dedicated `test_tier1_*` method calling unique logic)
99
+
-`tier1_extra` — dedicated `test_tier1_*` methods whose parameter set already exists in a `@parameterized.expand` list on the same class. These are excluded from normal Linux+Docker CI (via `-m "not tier1_extra"`) to avoid running the same test twice, but included in the tier1 Finch job (via `-m "tier1 or tier1_extra"`).
100
+
101
+
Run locally with: `pytest -m "tier1 or tier1_extra" tests/integration tests/regression`
102
+
103
+
To add a tier 1 test for a new feature:
104
+
105
+
1. Add a dedicated `test_tier1_*` method that calls the existing test logic with one specific parameter set:
2. If the parameter set already exists in a `@parameterized.expand` list, keep it in the original list and use `@pytest.mark.tier1_extra` instead of `@pytest.mark.tier1` on the dedicated method. This avoids running the same test twice in Linux+Docker CI.
121
+
122
+
3. If the parameter set does NOT exist in any parameterized list (i.e. it's unique to the tier1 method), use `@pytest.mark.tier1`.
123
+
124
+
4. Each runtime should have one non-container and one container tier 1 test.
125
+
126
+
63
127
## Finding contributions to work on
64
128
Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws/aws-sam-cli/labels/help%20wanted) issues is a great place to start.
0 commit comments