The following guide indicates how to run the tool against an implementation of one of the NMOS APIs.
| Description | Screenshot |
|---|---|
| Open a browser and go to the address of the server you deployed. |  |
| Select a test suite from the dropdown. |  |
| Provide the IP or hostname and port of the relevant API under test, along with the preferred API version. |  |
| Optionally, select a subset of the tests to run. This should default to 'all'. |  |
| Once all of the field have been completed, press 'Run'. The result of the tests will be shown after a few seconds. |  |
The result of each test case will be one of the following:
| Test Result | Reason |
|---|---|
 |
Successful test case. |
 |
Required feature of the specification has been found to be implemented incorrectly. |
 |
Not a failure, but the API being tested is responding or configured in a way which is not recommended in most cases. |
 |
Test is disabled due to test suite configuration; change the config or test manually. |
 |
Test was not run due to prior responses from the API, which may be OK, or indicate a fault. |
 |
Recommended/optional feature of the specifications has been found to be not implemented. |
 |
Test suite does not currently test this feature, so it must be tested manually. |
 |
Test is not applicable, e.g. due to the version of the specification being tested. |
The testing tool has a number of general configuration settings that affect its operation, defined in nmostesting/Config.py.
To alter these, copy nmostesting/UserConfig.example.py to nmostesting/UserConfig.py and make your changes there.
MAX_TEST_ITERATIONS can be set to restrict the maximum number of resources or test points that time-consuming tests run against.
The default of 0 means unlimited (all available resources or test points) for a really thorough test!
The following pages provide information on additional instructions for testing specific features:
- Unicast Discovery
- BCP-003-01
- Authorization
- SDP Files
- IS-07 MQTT
- Controllers
- Invasive Device Model Testing
The testing tool supports two modes of operation that support usage in larger scale or continuous integration tests.
Some of the tests contained within this tool perform a number of steps which may not be obvious without viewing the source code. To help with debugging, descriptions of such behaviour is covered by the advanced testing pages.