-
Notifications
You must be signed in to change notification settings - Fork 2
ANMS v2.0 TRR
- Review Board
- Software Overview
- Work Product Status (Documents)
- Requirements
- Improvements
- Defect Corrections
- Sustaining Activities
- Other
- Test Objectives
- Test Environment
- Key Test Cases
- Test Personnel
- Test Effort Rationale
- Test Constraint And Risks
- Action Item Status
- Deviations
| Review Board Chair | Jamie Shin |
| MGSS Chief Engineer | Michele Vogt |
| ANMS Task Lead | Brian Sipos |
| ANMS CogE | Madeline Farina |
| MGSS Software Quality Assurance Engineer | Eva Bokor |
| MGSS MCS AEM | Erik Monson |
| MGSS System Test Lead | Madonna Ryan |
| MGSS Cybersecurity Engineer | Mike Pajevski |
| MGSS Customer Support Lead | Scott Markham |
| Customer | Kathy Rockwell |
The Asynchronous Network Management System (ANMS) is a toolset to integrate a DTN Management Architecture (DTNMA) Manager with a Bundle Protocol (BP) Agent and provide user-facing tools to construct controls to remote Agents, as well as process and aggregate reports from Agents. Version 2 of ANMS is a refinement of capability and a synchronization with the latest IETF and CCSDS drafts. This release removes JPL ION as a direct deployment dependency. Because of this, the ANMS does not configure or depend upon BP security or any particular convergence layers. This removes the need to configure ION or any other BPA for the developer. The v2.0 update includes defect fixes, usability enhancements, and sustainment updates.
Release information for the software can be found in the ANMS User or Product guides found here and here respectively.
In terms of deviations, there are a number of open tickets in the ANMS repository that are not included in the v2.0 release plan, due to how certain tasks were out of scope for this release. Also note that CAMP and ACE are now managed under a different Github organization and therefore considered third-party libraries, so CAMP and ACE tickets were not included in the release plan or TRR.
In addition to the JPL-managed MGSS Controlling and Reference Documents (specifically v1.2, for SDR rev10 compliance), the following documents are being used for this release:
| Document Title | DMS Doc and Revision ID | DMS Document Status |
|---|---|---|
| ANMS TIP | DOC-002847 | Released |
| ANMS Release Plan | N/A | N/A |
| ANMS Concept of Operations for V0.1 | DOC-005437 | Released |
| ANMS Requirements (SRD) | DOC-005083 rev B | In signature |
| ANMS Architecture Design Document (ADD) | DOC-005089 | Released |
| ANMS Interface Specification (SIS) | DOC-005446 | In-review |
| ANMS Test Plan | DOC-005925 | In-review |
| ANMS Test Specification | DOC-005926 | In-review |
| ANMS Test Report | (pending) | In progress |
| ANMS Release Description (RDD) | DOC-005447 rev A | In progress |
| ANMS Product Guide | DOC-005444 rev B | In progress |
| ANMS User Guide | DOC-005443 rev B | In progress |
| ANMS Software Design Document (SDD) | DOC-005445 | Released |
We removed the following untestable requirements, once listed (and now removed) from the Software Requirements Document rev B:
• 4.2.1.2
• 4.2.4.1
• 4.3.2.1
• 4.3.2.3
• 4.4.2.4
• 4.5.1.2
• 4.5.2.2
• 4.5.3
• 4.6.3.5
Total: 9
Specific issues with enhancement label:
| Ticket | Customer | Estimate (hrs) | Security-related | Deviation | Procedure Test(s) |
|---|---|---|---|---|---|
| #38 EDDs: ARI selection on Build tab | no | unchanged from v1.2 | ANMS_FUN_BLD_007 (TYPEDEF) | ||
| #77 Upload New, Valid ADM | no | unchanged from v1.2 | ANMS_FUN_ADM_002 (Upload New, Valid ADM) | ||
| #106 Time-Based Rule: Control creation | no | unchanged from v1.2 | ANMS_FUN_BLD_001 (Time-Based Rule) | ||
| #134 Podman compatibility with ANMS | no | unchanged from v1.2 | ANMS_EXP_DAP_001 (Default Displays) | ||
| #141 Support for standardized building of containers | no | unchanged from v1.2 | ANMS_EXP_AGT_002 (Manage Agents) | ||
| #142 Network ports in docker compose file | no | unchanged from v1.2 | ANMS_EXP_SYS_002 (Login) | ||
| #143 Remove usage of docker.sock in anms-core image | no | unchanged from v1.2 | ANMS_EXP_ADM_001 (Display) | ||
| #146 Podman: review usage of privileged and cap_add flags | no | unchanged from v1.2 | ANMS_FUN_AGT_002 (Manage Agents) | ||
| #152 Make ANMS functions accessible through backend API | no | unchanged from v1.2 | ANMS_FUN_BLD_004 (Generate Tables) | ||
| #167 Remove ION dependency | no | new | ANMS_FUN_BLD_010 (State-Based Rule) | ||
| #168 Consolidate core route to translate and send command | no | new | ANMS_FUN_BLD_009 (Multiple Controls) | ||
| #169 Provide a "light" deployment | no | new | ANMS_EXP_SYS_002 (Login) | ||
| #171 Provide a direct-LDAP access control gateway | no | new | Verified by inspection | ||
| #176 Eliminate nginx container | no | new | ANMS_EXP_SYS_002 (Login) |
Total number of new improvements: 14
Note that for #171, the associate procedure test was determined to be not applicable to the testing done by our tester and was therefore removed from table 6.4 in the Test Plan.
Specific issues with bug label:
| Ticket | Customer | Estimate (hrs) | Security-related | Deviation | Procedure Test(s) |
|---|---|---|---|---|---|
| #39 EDDs: EDDs with REF | no | unchanged from v1.2 | ANMS_FUN_BLD_007 (TYPEDEF) | ||
| #112 Generate Reports: BP full report | no | unchanged from v1.2 | ANMS_FUN_BLD_002 (Generate Reports) | ||
| #121 no table generation | no | unchanged from v1.2 | ANMS_FUN_BLD_002 (Generate Reports), ANMS_FUN_BLD_004 (Generate Tables) | ||
| #140 Investigate purpose of Docker socket API | no | unchanged from v1.2 | ANMS_FUN_SYS_001 (System Status) |
| Defect Criticality | Correct Defect Count |
|---|---|
| CRIT-1 | 0 |
| CRIT-2 | 0 |
| CRIT-3 | 2 |
| CRIT-4 | 2 |
Specific issues with sustainment label:
| Ticket | Customer | Estimate (hrs) | Security-related | Deviation | Procedure Test(s) |
|---|---|---|---|---|---|
| #103 Clean up SQL scripts under amp-sql | no | unchanged from v1.2 | ANMS_FUN_DAP_001 (ARI Display), ANMS_FUN_DAP_002 (Received Reports), ANMS_FUN_AGT_001 (Agent Information), ANMS_FUN_BLD_002 (Generate Reports) | ||
| #125 Fix docker image build warnings | no | unchanged from v1.2 | ANMS_EXP_DAP_001 (Default Displays) | ||
| #126 Allow release building without starting containers | no | unchanged from v1.2 | ANMS_EXP_DAP_001 (Default Displays) | ||
| #136 Fix all Dependabot issues marked "Critical" | yes | unchanged from v1.2 | ANMS_FUN_BLD_001 (Time-Based Rule), ANMS_FUN_BLD_002 (Generate Reports) | ||
| #137 Fix all Dependabot issues marked "High" | yes | unchanged from v1.2 | ANMS_FUN_BLD_002 (Generate Reports) | ||
| #138 Fix all Dependabot issues marked "Moderate" Set 1 | yes | unchanged from v1.2 | ANMS_FUN_BLD_001 (Time-Based Rule), ANMS_FUN_BLD_002 (Generate Reports) | ||
| #139 Fix all Dependabot issues marked "Moderate" Set 2 | yes | unchanged from v1.2 | ANMS_FUN_BLD_001 (Time-Based Rule), ANMS_FUN_BLD_002 (Generate Reports) | ||
| #147 Update third-party libraries | no | unchanged from v1.2 | ANMS_FUN_AGT_002 (Manage Agents) | ||
| #148 Migrate CAMP repo | no | unchanged from v1.2 | ANMS_FUN_AGT_002 (Manage Agents) | ||
| #149 Migrate ACE repo | no | unchanged from v1.2 | ANMS_FUN_AGT_002 (Manage Agents) |
Total number of sustaining activities: 10
We are verifying the sustaining activities via regression testing (e.g. CI jobs to verify any branch or commit, and then actual regression testing done by our tester) and inspection (verified by our tester).
Specific issues with documentation label:
| Ticket | Customer | Estimate (hrs) | Security-related | Deviation | Procedure Test(s) |
|---|---|---|---|---|---|
| #28 System Status: Stopping services | no | unchanged from v1.2 | ANMS_FUN_SYS_001 (System Status) | ||
| #96 Manage Agents | no | unchanged from v1.2 | ANMS_FUN_AGT_002 (Manage Agents) | ||
| #105 Time-Based Rule: Displays | no | unchanged from v1.2 | ANMS_FUN_BLD_001 (Time-Based Rule) | ||
| #107 Time-Based Rule: parameter definition | no | unchanged from v1.2 | ANMS_FUN_BLD_001 (Time-Based Rule) | ||
| #118 ADM Listing: ANMS ADMs differ from earlier | no | unchanged from v1.2 | ANMS_FUN_ADM_001 (ADM Listing) | ||
| #144 Use volume mounts for docker/podman | no | unchanged from v1.2 | ANMS_EXP_DAP_001 (Default Displays) | ||
| #151 Update documentation of ANMS backend API | no | unchanged from v1.2 | ANMS_FUN_BLD_004 (Generate Tables) | ||
| #153 Generalize Postgres SQL accessor API | no | unchanged from v1.2 | ANMS_FUN_BLD_004 (Generate Tables) |
Total number of documentation issues: 8
Deviations from Release Plan v1.2 (i.e. these tickets were removed from RP v1.2 for RP v2.0):
| Ticket | Customer | Estimate (hrs) | Security-related | Label |
|---|---|---|---|---|
| #3 Time-Based Rule: Edit String | no | enhancement | ||
| #25 Navigation: ARI Submission | no | enhancement, documentation | ||
| #32 Time-Based Rule: Cancel option | no | enhancement | ||
| #115 State-Based Rule: Cancel option | no | enhancement | ||
| #18 User Profile: User Profile tabs | no | bug (CRIT-3) | ||
| #19 User Profile: User Profile lacks cancel option | no | bug (CRIT-3) | ||
| #27 Navigation: Page refresh | no | bug (CRIT-4) | ||
| #65 SLIM integration: Infusion of Lab-approved documentation and templating standards | no | sustainment | ||
| #132 Resolve anms-ui dependency deprecations | no | sustainment |
These tickets were not included in the v2.0 release plan because they are low-criticality bugs, low priority sustainment activities, or enhancements to the UI which were deemed unnecessary for the release. Several other tickets were created after the approval of the release plan and were deemed to be more appropriate for the next ANMS release.
The objective of these tests is to verify the fixes, enhancements, and sustainment updates identified in the release plan, as well as finding usability issues. Overall capability has changed since version 1.1.
- The ANMS proper runs on an RHEL 9 host within Podman (v5.2) containers
- Each container is based on the RedHat UBI 9 image
- All external authentication, authorization, and auditing (AAA) is provided by a CAM Gateway on the same host
- The local test environment uses an emulated CAM Gateway for isolation from external resources
- When installed with a CAM Gateway, the ANMS relies on the related CAM and user directory to operate properly
- Surrogate Agents and the Manager transport proxy for testing are provided by containers on the same host – the transport proxy and agents operate on an isolated test IP network
- The ADMs chosen for test are pre-existing and network-oriented
- A mission specific set of ADMs would be exercised as part of a more focused mission-integration test case
- Third-party libraries like CAMP and ACE are used by ANMS as well
Each of the major functional areas is described with a representative test case as follows:
Description: ADM Listing Requirements/Improvements verified:
- SRD-1977 (4.4.3) - The DAC shall allow for the query of data and the return of that data via interfaces
- SRD-2017 (4.6.2.3) - The ACT shall allow users to remove AMA mutable definitions from AMA Agents to and through the ANMS
- NOTE: The ACT will warn users if they are removing AMA Agent definitions from the local ANMS which are still known to be located on AMA Agents managed over an Agent Network
Description: Manage Agents Requirements/Improvements verified:
- SRD-1963 (4.3.5) - The CPC shall validate all data and control flows that it orchestrates through the ANMS
- SRD-1967 (4.3.5.4) -The CPC shall perform basic validation of commands from Agent Communications interfaces intended for delivery to AMA Agents
- SRD-1988 (4.5.2) -The AAC shall provide gateway interfaces for user applications on a UserNetwork to command AMA Agents
- SRD-2014 (4.6.2) The ANMS shall provide an AMA Agent Console (ACT) used for constructing and sending AMA controls to AMA Agents via the ANMS
- SRD-2016 (4.6.2.2) - The ACT shall allow users to send AMA mutable object definitions to AMA Agents to and through the ANMS
Description: Generate Reports Requirements/Improvements verified:
- SRD-1945 (4.3.1.2) - The CPC shall persistently store all AMA messages using Data Access interfaces
- SRD-1946 (4.3.1.3) - The CPC shall pass AMA messages to the TCC using AMA Packaging Interfaces
- SRD-2019 (4.6.2.5) - The ACT shall display available objects to the user to assist in the creation of AMA mutable objects and control execution
Description: Received Reports Requirements/Improvements verified:
- SRD-1956 (4.3.3.2) - The CPC shall track data processed to the extent necessary to determine the status of loaded automation controls
- SRD-1969 (4.4.1) - The DAC shall transcode and otherwise alter the representation of data to aid in the efficiency of data storage and retrieval
- SRD-1971 (4.4.2) -The DAC shall persist data for long-term storage in a way that can be retrieved by the on-line system
- SRD-1972 (4.4.2.1) - The DAC shall maintain records of all historical messages and events within the ANMS system
- SRD-2000 (4.6.1.3) -The DVA shall display network health statistics
Description: User Profile Requirements/Improvements verified:
- SRD-2006 (4.6.3.2) - The ADC shall provide mechanisms to create, update, and remove users from the ANMS.
- SRD-2007 (4.6.3.3) - The ADC shall provide mechanisms to create, update, and remove user roles and groups from the ANMS.
- SRD-2008 (4.6.3.4) - The ADC shall maintain the mapping of users to user roles and groups.
Description: Valid Message Conversion Requirements/Improvements verified:
- SRD-1930 (4.2.1) - The TCC shall convert AMA objects from one supported AMA format to another through the use of codecs
- SRD-1931 (4.2.1.1) - The TCC shall provide functions and codecs for the translation to/from defined AMA object formats
- SRD-1940 (4.2.4) - The TCC shall store information needed to properly transcode and verify AMA objects
| Person | Role | Assignment(s) | Availability (hours) |
|---|---|---|---|
| Karen Goldee | Test User | Execute test procedures | 100 |
| David Linko | Support | Supports SW components | 20 |
| Brian Sipos | Support | Supports SW deployment | 20 |
Total hours: 140
When used with a CAM Emulator and local surrogate Agents, there are no external dependencies associated with testing. The test effort schedule for formal testing is 9/4/2025 through 10/14/2025, and the total test execution time is summed from the individual test case estimates in the ANMS Test Plan. Test personnel are shared among a few other active projects.
If testing with a CAM Server is desired, then more coordination with the test environment is needed because the CAM Gateway must run with an associated CAM Server. Likewise, if testing with external Agents is desired then coordination with hosts/owners of those Agents is needed because the Agent—Manager association requires some manual configuration.
- Action Item MGSSAITS-1826 : opened at the v1.1 DDR (now closed)
None to report