HYRAX-2096 Replicate request-log information into response-log to facilitate troubleshooting.

[dh-opendap]

Modified resources/ngap/logback.xml; Mirrored 'collectionId' and 'query_string' into CloudWatchResponseLogAppender pattern.

[ndp-opendap]

[hannahilea]

I think we also want user_id, user_ip, host, and (maybe?) parameters, for full parity.

[dh-opendap]

Adding the suggested fields (.i.e., user_id, user_ip,...) begs the question as to how the current log searching, via a join operation between the response and request logs, how its currently used. My feeling is we don't add those fields until we're certain we'll need them. Once this change to the logging is implemented we'll be working with it to test whether it meets the current use case requirements.

[hannahilea]

Hmm, okay. My understanding of the original issue was that all fields currently present in the request log would now be included in the response log, not just a few additional fields. (If I'd realized we weren't going to copy all of them I would have encouraged us to be specific about which ones when we created the acceptance criteria).

For a different ticket (that's in my queue for next sprint) I will definitely need user_id and user_ip to be in the response log, so if we don't add those here I'll need to add a follow-up PR for adding them. (I don't know what host and parameters are used for, so I don't feel strongly about copying those in.)

[jgallagher59701]

We need all the fields from the request log in the response log. Basically, the request log is like the appendix in a persn; we inherited from a different animal and we don't need it (the different animal being Harmony). The original idea was that all the transformation services would use this 'schema' but that never happened because the services are all different.

Thanks and sorry for not making that obvious from the start.

[ndp-opendap]

@jgallagher59701 said:

We need all the fields from the request log in the response log

I would like to point out that there are a number of fields in the request log that we included because we were told to but we do not use:

NGAP Request Required Fields

    "required": [
        "request_id",
        "user_id",
        "user_ip",
        "rangeBeginDateTime",
        "rangeEndDateTime",
        "collectionId",
        "query_string", # Added by us because..."
        "bbox",
        "parameters"
    ]

Many of which are either static stuff in the parameters array or that we leave blank (rangeBeginDateTime & rangeEndDateTime) in our records sent to CloudWatch:

{ 
    "request_id": "%X{ID}", 
    "user_id": "%X{userid}", 
    "user_ip": "%X{host}", 
    "rangeBeginDateTime": "", 
    "rangeEndDateTime": "", 
    "collectionId": "%X{resourceID}", 
    "query_string": "%X{query}", 
    "parameters": { 
        "service_name": "hyrax", 
        "service_provider": "OPeNDAP", 
        "service_id": "hyrax_prod" 
    } 
}

So I think adding all that stuff is pretty pointless

[jgallagher59701]

Thanks

[dh-opendap]

OK. Mirrored the request log pattern into the response log. Tested locally, not sure why user_id and user_ip is returned as it is but otherwise everything looks good. Here is an example of the updated response log asking for 'fnoc1.nc.asc?lat'.

{ "request_id": "http-nio-8080-exec-10_37-03532546-ac42-4a8c-a29e-3e8b0bb9fc86", "user_id": "-", "user_ip": "0:0:0:0:0:0:0:1", "rangeBeginDateTime": "", "rangeEndDateTime": "", "collectionId": "/opendap/hyrax/data/nc/fnoc1.nc.ascii", "query_string": "lat", "parameters": { "service_name": "hyrax", "service_provider": "OPeNDAP", "service_id": "hyrax_prod" }, "job_ids": ["N/A"], "http_response_code": 200, "time_completed": "2026-04-21T14:39:21-0400", "total_time": 9, "output_size": 106 }

Changes have been pushed.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud