fix: DH-20255 Limit maximum depth to serialize redux log data#2544
fix: DH-20255 Limit maximum depth to serialize redux log data#2544ericlln merged 3 commits intodeephaven:mainfrom
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2544 +/- ##
=======================================
Coverage 44.95% 44.96%
=======================================
Files 765 765
Lines 42920 42924 +4
Branches 10799 10801 +2
=======================================
+ Hits 19296 19299 +3
- Misses 23611 23612 +1
Partials 13 13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
mofojed
left a comment
There was a problem hiding this comment.
Do you have examples of the keys that were problematic?
| ['layoutStorage'], | ||
| ['storage'], | ||
|
|
||
| // Below are confirmed enterprise specific keys, and will be moved in DH-20410 |
There was a problem hiding this comment.
draftManager is also an enterprise specific key.
Okay to have this here for now, but on Enterprise side we should just add to the list at https://github.com/deephaven-ent/iris/blob/rc/sanluis/web/client-ui/src/utils/exportEnterpriseLogs.ts#L18
Follow up PR to #2531 which failed testing, with an error from
JSON.stringify()ofRangeError: Invalid string length.These errors are caused by linked structures such as a Java LinkedHashMap that's included in the Redux data in various API related keys. We use the blacklist to filter out keys containing these structures, but these keys are not stable and will need to be constantly updated to prevent the log export from outright failing.
To preemptively deal with this, the maximum depth to serialize to can be limited in the safe-stable-stringify library to avoid linked structures from blowing up the log size. With no blacklist entries, the exported log size is only
1.2MB.With an updated blacklist, exporting the logs with or without the limit both yield log files of almost the exact same size, meaning that no useful data has been lost by limiting the depth. However, I'm open to suggestions to the default depth to serialize to, or whether it should be optional to have a limit on the depth instead of mandating a limit (
Infinityis not accepted by the library).