Skip to content

Remove examples and other unwanted artifacts from installed dependencies#4896

Merged
ashwin-pc merged 1 commit intoopensearch-project:mainfrom
AMoo-Miki:post-install-cleanup-unwanted
Sep 1, 2023
Merged

Remove examples and other unwanted artifacts from installed dependencies#4896
ashwin-pc merged 1 commit intoopensearch-project:mainfrom
AMoo-Miki:post-install-cleanup-unwanted

Conversation

@AMoo-Miki
Copy link
Copy Markdown
Collaborator

Description

Remove examples and other unwanted artifacts from installed dependencies using a postinstall script.

List of folders that will be deleted:

% find node_modules -type d -name "demo"
node_modules/opentracing/lib/examples/demo
node_modules/opentracing/src/examples/demo
node_modules/argparse/node_modules/sprintf-js/demo
node_modules/leaflet.heat/demo
node_modules/grunt-peg/node_modules/argparse/node_modules/sprintf-js/demo
node_modules/leaflet-vega/demo
node_modules/grunt-run/node_modules/argparse/node_modules/sprintf-js/demo
node_modules/tinycolor2/demo
node_modules/grunt-available-tasks/node_modules/argparse/node_modules/sprintf-js/demo

% find node_modules -type d -name "example" 
node_modules/@types/http-aws-es/node_modules/is-typed-array/node_modules/object-inspect/example
node_modules/@types/http-aws-es/node_modules/object-inspect/example
node_modules/@types/globby/node_modules/safe-regex/example
node_modules/fastest-stable-stringify/example
node_modules/defined/example
node_modules/is-typed-array/node_modules/object-inspect/example
node_modules/wordwrap/example
node_modules/vm-browserify/example
node_modules/brace/example
node_modules/copy-to-clipboard/example
node_modules/text-table/example
node_modules/fast-json-stable-stringify/example
node_modules/camelize/example
node_modules/resolve/example
node_modules/retry/example
node_modules/deep-equal/example
node_modules/fast-redact/example
node_modules/parse-link-header/example
node_modules/typedarray/example
node_modules/grunt/node_modules/resolve/example
node_modules/toggle-selection/example
node_modules/commondir/example
node_modules/json-stable-stringify/example
node_modules/minimist/example
node_modules/resumer/example
node_modules/grunt-cli/node_modules/resolve/example
node_modules/crypto-browserify/example
node_modules/json-stable-stringify-without-jsonify/example
node_modules/grunt-peg/node_modules/resolve/example
node_modules/grunt-peg/node_modules/concat-map/example
node_modules/lmdb/dependencies/lz4/lib/dll/example
node_modules/deep-freeze-strict/example
node_modules/@microsoft/api-documenter/node_modules/resolve/example
node_modules/@microsoft/tsdoc-config/node_modules/resolve/example
node_modules/@microsoft/api-extractor/node_modules/resolve/example
node_modules/glob-all/example
node_modules/concat-map/example
node_modules/detective/example
node_modules/tape/example
node_modules/tape/node_modules/resolve/example
node_modules/load-bmfont/node_modules/buffer-equal/example
node_modules/grunt-run/node_modules/resolve/example
node_modules/grunt-run/node_modules/concat-map/example
node_modules/eslint-plugin-react/node_modules/resolve/example
node_modules/dom-walk/example
node_modules/safe-regex/example
node_modules/@rushstack/node-core-library/node_modules/resolve/example
node_modules/@rushstack/rig-package/node_modules/resolve/example
node_modules/deep-is/example
node_modules/object-inspect/example
node_modules/grunt-available-tasks/node_modules/resolve/example
node_modules/grunt-available-tasks/node_modules/concat-map/example
node_modules/buffer-equal/example

% find node_modules -type d -name "examples"
node_modules/opentracing/lib/examples
node_modules/opentracing/src/examples
node_modules/jsonparse/examples
node_modules/needle/examples
node_modules/element-resize-detector/examples
node_modules/jimp/browser/examples
node_modules/JSONStream/examples
node_modules/archy/examples
node_modules/grunt/node_modules/colors/examples
node_modules/grunt/node_modules/nopt/examples
node_modules/ngreact/examples
node_modules/react-input-autosize/examples
node_modules/leaflet-draw/docs/examples
node_modules/license-checker/node_modules/mkdirp/examples
node_modules/pegjs/examples
node_modules/grunt-peg/node_modules/pegjs/examples
node_modules/grunt-peg/node_modules/colors/examples
node_modules/grunt-peg/node_modules/nopt/examples
node_modules/treeify/examples
node_modules/colors/examples
node_modules/react-dropzone/examples
node_modules/murmurhash3js/examples
node_modules/d3-cloud/examples
node_modules/traverse/examples
node_modules/socks/docs/examples
node_modules/wildcard/examples
node_modules/ui-select/docs/examples
node_modules/grunt-run/node_modules/colors/examples
node_modules/grunt-run/node_modules/nopt/examples
node_modules/grunt-available-tasks/node_modules/colors/examples
node_modules/grunt-available-tasks/node_modules/nopt/examples
node_modules/nopt/examples
node_modules/worker-farm/examples

Issues Resolved

/node_modules/leaflet-draw/docs/examples
Fixes #4722
Fixes #4723
Fixes #4725
Fixes #4727
Fixes #4728
Fixes #4729

/node_modules/leaflet-vega/demo
Fixes #3525

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
    • yarn test:ftr
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 1, 2023

Codecov Report

Merging #4896 (7208291) into main (83cd7f6) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4896      +/-   ##
==========================================
- Coverage   66.39%   66.39%   -0.01%     
==========================================
  Files        3397     3397              
  Lines       64804    64804              
  Branches    10360    10360              
==========================================
- Hits        43028    43026       -2     
- Misses      19217    19224       +7     
+ Partials     2559     2554       -5     
Flag Coverage Δ
Linux_1 34.85% <ø> (ø)
Linux_2 55.15% <ø> (ø)
Linux_3 44.23% <ø> (ø)
Linux_4 34.92% <ø> (ø)
Windows_1 34.87% <ø> (ø)
Windows_2 55.12% <ø> (ø)
Windows_3 44.24% <ø> (-0.01%) ⬇️
Windows_4 34.92% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 6 files with indirect coverage changes

@kavilla
Copy link
Copy Markdown
Member

kavilla commented Sep 1, 2023

So the post install is targeted to be a specific runtime configuration of OpenSearch Dashboards? Or can examples not be outside of a release build?

@AMoo-Miki
Copy link
Copy Markdown
Collaborator Author

So the post install is targeted to be a specific runtime configuration of OpenSearch Dashboards? Or can examples not be outside of a release build?

Having the unwanted stuff is not only undesirable for releases but also undesirable / dangerous for contributors.

Copy link
Copy Markdown
Member

@joshuarrrr joshuarrrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I pulled and tried it and it appears to work great.

We could consider a brief explanatory log message (something like "removing example and demo files from depenedencies") as output:

yarn osd bootstrap
yarn run v1.22.19
$ scripts/use_node scripts/osd bootstrap
 info [opensearch-dashboards] running yarn

$ scripts/use_node ./preinstall_check
[1/5] Validating package.json...
[2/5] Resolving packages...
warning Resolution field "typescript@4.0.2" is incompatible with requested version "typescript@~4.5.2"
[3/5] Fetching packages...
[4/5] Linking dependencies...
warning "workspace-aggregator-9f9bce3f-24d7-47b2-9651-cfdc201888fc > osd_tp_run_pipeline > @elastic/eui@1.3.0-beta.2" has incorrect peer dependency "typescript@^4.0.5".
warning "@osd/ace > raw-loader@4.0.2" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/interpreter > babel-loader@8.2.4" has unmet peer dependency "webpack@>=2".
warning "@osd/interpreter > copy-webpack-plugin@6.4.1" has unmet peer dependency "webpack@^4.37.0 || ^5.0.0".
warning "@osd/interpreter > css-loader@5.2.7" has unmet peer dependency "webpack@^4.27.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > @osd/ui-shared-deps > sass-loader@10.4.1-node-sass-9.0.0-libsass-3.6.5" has unmet peer dependency "webpack@^4.36.0 || ^5.0.0".
warning "@osd/interpreter > style-loader@1.3.0" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/interpreter > url-loader@2.3.0" has unmet peer dependency "webpack@^4.0.0".
warning "@osd/interpreter > webpack-cli@4.9.2" has unmet peer dependency "webpack@4.x.x || 5.x.x".
warning "@osd/interpreter > webpack-cli > @webpack-cli/configtest@1.1.1" has unmet peer dependency "webpack@4.x.x || 5.x.x".
warning "@osd/ui-framework > @osd/optimizer > clean-webpack-plugin@3.0.0" has unmet peer dependency "webpack@*".
warning "@osd/ui-framework > @osd/optimizer > @osd/ui-shared-deps > compression-webpack-plugin@4.0.1-rc.1" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > terser-webpack-plugin@2.3.8" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > file-loader@6.2.0" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > postcss-loader@4.3.0" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > val-loader@2.1.2" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > @osd/ui-shared-deps > mini-css-extract-plugin@1.6.2" has unmet peer dependency "webpack@^4.4.0 || ^5.0.0".
warning "workspace-aggregator-9f9bce3f-24d7-47b2-9651-cfdc201888fc > http-aws-es@6.0.1" has unmet peer dependency "aws-sdk@^2.138.0".
warning "workspace-aggregator-9f9bce3f-24d7-47b2-9651-cfdc201888fc > http-aws-es@6.0.1" has incorrect peer dependency "elasticsearch@^15.0.0".
warning "@osd/eslint-import-resolver-opensearch-dashboards > eslint-import-resolver-webpack@0.11.1" has unmet peer dependency "webpack@>=1.11.0".
[5/5] Building fresh packages...
$ scripts/use_node scripts/postinstall

 succ yarn.lock analysis completed without any issues
 info [@osd/cross-platform] running [osd:bootstrap] script
 info [@osd/stylelint-plugin-stylelint] running [osd:bootstrap] script
 info [@osd/test-subj-selector] running [osd:bootstrap] script
 info [@osd/utility-types] running [osd:bootstrap] script
 succ [@osd/test-subj-selector] bootstrap complete
 succ [@osd/utility-types] bootstrap complete
 succ [@osd/stylelint-plugin-stylelint] bootstrap complete
 succ [@osd/cross-platform] bootstrap complete
 info [@osd/config-schema] running [osd:bootstrap] script
 info [@osd/std] running [osd:bootstrap] build targets
 succ [@osd/std] bootstrap complete
 succ [@osd/config-schema] bootstrap complete
 info [@osd/logging] running [osd:bootstrap] script
 info [@osd/utils] running [osd:bootstrap] script
 succ [@osd/logging] bootstrap complete
 succ [@osd/utils] bootstrap complete
 info [@osd/apm-config-loader] running [osd:bootstrap] script
 info [@osd/dev-utils] running [osd:bootstrap] script
 succ [@osd/apm-config-loader] bootstrap complete
 succ [@osd/dev-utils] bootstrap complete
 info [@osd/ace] running [osd:bootstrap] build targets
 info [@osd/analytics] running [osd:bootstrap] script
 info [@osd/config] running [osd:bootstrap] script
 info [@osd/i18n] running [osd:bootstrap] build targets
 succ [@osd/analytics] bootstrap complete
 info [@osd/monaco] running [osd:bootstrap] script
 succ [@osd/ace] bootstrap complete
 info [@osd/opensearch-archiver] running [osd:bootstrap] script
 succ [@osd/i18n] bootstrap complete
 info [@osd/opensearch] running [osd:bootstrap] script
 succ [@osd/config] bootstrap complete
 info [@osd/plugin-generator] running [osd:bootstrap] script
 succ [@osd/opensearch] bootstrap complete
 info [@osd/telemetry-tools] running [osd:bootstrap] script
 succ [@osd/telemetry-tools] bootstrap complete
 info [@osd/test] running [osd:bootstrap] script
 succ [@osd/test] bootstrap complete
 succ [@osd/plugin-generator] bootstrap complete
 succ [@osd/monaco] bootstrap complete
 succ [@osd/opensearch-archiver] bootstrap complete
 info [@osd/interpreter] running [osd:bootstrap] script
 info [@osd/ui-shared-deps] running [osd:bootstrap] script
 succ [@osd/interpreter] bootstrap complete
 succ [@osd/ui-shared-deps] bootstrap complete
 info [@osd/optimizer] running [osd:bootstrap] script
 succ [@osd/optimizer] bootstrap complete
 info [@osd/plugin-helpers] running [osd:bootstrap] script
 succ [@osd/plugin-helpers] bootstrap complete
 info [opensearch-dashboards] running [osd:bootstrap] script
 succ [opensearch-dashboards] bootstrap complete
Done in 152.28s.

but maybe it's better for it to be transparent unless there's an error.

@ashwin-pc
Copy link
Copy Markdown
Member

I like the idea of a log message too

@ashwin-pc ashwin-pc merged commit cf07424 into opensearch-project:main Sep 1, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 2, 2023
…ies (#4896)

Signed-off-by: Miki <miki@amazon.com>
(cherry picked from commit cf07424)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md
AMoo-Miki pushed a commit that referenced this pull request Sep 5, 2023
…ies (#4896) (#4905)

Signed-off-by: Miki <miki@amazon.com>
(cherry picked from commit cf07424)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

4 participants