Skip to content

Commit 975d670

Browse files
opensearch-trigger-bot[bot]github-actions[bot]MikimengweiericSuZhou-Joe
authored
[2.x][Manual Backport ] Back port main to 2.x (#7477) (#7487)
* Bump OUI to 1.8.0 (#7363) (cherry picked from commit c1bae75) * [Auto Suggest] OpenSearch SQL autosuggest with ANTLR (#7336) * cherry pick * SQL keyword suggestion * column and values suggestions * adjust values suggestions and ignored token * suggestion type casting * ignore functions * case insensitive * change to use grammar for insensitive case * fix multi-line issue * code cleanup * move ppl out of scope * rename folder and add ignoring rules * resolve type issue * fix ppl suggestion provider issue * remove function suggestion * some code clean up and adding tests * cursor tests * remove testing setup * add changelog * update yarn file * add missing testing library * MDS integration * minor interface change and disable word based suggestion * update ID_LITERAL and recompile grammar * column suggest for agg function * revert version * opensearch sql syntax highlighting * add utility tests * symbol table test * error listener test * parse test * add suggestion provider to single line query editor * remove one test * add connection service type --------- (cherry picked from commit 9348bd4) * Move TopNavLinks to new Nav Bar Discover (#7326) This PR aims to introduce a new Nav bar in Discover that would contain the Date Picker as well as the Navigation links that are currently part of headers. The Navigation Links have been replaced with the corresponding Icons to provide a modern UX. These setting are currently controlled by query:enhancements:enabled Advanced Setting Flag and would be visible when the above flag is turned on. (cherry picked from commit e46e54a) * [navigation-next]fix: breadcrumb issue found in data source management and other similar cases. (#7401) * feat: add scoped breadcrumbs * Changeset file for PR #7401 created/updated * feat: move data source management register logic up a little bit * feat: add unit test * feat: revert the home related change * fix: overview error in all use case when workspace is enabled * fix: update snapshot * Changeset file for PR #7401 created/updated * fix: hide nav groups that should be displayed * feat: hide expand icon in left navigation * feat: update --------- (cherry picked from commit 3f78c94) * [Bug][Data Source] Move data source manageable feature flag to DSM plugin (#7440) * Move data source manageable feature flag to DSM plugin * Changeset file for PR #7440 created/updated --------- (cherry picked from commit dfcd2e1) * [Discover-next] data set picker (#7426) * update using query manager * aggs not working * almost there * stablish * thanks ashwin * update ref name * fix timefields * clean up some console logs * go safer route of setting language * its working again * fix names * restore code editor and indices * sql df * fix external datasources again --------- (cherry picked from commit 6a079d3) * Fix add filter for numeric scripted field (#7022) * Fix add filter for scripted field * add unit test * change from nested ternary to if else * add more unit tests * change unit tests * fix typo --------- (cherry picked from commit 7c1f8fc) * [Bug][Workspace] Add permission validation at workspace detail page (#7435) * [Bug][Workspace] Add permission validate at workspace detail page * Changeset file for PR #7435 created/updated --------- (cherry picked from commit 45d34b5) * Revert "[Discover-next] data set picker (#7426)" (#7479) This reverts commit 6a079d3. --------- (cherry picked from commit 0515fb2) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Miki <miki@amazon.com> Co-authored-by: Eric Wei <menwe@amazon.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: yuboluo <yubonluo@amazon.com> Co-authored-by: Kawika Avilla <kavilla414@gmail.com> Co-authored-by: Qingyang(Abby) Hu <abigailhu2000@gmail.com> Co-authored-by: Sean Li <lnse@amazon.com>
1 parent 95b2ef6 commit 975d670

File tree

76 files changed

+44032
-115
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

76 files changed

+44032
-115
lines changed

‎.eslintignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,3 +30,7 @@ target
3030
/packages/osd-test/src/functional_test_runner/lib/config/__tests__/fixtures/
3131
/packages/osd-ui-framework/dist
3232
/packages/osd-ui-shared-deps/flot_charts
33+
34+
# antlr overrides
35+
/src/plugins/data/public/antlr/opensearch_sql/.generated/*
36+
/src/plugins/data/public/antlr/opensearch_sql/grammar/**/*

‎.eslintrc.js‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -751,5 +751,14 @@ module.exports = {
751751
'no-undef': 'off',
752752
},
753753
},
754+
{
755+
files: [
756+
'src/plugins/data/public/antlr/opensearch_sql/.generated/*',
757+
'src/plugins/data/public/antlr/opensearch_sql/grammar/**/*',
758+
],
759+
rules: {
760+
'filenames/match-regex': 'off',
761+
},
762+
},
754763
],
755764
};

‎changelogs/fragments/7336.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
feat:
2+
- [Auto Suggest] OpenSearch SQL autosuggest with ANTLR ([#7336](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7336))

‎changelogs/fragments/7363.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
feat:
2+
- Bump OUI to 1.8.0 ([#7363](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/7363))

‎config/opensearch_dashboards.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -362,4 +362,4 @@
362362
# Set the backend roles in groups or users, whoever has the backend roles or exactly match the user ids defined in this config will be regard as dashboard admin.
363363
# Dashboard admin will have the access to all the workspaces(workspace.enabled: true) and objects inside OpenSearch Dashboards.
364364
# opensearchDashboards.dashboardAdmin.groups: ["dashboard_admin"]
365-
# opensearchDashboards.dashboardAdmin.users: ["dashboard_admin"]
365+
# opensearchDashboards.dashboardAdmin.users: ["dashboard_admin"]

‎package.json‎

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,12 @@
7777
"osd:bootstrap": "scripts/use_node scripts/build_ts_refs && scripts/use_node scripts/register_git_hook",
7878
"spec_to_console": "scripts/use_node scripts/spec_to_console",
7979
"pkg-version": "scripts/use_node -e \"console.log(require('./package.json').version)\"",
80-
"release_note:generate": "scripts/use_node scripts/generate_release_note"
80+
"release_note:generate": "scripts/use_node scripts/generate_release_note",
81+
"cypress:run-without-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=false",
82+
"cypress:run-with-security": "env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500",
83+
"osd:ciGroup10": "echo \"dashboard_sanity_test_spec.js\"",
84+
"osd:ciGroup11": "echo \"apps/vis_builder/*.js\"",
85+
"generate:opensearchsqlantlr": "./node_modules/antlr4ng-cli/index.js -Dlanguage=TypeScript -o ./src/plugins/data/public/antlr/opensearch_sql/.generated -visitor -no-listener -Xexact-output-dir ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLLexer.g4 ./src/plugins/data/public/antlr/opensearch_sql/grammar/OpenSearchSQLParser.g4"
8186
},
8287
"repository": {
8388
"type": "git",
@@ -109,7 +114,7 @@
109114
"**/qs": "^6.11.0",
110115
"**/semver": "^7.5.3",
111116
"**/set-value": "^4.1.0",
112-
"**/tar":"^6.2.1",
117+
"**/tar": "^6.2.1",
113118
"**/topo/hoek": "npm:@amoo-miki/hoek@6.1.3",
114119
"**/trim": "^0.0.3",
115120
"**/typescript": "4.0.2",
@@ -140,7 +145,7 @@
140145
"dependencies": {
141146
"@aws-crypto/client-node": "^3.1.1",
142147
"@elastic/datemath": "5.0.3",
143-
"@elastic/eui": "npm:@opensearch-project/oui@1.7.0",
148+
"@elastic/eui": "npm:@opensearch-project/oui@1.8.0",
144149
"@elastic/good": "^9.0.1-kibana3",
145150
"@elastic/numeral": "npm:@amoo-miki/numeral@2.6.0",
146151
"@elastic/request-crypto": "2.0.0",
@@ -177,6 +182,8 @@
177182
"JSONStream": "1.3.5",
178183
"abortcontroller-polyfill": "^1.4.0",
179184
"ajv": "^8.11.0",
185+
"antlr4-c3": "^3.4.1",
186+
"antlr4ng": "^3.0.4",
180187
"bluebird": "3.5.5",
181188
"chalk": "^4.1.0",
182189
"chokidar": "^3.4.2",
@@ -242,11 +249,13 @@
242249
"type-detect": "^4.0.8",
243250
"uuid": "3.3.2",
244251
"whatwg-fetch": "^3.0.0",
245-
"yauzl": "^2.10.0"
252+
"yauzl": "^2.10.0",
253+
"@opensearch-dashboards-test/opensearch-dashboards-test-library": "https://github.com/opensearch-project/opensearch-dashboards-test-library/archive/refs/tags/1.0.6.tar.gz"
246254
},
247255
"devDependencies": {
248256
"@babel/core": "^7.22.9",
249257
"@babel/parser": "^7.22.9",
258+
"@babel/plugin-transform-class-static-block": "^7.24.4",
250259
"@babel/register": "^7.22.9",
251260
"@babel/types": "^7.22.9",
252261
"@elastic/apm-rum": "^5.6.1",
@@ -361,6 +370,7 @@
361370
"@types/zen-observable": "^0.8.0",
362371
"@typescript-eslint/eslint-plugin": "^3.10.0",
363372
"@typescript-eslint/parser": "^3.10.0",
373+
"antlr4ng-cli": "^2.0.0",
364374
"archiver": "^5.3.0",
365375
"axe-core": "^4.0.2",
366376
"babel-eslint": "^10.0.3",
@@ -461,6 +471,7 @@
461471
"regenerate": "^1.4.0",
462472
"reselect": "^4.0.0",
463473
"resize-observer-polyfill": "^1.5.1",
474+
"rimraf": "^5.0.7",
464475
"selenium-webdriver": "^4.0.0-alpha.7",
465476
"simple-git": "^3.16.0",
466477
"sinon": "^7.4.2",

‎packages/osd-babel-preset/webpack_preset.js‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,12 @@ module.exports = () => {
5555
fileName: false,
5656
},
5757
],
58+
[
59+
require.resolve('@babel/plugin-transform-class-static-block'),
60+
{
61+
fileName: false,
62+
},
63+
],
5864
],
5965
env: {
6066
production: {

‎packages/osd-monaco/src/xjson/lexer_rules/index.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ import * as monaco from 'monaco-editor/esm/vs/editor/editor.api';
3333
import * as xJson from './xjson';
3434
import * as opensearchql from './opensearchql';
3535
import * as painless from './painless';
36+
import * as opensearchsql from './opensearchsql';
3637

3738
export const registerLexerRules = (m: typeof monaco) => {
3839
m.languages.register({ id: xJson.ID });
@@ -41,4 +42,6 @@ export const registerLexerRules = (m: typeof monaco) => {
4142
m.languages.setMonarchTokensProvider(painless.ID, painless.lexerRules);
4243
m.languages.register({ id: opensearchql.ID });
4344
m.languages.setMonarchTokensProvider(opensearchql.ID, opensearchql.lexerRules);
45+
m.languages.register({ id: opensearchsql.ID });
46+
m.languages.setMonarchTokensProvider(opensearchsql.ID, opensearchsql.lexerRules);
4447
};

‎packages/osd-monaco/src/xjson/lexer_rules/opensearchsql.ts‎

Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
/*
2+
* Copyright OpenSearch Contributors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
import { monaco } from '../../monaco';
7+
8+
export const ID = 'SQL';
9+
10+
const keywords = [
11+
'ALL',
12+
'AND',
13+
'AS',
14+
'ASC',
15+
'BOOLEAN',
16+
'BETWEEN',
17+
'BY',
18+
'CASE',
19+
'CAST',
20+
'CROSS',
21+
'COLUMNS',
22+
'DATETIME',
23+
'DELETE',
24+
'DESC',
25+
'DESCRIBE',
26+
'DISTINCT',
27+
'DOUBLE',
28+
'ELSE',
29+
'EXISTS',
30+
'FALSE',
31+
'FLOAT',
32+
'FIRST',
33+
'FROM',
34+
'GROUP',
35+
'HAVING',
36+
'IN',
37+
'INNER',
38+
'INT',
39+
'INTEGER',
40+
'IS',
41+
'JOIN',
42+
'LAST',
43+
'LEFT',
44+
'LIKE',
45+
'LIMIT',
46+
'LONG',
47+
'MATCH',
48+
'NATURAL',
49+
'NOT',
50+
'NULL',
51+
'NULLS',
52+
'ON',
53+
'OR',
54+
'ORDER',
55+
'OUTER',
56+
'OVER',
57+
'PARTITION',
58+
'REGEXP',
59+
'RIGHT',
60+
'SELECT',
61+
'SHOW',
62+
'STRING',
63+
'THEN',
64+
'TRUE',
65+
'UNION',
66+
'USING',
67+
'WHEN',
68+
'WHERE',
69+
'EXCEPT',
70+
];
71+
72+
const functions = [
73+
'AVG',
74+
'COUNT',
75+
'MAX',
76+
'MIN',
77+
'SUM',
78+
'VAR_POP',
79+
'VAR_SAMP',
80+
'VARIANCE',
81+
'STD',
82+
'STDDEV',
83+
'STDDEV_POP',
84+
'STDDEV_SAMP',
85+
'SUBSTRING',
86+
'TRIM',
87+
];
88+
89+
const operators = [
90+
'=',
91+
'>',
92+
'<',
93+
'!',
94+
'~',
95+
'\\|',
96+
'&',
97+
'\\^',
98+
'\\*',
99+
'/',
100+
'%',
101+
'\\+',
102+
'-',
103+
'DIV',
104+
'MOD',
105+
];
106+
107+
const brackets = [
108+
{ open: '(', close: ')', token: 'delimiter.parenthesis' },
109+
{ open: '[', close: ']', token: 'delimiter.square' },
110+
];
111+
112+
export const lexerRules = {
113+
defaultToken: 'invalid',
114+
ignoreCase: true,
115+
tokenPostfix: '',
116+
keywords,
117+
functions,
118+
operators,
119+
brackets,
120+
tokenizer: {
121+
root: [
122+
[
123+
/[a-zA-Z_$][a-zA-Z0-9_$]*/,
124+
{
125+
cases: {
126+
'@keywords': 'keyword',
127+
'@functions': 'function',
128+
'@default': 'identifier',
129+
},
130+
},
131+
],
132+
{ include: '@whitespace' },
133+
[/[()]/, '@brackets'],
134+
[new RegExp(operators.join('|')), 'operator'],
135+
[/[0-9]+(\.[0-9]+)?/, 'number'],
136+
[/'([^'\\]|\\.)*$/, 'string.invalid'], // non-terminated string
137+
[/'/, 'string', '@string'],
138+
[/"/, 'string', '@string'],
139+
],
140+
whitespace: [
141+
[/[ \t\r\n]+/, 'white'],
142+
[/\/\*/, 'comment', '@comment'],
143+
[/--.*$/, 'comment'],
144+
],
145+
string: [
146+
[/[^'\\]+/, 'string'],
147+
[/\\./, 'string.escape'],
148+
[/'/, 'string', '@pop'],
149+
[/"/, 'string', '@pop'],
150+
],
151+
comment: [
152+
[/[^/*]+/, 'comment'],
153+
[/\*\//, 'comment', '@pop'],
154+
[/[\/*]/, 'comment'],
155+
],
156+
},
157+
} as monaco.languages.IMonarchLanguage;

‎packages/osd-optimizer/src/worker/webpack.config.ts‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,16 @@ export function getWebpackConfig(bundle: Bundle, bundleRefs: BundleRefs, worker:
256256
loader: 'raw-loader',
257257
},
258258
},
259+
{
260+
test: /\.cjs$/,
261+
include: /node_modules/,
262+
use: {
263+
loader: 'babel-loader',
264+
options: {
265+
presets: [BABEL_PRESET_PATH],
266+
},
267+
},
268+
},
259269
],
260270
},
261271

0 commit comments

Comments
 (0)
⚡