feat: Add rawValues and rawFieldNames options for aggregation queries#10438
feat: Add rawValues and rawFieldNames options for aggregation queries#10438mtrezza merged 14 commits intoparse-community:alphafrom
rawValues and rawFieldNames options for aggregation queries#10438Conversation
…on for aggregation
… for positional alignment
|
🚀 Thanks for opening this pull request! We appreciate your effort in improving the project. Please let us know once your pull request is ready for review. Tip
Note Please respond to review comments from AI agents just like you would to comments from a human reviewer. Let the reviewer resolve their own comments, unless they have reviewed and accepted your commit, or agreed with your explanation for why the feedback was incorrect. Caution Pull requests must be written using an AI agent with human supervision. Pull requests written entirely by a human will likely be rejected, because of lower code quality, higher review effort and the higher risk of introducing bugs. Please note that AI review comments on this pull request alone do not satisfy this requirement. Our CI and AI review are safeguards, not development tools. If many issues are flagged, rethink your development approach. Invest more effort in planning and design rather than using review cycles to fix low-quality code. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughAdds per-query and server-level flags Changes
Sequence DiagramsequenceDiagram
participant Client as Client
participant Router as AggregateRouter
participant Controller as DatabaseController
participant Adapter as MongoStorageAdapter
participant Mongo as MongoDB
Client->>Router: POST /aggregate (body incl. rawValues/rawFieldNames)
Router->>Router: Extract options, apply req.config.query defaults
Router->>Controller: find(className, query, {..., rawValues, rawFieldNames})
Controller->>Adapter: aggregate(className, schema, pipeline, ..., comment, rawValues, rawFieldNames)
alt rawValues == true
Adapter->>Adapter: EJSON.deserialize(pipeline)
else
Adapter->>Adapter: Parse-style value conversions (Dates, pointers)
end
alt rawFieldNames == true
Adapter->>Adapter: Skip field-name/_id/_created_at remapping
else
Adapter->>Adapter: Apply Parse field mappings and key renames
end
Adapter->>Mongo: execute aggregation(pipeline)
Mongo-->>Adapter: aggregation results
alt rawValues == true
Adapter->>Adapter: EJSON.serialize(results)
else
Adapter->>Adapter: mongoObjectToParseObject(results)
end
Adapter-->>Controller: results
Controller-->>Router: results
alt rawValues OR rawFieldNames
Router->>Router: Skip removeHiddenProperties on results
else
Router->>Router: Remove hidden properties
end
Router-->>Client: JSON response
Estimated code review effort🎯 4 (Complex) | ⏱️ ~35 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 6 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (6 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Checkov (3.2.519)package.json2026-04-17 07:54:40,220 [MainThread ] [ERROR] Template file not found: package.json ... [truncated 2547 characters] ... [MainThread ] [WARNI] Secret scanning: could not process file package.json Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## alpha #10438 +/- ##
==========================================
+ Coverage 92.50% 92.51% +0.01%
==========================================
Files 192 192
Lines 16792 16820 +28
Branches 234 234
==========================================
+ Hits 15533 15561 +28
Misses 1236 1236
Partials 23 23 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/Options/docs.js (1)
158-162: Optional: add a brief README mention for aggregation raw options.Not required, but a short README pointer can improve discoverability for users who don’t inspect generated options docs.
Based on learnings: When reviewing Parse Server PRs that add new features, always check whether the feature is documented in the README.md file, though for new Parse Server options this is optional rather than required.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/Options/docs.js` around lines 158 - 162, Add a short README entry documenting the new QueryServerOptions fields: aggregationRawFieldNames and aggregationRawValues; mention their defaults (false), describe that they control default behavior for aggregation queries (field name rewriting vs native MongoDB names and EJSON/raw value coercion), and note that individual queries can still override via the rawFieldNames and rawValues options; place the pointer near other QueryServerOptions or options documentation in README.md for discoverability.src/Options/index.js (1)
394-397: Suggested PR title (Angular convention):feat(query): add raw aggregation defaults and per-request overridesThis title is changelog-friendly and highlights developer impact.
Based on learnings: For Parse Server PRs, always suggest an Angular commit convention PR title that would make a meaningful changelog entry for developers. Update the PR title suggestion on every commit. The format should be: type(scope): description.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/Options/index.js` around lines 394 - 397, Update the PR title suggestion logic to return an Angular-style commit message "type(scope): description" and ensure it is refreshed on every commit; specifically change the code that produces the suggested PR title (function or constant handling PR title generation, e.g., suggestPrTitle / PR_TITLE_SUGGESTION) so it outputs a value like "feat(query): add raw aggregation defaults and per-request overrides" and make sure any caching or single-run behavior is removed so the suggestion is regenerated on each commit.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@src/Options/docs.js`:
- Around line 158-162: Add a short README entry documenting the new
QueryServerOptions fields: aggregationRawFieldNames and aggregationRawValues;
mention their defaults (false), describe that they control default behavior for
aggregation queries (field name rewriting vs native MongoDB names and EJSON/raw
value coercion), and note that individual queries can still override via the
rawFieldNames and rawValues options; place the pointer near other
QueryServerOptions or options documentation in README.md for discoverability.
In `@src/Options/index.js`:
- Around line 394-397: Update the PR title suggestion logic to return an
Angular-style commit message "type(scope): description" and ensure it is
refreshed on every commit; specifically change the code that produces the
suggested PR title (function or constant handling PR title generation, e.g.,
suggestPrTitle / PR_TITLE_SUGGESTION) so it outputs a value like "feat(query):
add raw aggregation defaults and per-request overrides" and make sure any
caching or single-run behavior is removed so the suggestion is regenerated on
each commit.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 29bcadf1-ad47-4aa2-ac57-54552bcae70e
📒 Files selected for processing (10)
resources/buildConfigDefinitions.jsspec/ParseQuery.Aggregate.spec.jssrc/Adapters/Storage/Mongo/MongoStorageAdapter.jssrc/Adapters/Storage/StorageAdapter.jssrc/Controllers/DatabaseController.jssrc/Options/Definitions.jssrc/Options/docs.jssrc/Options/index.jssrc/RestQuery.jssrc/Routers/AggregateRouter.js
mtrezza
changed the title
rawValues and rawFieldNames options for aggregation queries
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
spec/ParseQuery.Aggregate.spec.js (2)
1718-1775: Consider adding a per-query override test forrawFieldNames.There is coverage for per-query
rawValues: falseoverriding the server default (Line 1734), but no analogous test verifying that per-queryrawFieldNames: falseoverrides a server-levelaggregationRawFieldNames: true. Given the router precedence logic insrc/Routers/AggregateRouter.jsapplies independently to each flag, an explicit test would round out the override matrix and catch regressions if the two options ever diverge in how precedence is handled.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@spec/ParseQuery.Aggregate.spec.js` around lines 1718 - 1775, Add a per-query override test that verifies rawFieldNames:false overrides a server-level aggregationRawFieldNames:true: reconfigureServer({ query: { aggregationRawValues: true, aggregationRawFieldNames: true } }); create and save a TestObject, build an aggregation pipeline that matches using raw field names (e.g. _id and _created_at with an EJSON date like in existing tests), call new Parse.Query('TestObject').aggregate(pipeline, { rawFieldNames: false, useMasterKey: true }) and assert the results length is 0 to confirm the per-query rawFieldNames flag takes precedence (mirror the pattern used in the rawValues per-query test).
525-542: Test name phrasing: "does NOT coerce" is ambiguous.The assertion here is that
{ __type: 'Date', iso }(Parse legacy encoding) is not interpreted underrawValues: true, which is the intended contract. The comment on Line 540 explains this well, but the test title could be clearer — something like'rawValues: true ignores Parse-style { __type: "Date", iso } encoding'would convey intent without implying that coercion is generally expected elsewhere.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@spec/ParseQuery.Aggregate.spec.js` around lines 525 - 542, Rename the ambiguous test title to clearly state intent: change the it_id(...) description from 'rawValues: true does NOT coerce Parse Date encoding `{ __type: "Date", iso }`' to something like 'rawValues: true ignores Parse-style { __type: "Date", iso } encoding' so the test for Parse.Query.aggregate (the block using variables pipeline, query, and results) clearly communicates that legacy Parse date encoding is intentionally ignored when rawValues: true is used.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@package-lock.json`:
- Line 20507: The Node engine constraint mismatch arises because parse@8.6.0
requires "node": ">=22.13.0" while Parse Server's engines (package.json)
currently permit ">=22.12.0"; pick one resolution and make the corresponding
change: either update Parse Server's engines field to "node": ">=22.13.0" (and
regenerate package-lock.json) to drop support for 22.12.x, or revert/replace the
parse dependency to a version that supports ">=22.12.0" (update package.json
dependency and regenerate package-lock.json). Ensure the chosen action is
applied consistently in package.json and package-lock.json and that parse@8.6.0
is the referenced package name when deciding which version to change.
---
Nitpick comments:
In `@spec/ParseQuery.Aggregate.spec.js`:
- Around line 1718-1775: Add a per-query override test that verifies
rawFieldNames:false overrides a server-level aggregationRawFieldNames:true:
reconfigureServer({ query: { aggregationRawValues: true,
aggregationRawFieldNames: true } }); create and save a TestObject, build an
aggregation pipeline that matches using raw field names (e.g. _id and
_created_at with an EJSON date like in existing tests), call new
Parse.Query('TestObject').aggregate(pipeline, { rawFieldNames: false,
useMasterKey: true }) and assert the results length is 0 to confirm the
per-query rawFieldNames flag takes precedence (mirror the pattern used in the
rawValues per-query test).
- Around line 525-542: Rename the ambiguous test title to clearly state intent:
change the it_id(...) description from 'rawValues: true does NOT coerce Parse
Date encoding `{ __type: "Date", iso }`' to something like 'rawValues: true
ignores Parse-style { __type: "Date", iso } encoding' so the test for
Parse.Query.aggregate (the block using variables pipeline, query, and results)
clearly communicates that legacy Parse date encoding is intentionally ignored
when rawValues: true is used.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c0b63951-b818-4ce4-8f1b-17c5e2115f9f
📒 Files selected for processing (3)
package-lock.jsonpackage.jsonspec/ParseQuery.Aggregate.spec.js
✅ Files skipped from review due to trivial changes (1)
- package.json
# [9.9.0-alpha.1](9.8.1-alpha.1...9.9.0-alpha.1) (2026-04-17) ### Features * Add `rawValues` and `rawFieldNames` options for aggregation queries ([#10438](#10438)) ([f26700e](f26700e))
|
🎉 This change has been released in version 9.9.0-alpha.1 |
2 participants
Issue
Closes #10426
Closes #9116
Closes #7868
Closes #6068
Closes #6333
Closes #6343
Closes #7720
Approach
Adds two opt-in options for aggregation queries that bypass the legacy schema-based heuristics:
rawValues— Disables schema-based date and pointer value coercion. Instead, the pipeline is deserialized using MongoDB Extended JSON (EJSON), so typed values like{ $date: '...' },{ $oid: '...' },{ $numberDecimal: '...' }are converted to their BSON types. On output, BSON types are serialized back to EJSON markers. Works at any nesting depth, in any pipeline stage.rawFieldNames— Disables automatic field name rewriting (createdAt→_created_at,objectId→_id, pointer →_p_*). Users write native MongoDB field names directly. On output, native field names are preserved as-is, andmongoObjectToParseObjectis bypassed — preventing the schema-based result modification and crash described in MongoDB aggregation query results modified #7868.Both options are available per-query:
And as server-level defaults via the new
queryoption namespace:Per-query options override server-level defaults. Both default to
false, preserving current behavior.Why
The current aggregation pipeline applies value transformations (date coercion, pointer prefixing) and field name rewriting based on schema field-type lookups. This has several structural problems:
_created_atare not inschema.fields, so date comparisons against them silently fail (Aggregation query JS Date values not converted to BSON when using internal field names like_created_at#10426,new Date()in aggregation node.js #9116).$projectstages are not in the schema, so date values in subsequent$matchstages are not converted (Aggregate match doesn't work with custom date field #6333, aggregate pipeline projected date field can not be recognized properly #6343).$grouppipelines break because field references (like$_id) are transformed against the original schema even when they refer to a previous group's output (Multiple group stage issue in aggregation pipeline #6068).$lookupafter$groupwith pointer fields and date comparisons fails due to schema-based transformations (parse-server aggregate function returns empty object #7720).mongoObjectToParseObjectwhich renames keys, coerces pointer values, and can crash the server when aggregation output doesn't match schema expectations (MongoDB aggregation query results modified #7868).Rather than extending the heuristic with more special cases, this PR introduces explicit opt-in options that use MongoDB's standard EJSON format for typed values. Since aggregation pipelines are inherently MongoDB-specific (the
StorageAdapter.aggregateinterface is shaped around MongoDB semantics), using MongoDB's native type conventions is the natural fit.Tasks
Add changes to documentation (guides, repository pages, code comments)Add security checkAdd new Parse Error codes to Parse JS SDK