MongoDB aggregation query results modified

[mtrezza]

New Issue Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.
• I can reproduce the issue with the latest version of Parse Server.

Issue Description

The results of a MongoDB aggregation query are modified in an opaque way on the server side as they are parsed like normal query results.

Why this is bad:

• An aggregation query may return any valid JSON object, so it is non-sensical to do any Parse-specific validation or modification on the result data.
• This makes it difficult to develop aggregation pipelines (which can be quite complex and require 3rd party tools) because Parse Server changes the results. An aggregation result should be considered raw data and be identical to the result of the same query via mongosh MongoDB Compass or any other tool

What is changed in results:

• Keys that have the same name as a pointer field are renamed to pointers with prefix _p_ and their value is converted to Parse Object which can easily crash the server.
• _id is converted to objectId.

An easy fix to prevent the crash could be to change the following line and add a condition that the value must be a string in parse pointer syntax <string>$<string>, otherwise ignore that it looks like a pointer:

parse-server/src/Adapters/Storage/Mongo/MongoTransform.js

Line 1190 in 16b1b2a

if (key.indexOf('_p_') == 0) {

However, that would still modify the aggregation results which should be avoided.

Steps to reproduce

1. Create a class A that has a field of type pointer to class B with name fakepointer.
2. Create an aggregation query on class A that returns the following result:

{
  fakepointer: { 
    _id: 1
  }
}

2. Server looks up schema and finds that fakepointer should be a pointer, so it tries to convert it to a pointer and expects the value to be of type string <ClassName>$<ObjectId> but its of type object so server crashes at:

parse-server/src/Adapters/Storage/Mongo/MongoTransform.js

Line 1071 in 16b1b2a

const objData = pointerString.split('$');

Parse Server does not allow to store data like this, because a class field of type pointer is managed by Parse Server and its value cannot be manually set. But an aggregation query can return any valid JSON object.

Actual Outcome

Results are modified.

Expected Outcome

Server should not modify the results in any way.

Suggestion Solution

To easier manage this breaking change, introduce a new Parse Server MongoDB adapter option like rawAggregation, which means the aggregation pipeline won't be modified before sending it to the DB (e.g. Parse Server server does not allow the dollar sign before the aggregation stage name but native MongoDB syntax requires it) and the query results won't be modified after receiving them from the DB. Make the option default to false and add a deprecation warning to make it default to true in the future probably remove the option in the future completely.

Environment

Server

• Parse Server version: 5.1.1

Logs

n/a

[parse-github-assistant]

Thanks for opening this issue!

• 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

[dblythy]

What do you think of adding a parameter such as ({raw: true}) to the aggregate? We could also make sure the existing aggregate fails safely.

[mtrezza]

What do you think of adding a parameter such as ({raw: true}) to the aggregate?

Yes, a query based option could make it easier for a developer to migrate. If they have many aggregate queries in their code, they could migrate one-by-one.

We could also make sure the existing aggregate fails safely.

Also agreed, the query should fail without crashing the server in the example above. But it should not fail gracefully and return empty results for example. The developer needs to be aware that there's an issue.

[dblythy]

I'm trying to write an aggregate that makes the server crash, and I haven't been able to get it going. You might have more experience with aggregate.

it('can aggregate with raw', async () => {
    const pointer = new PointerObject();
    const obj = new TestObject({ fakepointer: pointer, name: 'hello' });
    await obj.save();
    const pipeline = [
      { match: { objectId: obj.id } },
      { project: { name: 1 } },
      {
        addFields : { fakepointer : {'_id': 1} }
      }
    ];
    const query = new Parse.Query(TestObject);
    const results = await query.aggregate(pipeline);
    console.log(results[0]); // { name: 'hello', fakepointer: { _id: 1 }, objectId: '9P7ktS91Xg' }
  });

[mtrezza]

Could you open a PR that would help me to play around to make it fail (even if the test passes for you).

[dblythy]

Sure, see #8172