Skip to content

Commit 4d32cdb

Browse files
trevor-scheerabernix
trevor-scheer
authored and
abernix
committed
Update operation normalization to deterministically sort fragments.
This includes the work from #1027, #1115 and #1118, which first surfaced in an `apollo@next` CLI version which was released in order to provide a migration path for paying customers who utilize the Apollo Operation Registry through the CLI's `apollo client:push` features. Those customers were notified and advised to either pin their `apollo` version prior to this being released, so the hope is that we'll be able to released this under the `apollo@2` cover without incurring breaking changes on anyone else. For more information on the operation registry, see: https://www.apollographql.com/docs/platform/operation-registry.html And if you encounter any problems, please contact our customer support via Intercom from within your Engine UI. --- The summary of the relevant commit messages is below: 1) Remove duplication from client:push and client:extract. 2) Create a test to verify upcoming changes for this PR. Cutover to apollo-graphql * Add apollo-graphql as a dependency (and project reference) * Remove apollo-engine-reporting as a dependency * Update sortAST to normalize order of fragments w.r.t operations * Tests are failing expectedly at this point Centralize operation hashing function These two operations will likely be used in tandem, and we want this to be consistent across consumers. Incorporate rename suggestions Update version for extracted manifest output. Use empty string and add comment about unused metadata field. Revert changes to defaultEngineReportingSignature. Apply changes to new function, defaultOperationRegistrySignature. This new function is the effective interim fix, and the current existing function is now left alone. Pass operation name along to the operation registry signature function Leverage updated registerOperations API Now that registerOperations supports version as an argument to the mutation, we can leverage this for v2 oeprations manifest work.
1 parent 1938030 commit 4d32cdb

12 files changed

Lines changed: 3274 additions & 692 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,11 @@
22

33
## Upcoming
44

5+
## `apollo`
6+
7+
- `apollo`
8+
- Update operation normalization technique to deterministically order fragments within operations. This update affects those users of the [operation registry](https://www.apollographql.com/docs/platform/operation-registry.html) feature of the Apollo Platform. Anyone using the operation registry should re-register their operations with this new version of the `apollo` CLI via the `apollo client:push` command. Once all client operations are re-registered, the `apollo-server-plugin-operation-manifest` plugin within Apollo Server (which reads the manifest published with `apollo client:push`) should be updated to `0.1.0-alpha.1`. [#1158](https://github.com/apollographql/apollo-tooling/pull/1158)
9+
510
## `apollo-language-server`
611

712
- apollo-language-server

‎package-lock.json‎

Lines changed: 94 additions & 539 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/apollo-language-server/src/engine/operations/registerOperations.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,13 @@ export const REGISTER_OPERATIONS = gql`
55
$id: ID!
66
$clientIdentity: RegisteredClientIdentityInput!
77
$operations: [RegisteredOperationInput!]!
8+
$manifestVersion: Int!
89
) {
910
service(id: $id) {
1011
registerOperations(
1112
clientIdentity: $clientIdentity
1213
operations: $operations
14+
manifestVersion: $manifestVersion
1315
)
1416
}
1517
}

‎packages/apollo-language-server/src/graphqlTypes.ts‎

Lines changed: 1 addition & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ export interface RegisterOperationsVariables {
114114
id: string;
115115
clientIdentity: RegisteredClientIdentityInput;
116116
operations: RegisteredOperationInput[];
117+
manifestVersion: number;
117118
}
118119

119120
/* tslint:disable */
@@ -153,9 +154,6 @@ export interface SchemaTagInfo_service {
153154
}
154155

155156
export interface SchemaTagInfo {
156-
/**
157-
* Service by ID
158-
*/
159157
service: SchemaTagInfo_service | null;
160158
}
161159

@@ -194,13 +192,7 @@ export interface SchemaTagsAndFieldStats_service_stats_fieldStats_metrics {
194192

195193
export interface SchemaTagsAndFieldStats_service_stats_fieldStats {
196194
__typename: "ServiceFieldStatsRecord";
197-
/**
198-
* Dimensions of ServiceFieldStats that can be grouped by.
199-
*/
200195
groupBy: SchemaTagsAndFieldStats_service_stats_fieldStats_groupBy;
201-
/**
202-
* Metrics of ServiceFieldStats that can be aggregated over.
203-
*/
204196
metrics: SchemaTagsAndFieldStats_service_stats_fieldStats_metrics;
205197
}
206198

@@ -219,9 +211,6 @@ export interface SchemaTagsAndFieldStats_service {
219211
}
220212

221213
export interface SchemaTagsAndFieldStats {
222-
/**
223-
* Service by ID
224-
*/
225214
service: SchemaTagsAndFieldStats_service | null;
226215
}
227216

@@ -780,9 +769,6 @@ export interface GetSchemaByTag_service_Service {
780769
export type GetSchemaByTag_service = GetSchemaByTag_service_User | GetSchemaByTag_service_Service;
781770

782771
export interface GetSchemaByTag {
783-
/**
784-
* Current identity, null if not authenticated
785-
*/
786772
service: GetSchemaByTag_service | null;
787773
}
788774

‎packages/apollo/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,8 @@
4545
"apollo-codegen-scala": "file:../apollo-codegen-scala",
4646
"apollo-codegen-swift": "file:../apollo-codegen-swift",
4747
"apollo-codegen-typescript": "file:../apollo-codegen-typescript",
48-
"apollo-engine-reporting": "0.2.2",
4948
"apollo-env": "file:../apollo-env",
49+
"apollo-graphql": "file:../apollo-graphql",
5050
"apollo-language-server": "file:../apollo-language-server",
5151
"chalk": "2.4.2",
5252
"cli-ux": "4.9.3",

‎packages/apollo/src/commands/client/extract.ts‎

Lines changed: 29 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,10 @@
1-
import { createHash } from "crypto";
21
import { writeFileSync } from "fs";
3-
import {
4-
printWithReducedWhitespace,
5-
sortAST,
6-
defaultSignature as engineDefaultSignature
7-
} from "apollo-engine-reporting";
8-
import { DocumentNode } from "graphql";
9-
102
import { ClientCommand } from "../../Command";
11-
import { hideCertainLiterals } from "./push";
12-
13-
// XXX this is duplicated code
14-
const manifestOperationHash = (str: string): string =>
15-
createHash("sha256")
16-
.update(str)
17-
.digest("hex");
18-
19-
const engineSignature = (_TODO_operationAST: DocumentNode): string => {
20-
// TODO. We don't currently have access to the operation name since it's
21-
// currently omitted by the `apollo-codegen-core` package logic.
22-
return engineDefaultSignature(_TODO_operationAST, "TODO");
23-
};
3+
import {
4+
getOperationManifestFromProject,
5+
ManifestEntry
6+
} from "../../utils/getOperationManifestFromProject";
7+
import { ClientIdentity } from "apollo-language-server";
248

259
export default class ClientExtract extends ClientCommand {
2610
static description = "Extract queries from a client";
@@ -38,52 +22,31 @@ export default class ClientExtract extends ClientCommand {
3822
];
3923

4024
async run() {
41-
const { clientIdentity, operations, filename }: any = await this.runTasks(
42-
({ flags, project, config, args }) => [
43-
{
44-
title: "Extracting operations from project",
45-
task: async ctx => {
46-
const operations = Object.values(
47-
this.project.mergedOperationsAndFragmentsForService
48-
).map(operationAST => {
49-
// While this could include dropping unused definitions, they are
50-
// kept because the registered operations should mirror those in the
51-
// client bundle minus any PII which lives within string literals.
52-
const printed = printWithReducedWhitespace(
53-
sortAST(hideCertainLiterals(operationAST))
54-
);
55-
56-
return {
57-
signature: manifestOperationHash(printed),
58-
document: printed,
59-
metadata: {
60-
engineSignature: engineSignature(operationAST)
61-
}
62-
};
63-
});
64-
65-
ctx.operations = operations;
66-
ctx.clientIdentity = config.client;
67-
}
68-
},
69-
{
70-
title: "Outputing extracted queries",
71-
task: (ctx, task) => {
72-
const filename = args.output;
73-
task.title = "Outputing extracted queries to " + filename;
74-
ctx.filename = filename;
75-
writeFileSync(
76-
filename,
77-
JSON.stringify(
78-
{ version: 1, operations: ctx.operations },
79-
null,
80-
2
81-
)
82-
);
83-
}
25+
const { clientIdentity, operations, filename } = await this.runTasks<{
26+
clientIdentity: ClientIdentity;
27+
operations: ManifestEntry[];
28+
filename: string;
29+
}>(({ flags, project, config, args }) => [
30+
{
31+
title: "Extracting operations from project",
32+
task: async ctx => {
33+
ctx.operations = getOperationManifestFromProject(this.project);
34+
ctx.clientIdentity = config.client;
8435
}
85-
]
86-
);
36+
},
37+
{
38+
title: "Outputing extracted queries",
39+
task: (ctx, task) => {
40+
const filename = args.output;
41+
task.title = "Outputing extracted queries to " + filename;
42+
ctx.filename = filename;
43+
writeFileSync(
44+
filename,
45+
JSON.stringify({ version: 2, operations: ctx.operations }, null, 2)
46+
);
47+
}
48+
}
49+
]);
8750

8851
this.log(
8952
`Successfully wrote ${operations.length} operations from the ${

‎packages/apollo/src/commands/client/push.ts‎

Lines changed: 16 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -1,51 +1,9 @@
1-
import { createHash } from "crypto";
2-
import {
3-
printWithReducedWhitespace,
4-
sortAST,
5-
defaultSignature as engineDefaultSignature
6-
} from "apollo-engine-reporting";
7-
8-
import {
9-
visit,
10-
DocumentNode,
11-
IntValueNode,
12-
FloatValueNode,
13-
StringValueNode
14-
} from "graphql";
15-
161
import { ClientCommand } from "../../Command";
17-
18-
const manifestOperationHash = (str: string): string =>
19-
createHash("sha256")
20-
.update(str)
21-
.digest("hex");
22-
23-
const engineSignature = (_TODO_operationAST: DocumentNode): string => {
24-
// TODO. We don't currently have access to the operation name since it's
25-
// currently omitted by the `apollo-codegen-core` package logic.
26-
return engineDefaultSignature(_TODO_operationAST, "TODO");
27-
};
28-
29-
// In the same spirit as the similarly named `hideLiterals` function from the
30-
// `apollo-engine-reporting/src/signature.ts` module, we'll do an AST visit
31-
// to redact literals. Developers are strongly encouraged to use the
32-
// `variables` aspect of the which would avoid these being explicitly
33-
// present in the operation manifest at all. The primary area of concern here
34-
// is to avoid sending in-lined literals which might contain sensitive
35-
// information (e.g. API keys, etc.).
36-
export function hideCertainLiterals(ast: DocumentNode): DocumentNode {
37-
return visit(ast, {
38-
IntValue(node: IntValueNode): IntValueNode {
39-
return { ...node, value: "0" };
40-
},
41-
FloatValue(node: FloatValueNode): FloatValueNode {
42-
return { ...node, value: "0" };
43-
},
44-
StringValue(node: StringValueNode): StringValueNode {
45-
return { ...node, value: "", block: false };
46-
}
47-
});
48-
}
2+
import {
3+
getOperationManifestFromProject,
4+
ManifestEntry
5+
} from "../../utils/getOperationManifestFromProject";
6+
import { ClientIdentity } from "apollo-language-server";
497

508
export default class ServicePush extends ClientCommand {
519
static description = "Push a service to Engine";
@@ -54,35 +12,21 @@ export default class ServicePush extends ClientCommand {
5412
};
5513

5614
async run() {
57-
const {
58-
clientIdentity,
59-
operations,
60-
serviceName
61-
}: any = await this.runTasks(({ flags, project, config }) => [
15+
const { clientIdentity, operations, serviceName } = await this.runTasks<{
16+
clientIdentity: ClientIdentity;
17+
operations: ManifestEntry[];
18+
serviceName: string;
19+
}>(({ flags, project, config }) => [
6220
{
6321
title: "Pushing client information to Engine",
6422
task: async ctx => {
6523
if (!config.name) {
6624
throw new Error("No service found to link to Engine");
6725
}
68-
const operations = Object.values(
69-
this.project.mergedOperationsAndFragmentsForService
70-
).map(operationAST => {
71-
// While this could include dropping unused definitions, they are
72-
// kept because the registered operations should mirror those in the
73-
// client bundle minus any PII which lives within string literals.
74-
const printed = printWithReducedWhitespace(
75-
sortAST(hideCertainLiterals(operationAST))
76-
);
7726

78-
return {
79-
signature: manifestOperationHash(printed),
80-
document: printed,
81-
metadata: {
82-
engineSignature: engineSignature(operationAST)
83-
}
84-
};
85-
});
27+
const operationManifest = getOperationManifestFromProject(
28+
this.project
29+
);
8630

8731
const { name, referenceID, version } = config.client!;
8832
if (!name) {
@@ -96,13 +40,14 @@ export default class ServicePush extends ClientCommand {
9640
version
9741
},
9842
id: config.name,
99-
operations
43+
operations: operationManifest,
44+
manifestVersion: 2
10045
};
10146

10247
await project.engine.registerOperations(variables);
10348

10449
// store data for logging
105-
ctx.operations = operations;
50+
ctx.operations = operationManifest;
10651
ctx.serviceName = variables.id;
10752
ctx.clientIdentity = variables.clientIdentity;
10853
}

‎packages/apollo/src/utils/__tests__/__snapshots__/getOperationManifestFromProject.test.ts.snap‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
// Jest Snapshot v1, https://goo.gl/fbAQLP
2+
3+
exports[`getOperationManifestFromProject builds an operation manifest 1`] = `
4+
Array [
5+
Object {
6+
"document": "query SchemaTagsAndFieldStats($id:ID!){service(id:$id){__typename schemaTags{__typename tag}stats(from:\\"\\",to:\\"\\"){__typename fieldStats{__typename groupBy{__typename field}metrics{__typename fieldHistogram{__typename durationMs(percentile:0)}}}}}}",
7+
"metadata": Object {
8+
"engineSignature": "",
9+
},
10+
"signature": "135e8314de0c2f23f4a2be87b20e59f30ecf2cbcdfad6676a46aad21d29cdb5b",
11+
},
12+
Object {
13+
"document": "mutation CheckSchema($frontend:String,$gitContext:GitContextInput,$historicParameters:HistoricQueryParameters,$id:ID!,$schema:IntrospectionSchemaInput!,$tag:String){service(id:$id){__typename checkSchema(baseSchemaTag:$tag,frontend:$frontend,gitContext:$gitContext,historicParameters:$historicParameters,proposedSchema:$schema){__typename diffToPrevious{__typename changes{__typename code description type}type validationConfig{__typename from queryCountThreshold queryCountThresholdPercentage to}}targetUrl}}}",
14+
"metadata": Object {
15+
"engineSignature": "",
16+
},
17+
"signature": "3fd18a0f0369d801024bb42268bc4d4d1f6edd84a412923c4678c026bc1bdc62",
18+
},
19+
Object {
20+
"document": "mutation RegisterOperations($clientIdentity:RegisteredClientIdentityInput!,$id:ID!,$operations:[RegisteredOperationInput!]!){service(id:$id){__typename registerOperations(clientIdentity:$clientIdentity,operations:$operations)}}",
21+
"metadata": Object {
22+
"engineSignature": "",
23+
},
24+
"signature": "1428ab44b0f2f4ea25bc28c08b5a7235cb96a2d535d1a15e0edd5f0695a4903f",
25+
},
26+
Object {
27+
"document": "mutation UploadSchema($gitContext:GitContextInput,$id:ID!,$schema:IntrospectionSchemaInput!,$tag:String!){service(id:$id){__typename uploadSchema(gitContext:$gitContext,schema:$schema,tag:$tag){__typename code message success tag{__typename schema{__typename hash}tag}}}}",
28+
"metadata": Object {
29+
"engineSignature": "",
30+
},
31+
"signature": "a2497bb8c32f97870dfc58823b1377f6339c3e5ba53010b2a6b1e5d8fa8b8634",
32+
},
33+
Object {
34+
"document": "mutation ValidateOperations($gitContext:GitContextInput,$id:ID!,$operations:[OperationDocumentInput!]!,$tag:String){service(id:$id){__typename validateOperations(gitContext:$gitContext,operations:$operations,tag:$tag){__typename validationResults{__typename code description operation{__typename name}type}}}}",
35+
"metadata": Object {
36+
"engineSignature": "",
37+
},
38+
"signature": "600e261efe1f25ee30c53edd0b2b83c7eb6691ef5ae1c8f460ace2d303ad053d",
39+
},
40+
Object {
41+
"document": "fragment IntrospectionFullType on IntrospectionType{__typename description enumValues(includeDeprecated:true){__typename depreactionReason description isDeprecated name}fields{__typename args{__typename...IntrospectionInputValue}deprecationReason description isDeprecated name type{__typename...IntrospectionTypeRef}}inputFields{__typename...IntrospectionInputValue}interfaces{__typename...IntrospectionTypeRef}kind name possibleTypes{__typename...IntrospectionTypeRef}}fragment IntrospectionInputValue on IntrospectionInputValue{__typename defaultValue description name type{__typename...IntrospectionTypeRef}}fragment IntrospectionTypeRef on IntrospectionType{__typename kind name ofType{__typename kind name ofType{__typename kind name ofType{__typename kind name ofType{__typename kind name ofType{__typename kind name ofType{__typename kind name ofType{__typename kind name}}}}}}}}query GetSchemaByTag($tag:String!){service:me{__typename...on Service{schema(tag:$tag){__typename hash __schema:introspection{__typename directives{__typename args{__typename...IntrospectionInputValue}description locations name}mutationType{__typename name}queryType{__typename name}subscriptionType{__typename name}types{__typename...IntrospectionFullType}}}}}}",
42+
"metadata": Object {
43+
"engineSignature": "",
44+
},
45+
"signature": "f7f71dac7423a856fcc1b05a944e92247d0bba4beafd508410890242d4cf6d5f",
46+
},
47+
]
48+
`;

0 commit comments

Comments
 (0)
⚡