[Swift] Huge performance regression

[cltnschlosser]

Going from 2.17.4 to 2.18.3. Codegen time ballooned from 40s to 240s.

codegen:generate --includes='./**/*.graphql' --localSchemaFile="schema.json" --passthroughCustomScalars --target=swift Sources/Core/Generated/

How to reproduce the issue:

To help understand the size of this project:
87 .graphql files
schema is 136,639 lines
Sources/Core/Generated/ is 2.2MB

Versions

2.18.3

cc @designatednerd

[cltnschlosser]

If you can give me any tips on how to debug a typescript application, I can get more information. Can you use instruments or something like it?

[designatednerd]

I know @lilyball had some concerns there might be some performance issues with stuff which was recently changed - Lily do you have specific thoughts around where that might be?

Asking the tooling team for suggestions on debugging, I'm definitely in dog-in-a-lab-coat-and-goggles territory with Typescript, unfortunately.

[lilyball]

@designatednerd For my recent SwiftSource changes, I ran apollo client:codegen on my production project with both versions (a version from before the change and the version published after) and saw no meaningful performance difference.

I have noticed that it's taking a lot longer than I expected in general though. I may have to do some version bisecting to see when the performance regression happened.

If possible, we should also look into adding some benchmarks to the project. I don't know offhand how to do that though.

[lilyball]

Oh wow, I must have measured incorrectly before. 2.18.0 take 50 seconds on my project. 2.17.4 takes 7. That's bad.

If this is due to the SwiftSource changes, then a) ouch, that's a really bad performance hit for wrapping strings in objects, and b) I have some thoughts about an alternative approach that uses type punning instead of constructing actual objects at runtime.

[lilyball]

This would be significantly easier to test if I could actually get packages/apollo/bin/run client:codegen to work on my machine. I don't know why that fails for me.

[lilyball]

Oh crud, my type punning approach isn't compatible with letting you interpolate these values with the swift template literal tag because that requires runtime representation, which is a pretty big usability issue.

I wonder if I could hack this by prefixing these strings with some magic characters and then stripping that out when interpolating them or printing them? That's nasty though.

[designatednerd]

Yeah magic characters seems like it could be extremely prone to error as well.

I plan to add performance tests on the Swift codegen to prevent stuff like this in the future, but my lack of typescript knowledge is really hurting in terms of figuring out why a regression like this would happen

[lilyball]

Magic characters would be coupled with the type punning approach, and would exist so the swift template tag could then detect that it's a SwiftSource and avoid escaping it. This would of course be a problem if we pass this to any other code that takes an any and relies on runtime type-checking because anything else that looks for a string would then misinterpret it. If we're ok with "passing it as any is unsafe" then this is doable, though awkward.

I do wonder if instead we could just reduce the number of independent SwiftSource instances we create?

I also haven't yet checked if there's anything else in the 2.18.0 release that could be impacting performance.

[lilyball]

I wish I knew how to do actual performance profiling on JavaScript.

[cltnschlosser]

@lilyball It's not the easiest to use, but you can do this: https://nodejs.org/en/docs/guides/simple-profiling/ or run with node --inspect and use chrome chrome://inspect, to use the profiler in chrome.

node packages/apollo/bin/run client:codegen should work, just add the flags you want

[lilyball]

Turns out I was right the first time; SwiftSource was in 2.18.2, but the performance regression was in 2.18.0.

I tried the node --inspect approach but the Chrome profiler seems really hard to use. I think we're going to have to bisect the changes from 2.17.4 to 2.18.0 to see where the slowdown came from.

[designatednerd]

That's bizarre! Looking on master at the commits between those two releases doesn't show anything that jumps out at me as suspicious:

Here's the diff between those two tags - nothing seems to have been touched in the Swift package in that changeset. @JakeDawkins Any thoughts on what this could possibly be?

[lilyball]

Bisect says the bad commit was 205987d. That commit only touches apollo-language-server and vscode-apollo though, so I don't know what's going on. But it is reproducible; running that commit is slow, running the parent commit is fast.