fix: introduce file cache to simplify fetching files including file mode#1280
fix: introduce file cache to simplify fetching files including file mode#1280
Conversation
|
Oh cool, so the approach you came up with doesn't use GraphQL, it just pulls the recursive logic for looking up files into a wrapper, and ensures that files are only fetched once. If I'm understanding correctly you only cache files as they're required, and the idea is that you would be able to get both their contents and mode without two requests? |
chingor13
changed the title
We lazy cache git tree data as we need to fetch files. We avoid the simple contents API because we can't get file mode data. We still need separate requests for the file metadata and the actual contents though. |
bcoe
left a comment
bcoe
left a comment
There was a problem hiding this comment.
I like that this encapsulates the increasingly ugly file contents logic into a helper class 👍
A couple thoughts:
- I'm curious to see how this implementation behaves on a paticularly large repository (could you test against yoshi-go perhaps?).
- I think it would be worthwhile having some basic smoke tests for the cache behavior -- I philisophically coming around to nock being used the same way you'd use an integration tests (don't have may nock tests, but it's good to have a couple for coverage).
Running against google-cloud-go seems to still work. That repo is not actually big enough to trigger the recursive file fetching. |
| )) | ||
| ) | ||
| .get( | ||
| '/repos/testOwner/testRepo/git/blobs/3c3629e647f5ddf82548912e337bea9826b434af' |
🤖 I have created a release *beep* *boop* --- ### [13.4.11](v13.4.10...v13.4.11) (2022-02-18) ### Bug Fixes * introduce file cache to simplify fetching files including file mode ([#1280](#1280)) ([d7280b7](d7280b7)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
2 participants
This implements a file cache used by the
GitHubinstance. All file fetches will go through this file cache. It uses the git tree API. First, it tries to use therecursive=trueoption to fetch the entire file tree.If we receive a non-truncated set, we will cache that response and looking all file blob SHAs from that cached tree.
If we receive a truncated set, we will use the tree API without
recursiveand traverse the tree structure until we find the desired node (caching each tree node along the way).By using the tree API, we can actually fetch the file mode of the fetched file.
Fixes #1090
Fixes #1027