ZE FileSystemProvider `stat` function throws the `FileNotFound` error for files that exist

[hohy]

Describe the bug

When attempting to retrieve the stats of a dataset member that is not listed in the Zowe Explorer (ZE) dataset tree, the FileNotFound error is returned, even though the file exists.
However, if the dataset is added to the tree and its members are listed, the stats are correctly retrieved.

To Reproduce

I'm using following function that I execute as a VSCode command to call the stat function of the ZE FS provider.

  public async testFSProviderStat() {
    const treeFileUri = vscode.Uri.parse(
      "zowe-ds:/zosmf/WORK.ASMMAC/TEST",
    );
    try {
      const treeFileStats = await vscode.workspace.fs.stat(treeFileUri);
      debug(
        `File stats ${treeFileUri.toString()} ${JSON.stringify(treeFileStats)}`,
      );
    } catch (err) {
      debug(
        `File stats error ${treeFileUri.toString()} ${JSON.stringify(err)}`,
      );
    }

    const notInTreeFileUri = vscode.Uri.parse("zowe-ds:/zosmf/SYS1.MACLIB/ABEND");
    try {
      const notInTreeStats = await vscode.workspace.fs.stat(notInTreeFileUri);
      debug(
        `File stats ${notInTreeFileUri.toString()} ${JSON.stringify(notInTreeStats)}`,
      );
    } catch (err) {
      debug(
        `File stats error ${notInTreeFileUri.toString()} ${JSON.stringify(err)}`,
      );
    }
  }

1. Open VSCode with empty ZE dataset tree
2. Run the sample code that executes vscode.workspace.fs.stat function
3. FileNotFound is thrown for both dataset members
4. Add the WORK.ASMMAC dataset to the ZE tree and list its members
5. Run the test command again
6. It returns stats for the WORK.ASMMAC/TEST member, but still throws FileNotFound error for SYS1.MACLIB/ABEND
7. Add SYS1.MACLIB to the ZE Tree
8. Run the command again
9. Stats for both files are returned correctly.

Expected behavior

The stats for a dataset member should be retrievable even if the parent dataset is not listed in the ZE dataset tree. The FileNotFound error should not occur when querying existing files.

Screenshots

Desktop (please complete the following information):

• OS: MacOS
• Zowe Explorer Version: 3.1.2
• (Optional) Zowe CLI Version:
• (Optional) Are you using Secure Credential Store?

Additional context

[github-actions]

Thank you for creating a bug report.
We will investigate the bug and evaluate its impact on the product.
If you haven't already, please ensure you have provided steps to reproduce the bug and as much context as possible.

[JWaters02]

Since a vscode URI is an identifier for an object that exists within vscode, if you try to query on a URI that does not exist (e.g. on a data set that exists on the mainframe, but not pulled down into vscode by searching), you will get the error that the file is not found.

To get information about your file, the way that I'd do it (there is probably a better way that one of the other zowe devs can tell) is to get the session from the zowe explorer API and then do a call on zowe-files-sdk to get the attributes for the data set member of your choice.

[hohy]

I don't think a vscode URI is only for files already opened within VSCode, but for all files that can potentially be opened by VSCode. And for example the readFile function from ZE FSProvider is able to read a member of a dataset that is not already opened or listed in the tree. I would expect the stat function to behave in the same way.

    const treeFileUri = vscode.Uri.parse(
      "zowe-ds:/zosmf/WORK.ASMMAC/TEST",
    );
    try {
      const treeFileStats = await vscode.workspace.fs.stat(treeFileUri); // THIS FAILS
      debug(
        `File stats ${treeFileUri.toString()} ${JSON.stringify(treeFileStats)}`,
      );
    } catch (err) {
      debug(
        `File stats error ${treeFileUri.toString()} ${JSON.stringify(err)}`,
      );
    }
    const treeFileContent = await vscode.workspace.fs.readFile(treeFileUri); // THIS SUCCEEDS
    debug(
      `File content ${treeFileUri.toString()} ${Buffer.from(treeFileContent).toString()}`,
    );

[JWaters02]

Ah I see, my apologies. I was wondering how it was possible, but I see now that ZE extends vscode.FileSystemProvider to lookup the files on the mainframe if they don't exist in vscode yet. Ignore my reply then 😅

[traeok]

@hohy This is by design and you need to add the ?fetch=true query parameter to the URI to fetch the resource from the remote file system. See the FileSystemProvider page for more info, specifically this section for fetching resources on-demand.

This allows for explicit separation between "getting the stats of a resource that's already cached" and "fetching a resource from the remote file system." Without this opt-in query parameter, every stat call would make a remote request, which is inefficient and would cause slowdowns as VS Code calls stat frequently for resources with a matching URI in the tree view.

[slavek-kucera]

@traeok If this is working "as designed" then it is simply following a wrong design.

The whole idea of the FileSystemProvider is to offer extensions a common API to access filesystem-like resources. The idea to introduce a "secret handshake" that tells the implementation that the request being made is to be actually taken seriously goes against the whole idea. Every extension that wants to use the FSP would need to contain a database of these secret handshakes for every single FSP implementation to use the "common API" correctly.

Based on your comment, there is already a cache in place to reduce the number of remote requests so I fail to see the problem.

@VitGottwald

[traeok]

@traeok If this is working "as designed" then it is simply following a wrong design.

The whole idea of the FileSystemProvider is to offer extensions a common API to access filesystem-like resources. The idea to introduce a "secret handshake" that tells the implementation that the request being made is to be actually taken seriously goes against the whole idea. Every extension that wants to use the FSP would need to contain a database of these secret handshakes for every single FSP implementation to use the "common API" correctly.

Based on your comment, there is already a cache in place to reduce the number of remote requests so I fail to see the problem.

@VitGottwald

I understand your concern of not fetching by default @slavek-kucera, and I'm sorry if this has introduced any complexities in its usage. Ideally, we could fetch by default if a stat API was exposed from z/OSMF. However, given z/OSMF API's current endpoints, this isn't possible, so if we fetched by default, every stat function call would introduce a new list request. If you list data sets in Zowe Explorer and take a look at the output channel, you'll quickly notice why fetching by default through the FSP can introduce problems. When listing resources at scale, the number of requests balloon as VS Code calls stat for every item with a resourceUri in the tree. This would severely degrade the experience of listing items through Zowe Explorer.

I tested fetching by default during the implementation phase. When listing several items in the tree view, concurrent requests (coming from stat) hampered a z/OSMF instance, causing huge, consistent spikes in CPU activity on dev LPARs. Also, since we had already received the list of items matching a user-defined pattern, fetching by default indirectly caused a ton of redundant, subsequent requests through stat - even though we already had enough info to list items from the initial request.

Based on your comment, there is already a cache in place to reduce the number of remote requests so I fail to see the problem.

Sorry for any confusion inspired by my previous reply, but I was trying to imply that the fetch parameter reduces the number of remote requests and prioritizes use of the cached resource. In the example I mention earlier in this reply, the stat behavior can be reproduced and seen in ZE's output channel just by listing a pattern like SYS1.* under the data sets tree. We introduced more aggressive caching by only returning cached data if the URI doesn't explicitly request the resource to be fetched.

Is there a particular pain point about this design pattern that makes this workflow unusable for your use cases? In theory, you could always attach fetch=true to your URIs using a wrapper function, if your extension or profile type is not concerned about the increased network traffic and request scaling.

[slavek-kucera]

If I add SYS1.* to the tree as suggested - I get a stat call per matching dataset, when I expand one of the datasets, I get (presumably) a stat request per member (cached at this point). And that's it, I see no additional requests.

So what exactly is the problem here? If the entry is already known from the cache, you return the cached value, if it is not known you fetch it.
If there are multiple stat calls that are resolved by the same (in-progress) request you queue the stat call and reuse the results when the pending request eventually finishes.

[traeok]

If I add SYS1.* to the tree as suggested - I get a stat call per matching dataset, when I expand one of the datasets, I get (presumably) a stat request per member (cached at this point). And that's it, I see no additional requests.

From my testing, this was enough to cause the CPU spikes on the server when we had fetching by default. I can test again by removing the fetch checks and always fetching when listing data sets. Fetching by default also feels redundant to me in this scenario as we already have a list from the initial request we made.

We can always revisit this, but I'm still curious to know if this parameter is making any of your intended use cases impossible. I don't think this opt-in fetching goes against the design requirements of the FileSystemProvider, it's just a documented caveat of its current implementation. I'm always eager to improve it when possible, we just wanted to strike the balance of requests and caching to avoid mainframe slowdowns.

So what exactly is the problem here? If the entry is already known from the cache, you return the cached value, if it is not known you fetch it.

I like the idea of using the cached data first, but I think we avoided this because we still saw behavior in ZE that caused extra remote lookups. Also, you can't just rely on the cached data in stat because VS Code uses stat to detect whether a resource has been updated (which you need to check mtime on the remote system to verify).

Ideally the tree views would be a 1:1 representation of what's in the file system, but this is currently not the case - something we'd like to resolve as v3 continues / v4 development starts.

[slavek-kucera]

Based on a quick read through the code I don't see any synchronization/queueing before issuing the MF requests.
Multiple requests for the same resource result in independent backend requests even if there is one already in progress. I think that's the core issue that should be addressed here.