Program truncation on saving

[DaleHurlin]

Hi

I have had a couple of reports that program source is being truncated when saving. The example I was given was a 5375 line program had 166 lines added. However when saving it truncated almost 4000 lines off the bottom of the program.

They are currently on VSCODE 1.100.1 and Zowe Explorer 3.2.0. and report that it is an intermittent issue, and it has not reoccurred since restarting VSCode.

I am on VSCODE 1.100.0 and Zowe Explorer 3.3.3. and it didn't happen for me.

I don't know if anyone else has experience with this issue and/or any recommendations?

[zFernand0]

After some discussion with both (CLI and ZE) squads, we concluded on a few different ways we can help in this situation.

Here are some action items we came up with:

• We could perform a client-side validation to check that no records go above the lrecl
  • Since this can impact performance, we might expose this in ZE via the command palette or as a setting.
• We could process the buffer (or stream) before uploading
  • This will require an enhancement in the SDK to ensure we are consisten in how we process the data
• We could leverage the X-IBM-Data-Type: text;wrap=true which will handle this for data sets with recfm: F | FB
  • Source: https://www.ibm.com/docs/en/zos/2.5.0?topic=interface-write-data-zos-data-set-member

That being said, the quickest way to address this is by using text;wrap=true when uploading.

---

Reference this issue to track the progress of this implementation:

• SDK: Expose a way to add word-wrapping when writing to a data set zowe-cli#2522

[JWaters02]

Hey @zFernand0 is this related to the original issue? I am slightly confused, as you seem to be talking about characters past the lrecl of the file being truncated on save, while OP mentions lines at the bottom of the file being truncated on save. My thought with the original issue was maybe the z/OSMF server had a moment of instability (we have moments where z/OSMF server runs out of memory with high usage and goes into breakdown mode for a few seconds or minutes) while the huge file was being saved and caused the save to cutoff prematurely and thus result in the text truncated off the bottom.

[JWaters02]

For others reading this, I asked for clarification in the ZE standup and this is how I understand it now. The reason why the user was getting lines cutoff the end of the file is because of a fatal flaw in the underlying z/OSMF API when handling lines past the lrecl.

So let's say in line 1376 of their program they have an extra character that is past the lrecl of their PDS. When we attempt to save this program through the API, it says "there's an error saving because a line is past the lrecl" while simultaneously saving all the lines up until that erroneous line. Meaning it saves up to line 1375, sees a character past lrecl, errors out, then stops. So that's why all the lines underneath got cut off at the end.

[FALLAI-Denis]

Hi,

Entering word wrapping (line breaks) when uploading a file will avoid losing information, but it won't produce a correct result based on what the user entered in the editor.

The most Zowe Explorer can do is validate line lengths against the LRECL declared for the file before saving (uploading) the file, and block the save (upload) if some lines are too long.

Moreover, some editors, such as IBM Z Open Editor, already perform a line length check at save time, even for local files that aren't uploaded to the mainframe.
We should see if Zowe Explorer can detect which editor/language mode is being used. If it's not the "plain text" editor/language mode, then delegate the responsibility to that editor to perform the line length check when saving, rather than repeating it itself.
If it's the "plain text" editor/language mode, (or a list of language modes?), then trigger a line length check.

Thanks.