Understanding blanking and verification

[trikti]

I would really like to understand how ShredOS works, especially the concepts of blanking and verification. I'm not very experienced so I apologize if some of my questions are basic or obvious. I just want to make sure I fully grasp how these processes work.

If I understand correctly, blanking simply means filling the entire disk with zeros.

1. So, if I don’t disable blanking and choose the "Fill With Zeros" method, does that mean the disk gets written with zeros twice?
2. Why blanking is enabled by default, is it necessary to have zeros at the end of the wipe? For example, right now I am using PRNG Stream without final blanking pass, is it a problem?

Additionally, I’d like to understand how verification works.
3) If the previous round wrote only 1s or only 0s, verification checks whether the disk contain the expected values (only 1s or 0s). But what happens if the previous round used a PRNG stream? In that case, does verification still make sense and what exactly does it check?
4) How important is verification?
5) Does verifying between passes matter, or is the verification after the final pass the only one that truly matters?
6) Does verification causes additional wear on the disk?
7) If I didn’t enable the last pass verification during the wipe, would running 'Verify Zeros' (or 'Verify Ones') afterward achieve the same result?
8) Would it be sufficient to disable verification and manually check the hex data myself to confirm that the wipe was successful?

Thank you

[PartialVolume]

If I understand correctly, blanking simply means filling the entire disk with zeros.

Yes

1. So, if I don’t disable blanking and choose the "Fill With Zeros" method, does that mean the disk gets written with zeros twice?

Yes

2. Why blanking is enabled by default, is it necessary to have zeros at the end of the wipe? For example, right now I am using PRNG Stream without final blanking pass, is it a problem?>

No, not a problem. Some users just like to see the disc full of zeros rather than random data as random data could be mistakenly seen as encrypted data while zeros is an empty disc when looked at in a hex editor.

Additionally, I’d like to understand how verification works.
3) If the previous round wrote only 1s or only 0s, verification checks whether the disk contain the expected values (only 1s or 0s).

Yes, that's correct.

But what happens if the previous round used a PRNG stream? In that case, does verification still make sense and what exactly does it check?

A PRNG stream is created from a single seed, if you know the seed you can reproduce the stream. That's how a prng verification works, nwipe knows the seed it uses to create the stream, therefore it can recreate the exact same prng stream for verification that it created for the wipe . Also the seed is randomly generated so is different for every wipe and for every disc being wiped

4. How important is verification?

Personally, I think it's very important because a PRNG and verification really does a good test of the disc and I like to makesure the disc is working correctly. I've seen discs that will pass a zeros wipe but fail a ones or prng wipe because of a stuck bit at 1 in the drives electronics, so personally I prefer a prng wipe as it provides me with confidence that the drive has no faults.

5. Does verifying between passes matter, or is the verification after the final pass the only one that truly matters?

Depends, what ever method you use I would make sure a verification occurs on a prng pass. So for me I typically would do a prng with blanking, verification on all passes, so in all the disc is written twice and read twice. But that's just me. Others may prefer other methods.

6. Does verification causes additional wear on the disk?

No, neither on a HDD or SDD for verification, however the wipe may be a different matter on a SSD, ATA secure erase may be preferable on a SSD, or at least keeping a traditional wipe on a SSD to a single PRNG pass plus ATA secure erase.

7. If I didn’t enable the last pass verification during the wipe, would running 'Verify Zeros' (or 'Verify Ones') afterward achieve the same result? 8) Would it be sufficient to disable verification and manually check the hex data myself to confirm that the wipe was successful?

Yes, in fact this should be the method you use if wiping very small drives. If a drive, typically an old USB memory stick is smaller than your CPU RAM you should always verify as a separate operation having removed the USB device and plugged it back in. So if your CPU RAM is 16GB any USB stick smaller than that should be verified with a zeros or ones as a separate operation. The reason being is that nwipe uses cached I/O so if you verify a disc that's smaller than your CPU RAM nwipe will actually verify the cache not read the disc. If the disc is larger than your RAM then the disc gets read. This will not be necessary when nwipe gets switched to Direct I/O when it controls buffering rather the kernel.

[trikti]

Thank you so much for the great answers, they give me confidence that what I'm doing is correct.

1. Regarding your last answer, is it possible that even with a driver larger than CPU RAM, nwipe still verifies the cache for part of it? If so, would it be more secure to always verify as a separate operation?

I also have other questions, which are not related to this topic, I hope it is okay to ask them here (I just created a GitHub account today, so I don't know the rules).

I have read another issue post, in which you mentioned that to erase an SSD you would do an ATA Secure Erase using hdparm, followed by a verify ones or verify zeros method.
2) Is this still your preferred method to wipe an SSD?
3) What is instead your preferred method to wipe an NVME?
4) Since now ShredOS can do ATA secure erase, can I trust it as much as hdparm?
5) Does encrypting an SSD or NVME, before or after the wiping, serve any purpose? I have seen many sources mention it but I don't understand whether the encryption is done already by the ATA Secure Erase or if I should do encryption as a separate operation using a tool like VeraCrypt.

Thank you in advance for the help :)

[PartialVolume]

@trikti Sorry, I must have missed your last post back in March, I'll close this issue but if you still would like me to answer those questions let me know.

[trikti]

@trikti Sorry, I must have missed your last post back in March, I'll close this issue but if you still would like me to answer those questions let me know.

Thank your for the reply, I am still interested to know what it is at this point in time your preferred method to wipe NVMEs and SSDs.

[saelic]

I have another question - not sure if it was fully covered above.

DBAN used Verification to actually fill the disk with zeros as a last pass - this fact is described in the "Verify Last Pass" description to this day:
"Check whether the device is actually empty after the last pass fills the device with zeros" - description didn't change since DBAN after all.

If that's not changed, in the default scenario we have a situation, where after the erasure (set in "Method") is completed, the disk is filled with zeros twice:

1. First time, because of verification - verification filled disk with zeros in DBAN.
2. Second time, because of "Rounds plus blanking pass".

I'm not sure if that's the case in ShredOS, because I'm just starting migration from DBAN, but will be very happy to hear from the author about this.

---

Also I don't understand, why in the "Method" we have "Verify Zeros" and "Verify Ones" - that should be placed somewhere else, as those variants do not erase data and are used just for verification. It is not a good idea to mix such things, especially in security software.

[PartialVolume]

I can't comment on DBAN's functionality as regards verification but this is how nwipe/ShredOS verification works,

Verification in Nwipe/ShredOS means to read the entire disc, selecting verification doesn't automatically select any sort of write or do any write. Verification is only a read function and is optionally applied to every write pass or the last write pass. A verification pass could be verifying zeros, ones, random data or some unique pattern as defined in the chosen method.

Here's some examples showing how many read/write passes are completed with different wipe options.

Method = Ones
Verification = Last pass
Blanking = OFF

Pass 1 Write zeros (0x00)
Pass 2 Read (verify) zeros written in pass 1

So the entire disc is written once and read once. (Two passes in total)

Method = PRNG
Verification = Last pass
Blanking = ON

Pass 1 Write a random stream of data using the selected PRNG generator
Pass 2 Write zeros (0x00) This is the blanking pass. 
Pass 3 Read (verify) zeros that were written in pass 2

So the entire disc is written twice (prng+zeros) and read once (Three passes in total)

Method = PRNG
Verification = ALL
Blanking = ON

Pass 1 Write a random stream of data using the selected PRNG generator
Pass 2 Read (verify) PRNG data that was written in pass 1
Pass 3 Write zeros (0x00) This is the blanking pass. 
Pass 4 Read (verify) zeros that were written in pass 3

So the entire disc is written twice (prng+zeros) and read twice (Four passes in total)

Also I don't understand, why in the "Method" we have "Verify Zeros" and "Verify Ones" - that should be placed somewhere else, as those variants do not erase data and are used just for verification. It is not a good idea to mix such things, especially in security software.

Verify ones and verify zeros are important methods. They may not write data only verify data, but these are important and useful in security software. Sometimes users may want to confirm that a drive that was zeroed with other erasure software want to confirm that the disc was indeed fully wiped with zeros. Nwipe/ShredOS can independently verify this as you can select verification without having to erase anything first. If a user selected verify zeros on a disc that had not been wiped it would show many verification errors and report on the PDF that the disc was not erased.

[PartialVolume]

Also if you switch blanking on, this writes zeros (0x00) only. Nothing else. Why have a blanking pass? Well sometimes when wiping discs users may prefer to see a disc full of zeros on erasure rather than data patterns or random data, which can look like encrypted data. It really depends on personal preference or local policy.

[PartialVolume]

So in summary, unlike DBAN nwipe/ShredOS separates blanking and verification. Blanking is optional so you can just do a zeros write and verify. Verification in ShredOS doesn't force a final zeros blanking pass. Verify last will verify whatever happens to be the last pass in the method. If you switch blanking on which will be a zeros pass, then verify last will verify the zeros blanking pass.