Can't start 16-bit apps with OTVDM after uninstalling NTVDMx64 (As well as a few other issues)

[AngryDavid808]

Hello! I have a Windows 10 machine and I have recently installed NTVDMx64
It kept crashing with a different error code every time, so I uninstalled it through Control Panel.
I have set my machine to start 16-bit apps with otvdm, but now that I've uninstalled NTVDMx64, this error message appears instead:
"APP PATH HERE is a 16-bit application. You don't have permission to execute 16-bit applications. Contact your system Adminstrator."
Now, I am the adminstrator, but I still couldn't find my way out of this.
Also, for some reason, ldntvdm.dll is still in System32, and injected into winlogon.exe, and I honestly want to remove the injection and delete the file to get everything back to what it was before. How can I do this?
Thanks in advance.
Regards,
David

[leecher1337]

There was a bug in the uninstaller leaving the loader intact after uninstallation which was fixed in 7f12eff
To completely remove the loader, remove it from
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\AppInit_DLLs key. after a reboot, you should be able to delete the ldntvdm.dll in System32 directory.

The loader should be compatible with OTVDM and give it precedence on 16bit Windows applications,if it is installed.

Out of interest, which errors did you get? It runs flawlessly here, so I wonder what problems occured on your machine. Did they occur in ntvdm.exe directly?

[AngryDavid808]

There was a bug in the uninstaller leaving the loader intact after uninstallation which was fixed in 7f12eff
To completely remove the loader, remove it from
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows\AppInit_DLLs key. after a reboot, you should be able to delete the ldntvdm.dll in System32 directory.

The loader should be compatible with OTVDM and give it precedence on 16bit Windows applications,if it is installed.

Out of interest, which errors did you get? It runs flawlessly here, so I wonder what problems occured on your machine. Did they occur in ntvdm.exe directly?

Well, to be honest, I can't remember what these errors exactly say, but I do remember that they give a different error code every single time, and my 16-bit apps as well as my DOS apps never start. And yes, same goes for ntvdm.exe directly, I tried to directly execute it before reinstalling Windows because of a BsoD with stop code 0x0000c1, I believe that was the stop code. Since I now know how to get rid of the injection, I may actually reinstall NTVDMx64 and try to capture some of these error for you.
Thank you for responding btw.

[leecher1337]

Sounds like ERROR_BAD_EXE_FORMAT. May be a problem with the loader. Which Windows version and which file date of the loader? Weird, I never managed to produce a BSOD with the loader yet...

Btw., 16bit support was only added recently (3 days ago), but also the old loader left 16bit apps alone so that they can be handled by otvdm (the new loader checks for the presence of OTVDM and if OTVDM is installed, it takes OTVDM, otherwise it handles it with NTVDM to not interfere with it).

The loader generally logs to DebugView console, so its log output may shed some light on the issue. Generally, the injection is somewhat time critical when injecting into console processes and therefore can be a bit unrealiable sometimes (the conhost methos still turned out to be the most reliable compared to other methods), however the AppInit-injection into windowed processes should be OK.

[AngryDavid808]

Sounds like ERROR_BAD_EXE_FORMAT. May be a problem with the loader. Which Windows version and which file date of the loader? Weird, I never managed to produce a BSOD with the loader yet...

Btw., 16bit support was only added recently (3 days ago), but also the old loader left 16bit apps alone so that they can be handled by otvdm (the new loader checks for the presence of OTVDM and if OTVDM is installed, it takes OTVDM, otherwise it handles it with NTVDM to not interfere with it).

The loader generally logs to DebugView console, so its log output may shed some light on the issue. Generally, the injection is somewhat time critical when injecting into console processes and therefore can be a bit unrealiable sometimes (the conhost methos still turned out to be the most reliable compared to other methods), however the AppInit-injection into windowed processes should be OK.

Thank you for your response.

I was and still am on Windows 10 Build 1909. Before I reinstalled Windows, I actually had a user-made registry problem that caused Windows to reboot with no errors right whenever it finishes the boot process. I tried to do a System Restore, and this BsoD started appearing. Note that I had the same problem I have now when I did that System Restore. Because of this, I'm actually kind of scared to update Windows 10 to Build 2004. As for which version of the loader, I don't quite know, I fetched a pre-built copy from columbia.edu.

Also, where can I find the latest pre-built version of NTVDMx64? I actually want to give it another shot before I remove the injection.

I honestly can't read debug logs, but if you want some, please tell me how I can send them over for you to take a look at it.

[leecher1337]

My current test machine is Windows 10 Build 1903 (I didn't get any offers for update yet, not even to 1909... Don't know why, it says,I'm on most current release).

Also, where can I find the latest pre-built version of NTVDMx64? I actually want to give it another shot before I remove the injection.
Most recent loaders can be found in this repo: https://github.com/leecher1337/ntvdmx64/tree/master/ntvdmpatch/release/ldntvdm

As for ntvdm executable and libraries, as I cannot provide them due to legal reasons, you either have to build them yourself using the minnt build instructions or wait, until columbia.edu updates their build. E. Mendelson usually updates the files within a few weeks after a new release here.

To get the debug logs, just download DebugView and let it run, the loader ldntvdm should spit out some log lines that you can copy as soon as you start an application. You can copy and paste them here together with an error description (possibly screenshot). I cannot promise that I can see something from it, but we can try at least.

[AngryDavid808]

My current test machine is Windows 10 Build 1903 (I didn't get any offers for update yet, not even to 1909... Don't know why, it says,I'm on most current release).

Maybe you can try downloading the Windows 10 Disc Images from microsoft.com and install the latest version using them. I used their Disc Images to reinstall Windows, so yeah, maybe you can try giving it a go in a VM and try testing NTVDMx64 on it.

[AngryDavid808]

Update on the situation:

So, I downloaded a pre-built version of NTVDMx64 from columbia.edu (Late April build), installed it with making sure that Avast is turned off and that the Windows folder is added as an exception in Avast. I directly start ntvdm.exe a few times, it gives me some error about preparing the environment every time. Now for some ducking reason, NTVDMx64 works just fine with no errors, and OTVDM started functioning like normal again. I absolutely have no idea how I did it, but it just happened. I suspect it was all Avast's fault since it kept detecting and deleting ldntvdm.dll from pretty much everywhere it existed on my computer. So yeah, all is working.... Except for Duke Nukem 3D, setup.exe throws an error. I'll send it tommorow when I get enough sleep.

[leecher1337]

Just to let you know: Starting ntvdm.exe directly must fail, that's not how it works. The whole loader is just there to prepare everything so that ntvdm can launch. Technically, the loader registers the next command to execute with the console host and when ntvdm.exe starts, it fetches the next command to execute from console host.

Antivirus programs are a real pest, I even tried to encrypt the loader, but it didn't help against these annoying AV-programs. And when I apply for a whitelisting with an AV-vendor, the drama starts again as soon as a new loader version is compiled.

Running games with NTVDMx64 won't make you very happy anyway, as the CCPU is just way too slow for an acceptable gaming experience, so I'd say you should resort to DosBox for gaming, which is highly optimized for this.

[AngryDavid808]

Indeed, I faced problems with the games that actually worked. And apps as well.

Sometimes any app I use (I tried EDIT and QBASIC) doesn't respond well to input, so it disregards some of the keys I press. Hope you know what I mean. As for Duke Nukem and Duke Nukem 2, they worked, but with slowdowns and no PC Speaker output, except for a long beep on Duke Nukem 2 (I use a laptop that has a PC Speaker in it). I also couldn't check Soundblaster in Duke Nukem 2's settings to get some music and sound output. Both games give me no audio output.

[AngryDavid808]

Alright! So, I finally had time to log what exactly is going on when starting Duke Nukem 3D's setup program, as well as the error itself. Both of them are inside this ZIP file here.

[leecher1337]

For music, you need to set the sound to Adlib (as stated in the readme).
I suspect you didn't read the readme file, otherwise you would have noticed the comment about PC Speaker output solution in there.

I never experienced any missed keystrokes, I cannot reproduce this no matter how fast I type.

[AngryDavid808]

I'm deeply sorry, nevermind about the keystroke problem. I felt it while using EDIT.exe, but now that I'm adapted to how it works, I don't feel it anymore. Btw, can I somehow maximize the window of whatever DOS app I'm using and stretch its display (sorry if that's not the correct expression), or put it in fullscreen mode?

[leecher1337]

There used to be the -EyeStrain[xx] parameter on non-x86 builds where [xx] is a hex-code from the list of VK_ keycodes for a key to be assigned to switch between 3 stretching modes (normal - medium - big) via a hotkey. This worked in NT/2k - times when the loader fetched the commandline to execute from cmdline regkey in WOW registry node, but as the newer loaders have harcoded ntvdm.exe in the kernelbase.dll the parameter cannot be specified this way anymore. I'll issue a little patch that will allow to read the parameter from the VDM environment regkey to enable this useful feature again, let's see, if it works.

The other sad history part is fullscreen: This used to work on Windows versions up to Windows 8, until Microsoft enforded their new WDDM Display driver model which doesn't allow direct hardware pass-through anymore and therefore fullscreen was effectively dead.
As Windows 7 supported both XPDDM and WDDM (no idea why they pulled XPDDM support in later Windows versions even though it worked on Windows 7), it was possible to use with some tricks on Win 7 x32. BUT: This functionality relies on IO permission bitmap and this isn't supported on 64bit Windows, so it still isn't possible on Windows 7 x64.
Fullscreen support would be ideal for gaming etc., as it would allow direct harware access to the card without having to do the slow VGA emulation, but sadly, it's not possible with the current Windows driver architecture.