VT mouse mode: no reaction to touch inputs

[ramonacat]

I am using the latest version of Windows Terminal, on the latest version of Windows 11, on a Surface Go 3 tablet.
When I try to use the touch screen to scroll by dragging my finger, or to select a tmux pane by tapping it, there is no effect.
Those things work perfectly fine with a mouse or touchpad.

Windows Terminal Version: 1.22.11141.0
Windows version information:

Edition	Windows 11 Home
Version	24H2
Installed on	‎15/‎03/‎2025
OS build	26100.4061
Experience	Windows Feature Experience Pack 1000.26100.84.0

Hardware/drivers details from the Surface app:

Device information: 
Surface model: Surface Go 3 Model 2022 i3
UEFI: 15.102.143.0
EC: 4.26.128.0
App version: 75.5150.121.0
BIOS Version/Date: Microsoft Corporation 15.102.143 10/18/2024
Touch driver: 6.4.38.0
Wi-Fi driver: 23.80.0.7
Edition: Windows 11 Home
OS build: 26100.4061
Processor: Intel(R) Core(TM) i3-10100Y CPU @ 1.30GHz
Installed RAM: 8 GB
Storage size: 510 GB free of 595 GB
GPU: Intel(R) UHD Graphics 615
Screen resolution: 1920 x 1280

[DHowett]

Thanks for the report! I'm guessing you're using tmux exclusively, right? If you open a tab containing Windows PowerShell or Command Prompt, does scrolling work for you?

Today, we expressly skip touch input for mouse-driven VT applications (like tmux). We probably shouldn't.

terminal/src/cascadia/TerminalControl/TermControl.cpp

Lines 1954 to 1968 in 8a01833

	void TermControl::_TappedHandler(const IInspectable& /*sender*/, const TappedRoutedEventArgs& e)
	{
	Focus(FocusState::Pointer);
	if (e.PointerDeviceType() == Windows::Devices::Input::PointerDeviceType::Touch)
	{
	// Normally TSF would be responsible for showing the touch keyboard, but it's buggy for us:
	// If you have focus on a TermControl and type on your physical keyboard then touching
	// the TermControl will not show the touch keyboard ever again unless you focus another app.
	// Why that happens is unclear, but it can be fixed by us showing it manually.
	setInputPaneVisibility(reinterpret_cast<HWND>(OwningHwnd()), true);
	}
	e.Handled(true);
	}

terminal/src/cascadia/TerminalControl/TermControl.cpp

Lines 2000 to 2015 in 8a01833

	if (type == Windows::Devices::Input::PointerDeviceType::Touch)
	{
	// NB: I don't think this is correct because the touch should be in the center of the rect.
	// I suspect the point.Position() would be correct.
	const auto contactRect = point.Properties().ContactRect();
	_interactivity.TouchPressed({ contactRect.X, contactRect.Y });
	}
	else
	{
	const auto cursorPosition = point.Position();
	_interactivity.PointerPressed(TermControl::GetPressedMouseButtons(point),
	TermControl::GetPointerUpdateKind(point),
	point.Timestamp(),
	ControlKeyStates{ args.KeyModifiers() },
	_toTerminalOrigin(cursorPosition));
	}

While we figure out the right solution, though, I was thinking that #3656 (which has an open PR at #18928) provides a higher-level solution by promoting tmux panes to top-level interactable UI elements. 🤔

[ramonacat]

I did a couple more tests:

1. Just powershell with a bunch of output, not running any apps - scrolling by finger works
2. Powershell session with ssh to a remote server, with bash, with a bunch of output - scrolling by finger also works
3. Neovim without tmux - clicks and finger scrolls don't work

I strongly suspect that neovim without tmux uses the same APIs though, so that probably just confirms your comment above.