Skip to content

[Settings] Add UI smoke test that visits every shell NavigationView item#48411

Closed
crutkas wants to merge 1 commit into
microsoft:mainfrom
crutkas:user/crutkas/uitest-settings-navigation-smoke
Closed

[Settings] Add UI smoke test that visits every shell NavigationView item#48411
crutkas wants to merge 1 commit into
microsoft:mainfrom
crutkas:user/crutkas/uitest-settings-navigation-smoke

Conversation

@crutkas

@crutkas crutkas commented Jun 9, 2026

Copy link
Copy Markdown
Member

Summary

Adds a WinAppDriver smoke test that drives the real PowerToys.Settings.exe through every NavigationView item in the shell and asserts the Settings process stays alive after each navigation. This is the runtime counterpart to the unit tests being added for ShellViewModel.HandleNavigationFailure - the unit tests prove the helper does not throw for null inputs, this UI test proves no page in the shell triggers a navigation-failure path that crashes the process.

Background

The Settings shell wires Frame.NavigationFailed to a handler that, in the past, simply re-threw e.Exception. Throwing from a NavigationFailed handler propagates back through the WinRT marshalling layer as a stowed exception and the runtime tears the process down via RoFailFastWithErrorContextInternal2. A regression of this kind in ShellViewModel.Frame_NavigationFailed is invisible to the existing Settings.UI.UnitTests because NavigationFailedEventArgs is a sealed WinRT type that cannot be constructed from MSTest - the failure can only be observed by actually attempting navigation against a running WinUI Frame.

This UI test fills that gap: any future change that re-introduces a throw from Frame_NavigationFailed, or breaks a module page's constructor, will turn this test red instead of shipping to users.

Change

New file: src/settings-ui/UITest-Settings/NavigationSmokeTests.cs.

  • Single class NavigationSmokeTests deriving from Microsoft.PowerToys.UITest.UITestBase, scoped to PowerToysModule.PowerToysSettings.
  • Hardcoded list of (ParentGroupId, ItemId) tuples covering all NavigationView items in the shell (Dashboard, General, plus every module under System tools, Windowing & layouts, Input / Output, File management, Advanced). Items reference UIA AutomationId rather than Content, so the test is localization-independent.
  • Single test method AllNavigationViewItems_NavigateWithoutCrashing iterates the list, expands each parent group on first encounter, clicks each item, and asserts IsWindowOpen("Settings") returns true after every navigation.

Validation

MSBuild src\settings-ui\UITest-Settings\UITest-Settings.csproj /p:Configuration=Release /p:Platform=x64 builds clean.

The UITest project has <RunVSTest>false</RunVSTest> so it does not execute as part of normal MSBuild PR validation - it runs in the dedicated UI-test pipeline alongside the existing SettingsTests and OOBEUITests. The test follows the same patterns those existing classes use (UITestBase constructor scope, Find<NavigationViewItem>(...) for element lookup, Task.Delay for navigation settle).

Notes

  • I deliberately did not add a search-box submission case in this PR - that requires interacting with the AutoSuggestBox (x:Name="SearchBox", x:Uid="Shell_SearchBox") and asserting on free-text query navigation, which deserves its own test class once we settle on the right pattern for typed text in WinAppDriver. The navigation-item smoke is the highest-value first cut.
  • The list of nav items is hardcoded rather than discovered dynamically because dynamic enumeration would mask additions (a new module added without a smoke entry would silently go untested). Adding a new module's nav item to this list is the intended way to opt that page into the smoke.

Drives PowerToys.Settings.exe end-to-end via WinAppDriver and asserts the Settings process stays alive after navigating to every shell NavigationView item. Covers Dashboard, General, and every module page across the System tools, Windowing and layouts, Input/Output, File management, and Advanced groups.

This is the runtime counterpart to the unit tests on ShellViewModel.HandleNavigationFailure. Unit tests cannot exercise NavigationFailedEventArgs (sealed WinRT type) so any regression that re-introduces a throw from Frame_NavigationFailed - or breaks a module page constructor - can only be caught by actually attempting navigation against a running WinUI Frame.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@crutkas

crutkas commented Jun 9, 2026

Copy link
Copy Markdown
Member Author

Replacing this with a winapp CLI-based version that does not depend on WinAppDriver/Appium. Will open a follow-up PR shortly with the same coverage on the new framework.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant