If a unit test fails and leaves a window open, other instances of Brackets will have wrong preferences later

[peterflynn]

1. Run the unit tests in a manner that will leave one of the test windows open (seems to happen if Live Edit tests ever fail with a dialog shown, but also seems to happen in most cases where an exception is thrown by code in the test window)
2. Close the stray test window
3. Refresh your original dev environment Brackets window

Result: dev environment window switches its root folder to brackets/src, and I believe all your other prefs are reset to defaults.

Expected: dev environment window is not affected by unit tests.

The unit tests run against a clean copy of the prefs store. The way this works is that the test runner sets a persistent master pref that says where the rest of the prefs store lives, launches the test window, waits for the test to finish, and then resets the persistent master setting to point back to the "real" user prefs.

Something is going wrong in that process where the master setting gets stuck pointing to the clean unit-test copy of the prefs. Jason's theory is that the master setting gets reset correctly by the test runner, but when you manually close the leftover test window it rewrites the master setting again to point to whatever it thinks its current prefs store is.

Seems like there are a few potential fixes:

• Make failed tests not leave stray windows open. This would be nice to fix in general, but might be a bunch of work.
• I'm not sure why Brackets would want to rewrite the master setting on quit. Maybe we could remove that step?
• Ideally, if you wanted to override the default prefs store location you'd do it by passing a parameter at launch (URL param, command line arg, whatever). That way there's no persistent pointer that the unit tests have to overwrite and restore. Lots of other apps have a feature like this, so it's a generally useful thing to have... but probably a bunch of work.

[pthiess]

Reviewed

[peterflynn]

The end result is that in some cases it feels like #467 still isn't fixed (because your main Brackets instance starts using the unit-test copy of the prefs, which were supposed to be isolated).

[dangoor]

Taking this as preferences have totally changed and I'm looking at making the prefs more reliable in tests.

[dangoor]

Woohoo.. 2 year old bug is fixed by #7163 (a URL param overrides the preferences creation for tests)

[redmunds]

FBNC back to @peterflynn