feat(cypress): Component Testing

[Blfrg]

As of today Cypress released v7.0.0 with support for "Component Testing" (alpha - no longer experimental).
This will allow us to perform the equivalent of 'unit tests' from within the real browser environment.

---

Benefits of [Cypress-based] Component tests

• An optimal bridge between traditional unit tests (Jest) and traditional e2e tests (Cypress)
• Isolated component tests should run much quicker than Full App e2e tests
• Allow for catching inconcistencies between browsers at the component level
• May be easier to write or reason about in some instances
  (than comparable Jest [JSDOM-based] unit tests) when testing:
  • visual results (CSS or [real] DOM)
  • viewport-size constrained components
  • natural human interaction (click, keypress, etc)
• Works with Vue2, Vue3, and TypeScript
  • Cypress mount() is based on vue-test-utils
    • I'm suspecting we might need a Cypress plugin which implements Quasar mountFactory() the way Cypress needs
    • This Cypress issue asks about usage with Quasar, it could be a good place to start working with Cypress to ensure functionality with Quasar's wrapper

---

Additional Cypress 'Component Testing' release literature

Release Blog Post
Vue specific docs
Vue specific example code
General Component Testing Guide
Cypress Vue Docs

---

I believe it will take some prep and examples from Quasar side to support Cypress Component Testing.
The Quasar Team (ping @IlCallo) is currently focused on the transition to Vue3 so it may be some time before this can be fully addressed.
I wanted to at least put it on the roadmap as a milestone to discuss and work towards when time is permitting.

[IlCallo]

This is on my "wish list" but still haven't been able to explore it.
I'd love to have a "unified" Cypress testing experience instead of having to mix Jest and Cypress to cover the whole testing spectrum.

If anyone is up to help, I can offer insight and guidance on the repo structure and inner workings to help you get started

[asaloff]

@IlCallo @Blfrg - I'm very interested in getting Cypress-based component tests working with our Quasar application. I have to admit though, I took a look at this repo and was a little lost. Not sure how much help I would be, but happy to help if I can.

It looks to me like Cypress' mount is just an extension of vue-test-utils's same function meaning it should support passing in a localVue instance (https://vue-test-utils.vuejs.org/api/options.html#localvue)

As far as the webpack config, I tried to run the Quasar dev server with Cypress component tests but it looks like they require the use of the @cypress/webpack-dev-server with a valid webpack.config.js file. The webpack config that Quasar generates throws some validation errors when passed to their startDevServer function.

I believe the following would do the trick (based on my limited Quasar knowledge):

• A way to generate a usable (for Cypress) webpack.config.js to pass to the Cypress dev server.
• A localVue that uses the boot files, store, and router.

[IlCallo]

For localVue, right now I preferred to keep the initialization of boot files, router and store outside the helpers, as I currently don't have enough examples to abstract that part in a useful and flexible way.

Check out Jest AE docs to see how I suggest to proceed on that aspect.

Regarding the repo: every AE has its own package under packages folder.
Currently maintained ones are Cypress and Jest.

Into those folders, most of other folders are template folders: they're copied over when installing the AE, depending on which options has been selected.
If you're not used to how AEs work, check out Quasar docs

Regarding webpack, that could be a problem, because Quasar webpack config is pretty complex... I don't fully understand it too, you can see it here

[asaloff]

I looked into the webpack config a bit and found that Quasar exposes the ability to inspect the webpack config here https://github.com/quasarframework/quasar/blob/a59f2a180377c5798cfa9601fcdc029403e21990/app/bin/quasar-inspect

Based on that I was able to generate the webpack config for my project using this script:

module.exports = async function inspect() {
  const extensionRunner = require('@quasar/app/lib/app-extension/extensions-runner');
  const getQuasarCtx = require('@quasar/app/lib/helpers/get-quasar-ctx');

  const ctx = getQuasarCtx({
    mode: 'spa',
    target: void 0,
    debug: false,
    dev: true,
    prod: false,
  });

  const QuasarConfFile = require('@quasar/app/lib/quasar-conf-file');

  // register app extensions
  await extensionRunner.registerExtensions(ctx);

  const quasarConfFile = new QuasarConfFile(ctx);

  try {
    await quasarConfFile.prepare();
  } catch (e) {
    console.log(e);
    return;
  }
  await quasarConfFile.compile();
  return quasarConfFile.webpackConf;
};

It hardcodes to spa mode, but it can be easily modified. I was able to start the Cypress dev server using the generated webpack config.

// cypress/plugins/index.js

const { startDevServer } = require('@cypress/webpack-dev-server');
const generateWebpackConfig = require('../../generate-webpack');

module.exports = (on, config) => {
  on('dev-server:start', async (options) => {
    const webpackConfig = await generateWebpackConfig();
    webpackConfig.resolve.alias.vue = 'vue/dist/vue.js';
    return startDevServer({
      options,
      webpackConfig,
    });
  });

  return config;
};

Next step is to figure out the localVue part. I'll post back if I make any headway over the weekend.

[asaloff]

Okay I got this working in my application.

As far as the localVue - I can see how it would be difficult to make this flexible. Here is an example of how I create a localVue:

import createStore from '../../store';
import createRouter from '../../router';
import '../../css/app.scss';
import 'quasar/src/css/index.sass';
import 'material-icons/iconfont/material-icons.css';
import { createLocalVue as createLocalVueUtil } from '@vue/test-utils';
import { VueConstructor } from 'vue';
import Vuex from 'vuex';
import VueRouter from 'vue-router';
import { mount as cypressMount } from '@cypress/vue';
import Quasar from 'quasar';
import Vue from 'vue';

Vue.use(Vuex);
Vue.use(VueRouter);
Vue.use(Quasar);

const bootFiles = [
  'composition-api',
  'axios',
  'auth',
  'jsonapi',
  'components',
];

const initLocalVue = (vue: VueConstructor) => {
  const store = createStore({ Vue: vue });
  const router = createRouter({ Vue: vue });

  for (const file of bootFiles) {
    const boot = require(`../../boot/${file}`).default;
    boot({
      Vue: vue,
      store,
      router,
    });
  }

  return { store, router };
};

const createLocalVue = () => {
  const localVue = createLocalVueUtil();
  const { store, router } = initLocalVue(localVue);
  return { store, router, localVue };
};

export const mount = (
  component: VueConstructor,
  additionalOptions?: { [key: string]: unknown }
) => {
  const { store, router, localVue } = createLocalVue();
  return cypressMount(component, {
    localVue,
    store,
    router,
    ...additionalOptions,
  });
};

Obviously it's very specific to my application...

[IlCallo]

As far as the localVue - I can see how it would be difficult to make this flexible
Yeah, since unit testing is all about testing stuff in isolation, automatically loading all boot files from the quasar.conf could be harmful most of the times.
But we could surely add an helper which loads them under the hood if you specify their names, as you did

Thanks for sharing about your setup, it will really help me when I get on it!
Do you want/have the time to try adding support for it with a PR? We can plan what to put into the PR by discussing it in a short call

[asaloff]

I'm glad that was helpful! While I'd like to put time into making a PR, I'm afraid I don't have the time at the moment without neglecting my other responsibilities.

[IlCallo]

No problem, if you find some time go for it.
Otherwise I'll try to work on this when I get back into testing scope later this month :)

[denisraison]

That's neat @asaloff!!
Looking forward to having that added to the project.

[emahuni]

Hi guys. I don't know whether this is another way of doing this or not, or whether something can come out of it or not. But I am currently using this and it works like a charm.

I wanted to access running components from a running Quasar app and exposed Quasar through a boot file this way:

export default async ({ app, router, Vue, store }) => {
  /**
   *  expose these variables during development only
   */
  if (process.env.DEV) { // this can be some other env like testing...
    console.info(`App is running in development mode, exposing inner Vue objects to make testing easier...`);
    
    const QApp = {
      components: {},
      app,
      store,
      router,
      Vue,
      quasar:     { date, colors, dom, format, scroll }, // some Quasar plugins imported as usual  above...
    };
    
    window.QApp = QApp;
    /**
     * inject each component into the global App object so that we can easily access it in the browser during development
     */
    Vue.mixin({
      created () {
        // console.debug('context of this: ', this);
        
        let name = this.$options.name;
        if (name) {
          // inject component to components object, I know about flackyness here, not really useful just to demo what I mean
          QApp.components[name + this._uid] = this;
          if (name === 'App') {
            self['AppVue'] = this;
            QApp.AppVue = this; // this is the main App.vue file, useful for getting other Vue prototype stuff
          }
        }
      },
      
      beforeDestroy () {
        let name = this.$options.name;
        if (name) {
          // remove component from components object
          delete QApp.components[name + this._uid];
          delete QApp.components[name + this._uid];
        }
      },
    });
  }
 }

This way I can test components as they are in the app without creating another Quasar instance or something. This way, a small instance can also be created as indicated above, then exposed to Cypress as Cypress.QApp this way:

cy.window().then(win => { // win is the global window object of the browser in which the Quasar app is running.
              expect(win.QApp).to.be.ok; // as soon as it loads we have that QApp object in there
              Cypress.QApp = win.QApp;
            });

Now I can use Vue and Quasar in Cypress using the configured and running Quasar App. I then visit some known testing route in the app so that it loads Quasar and load that boot file. That route will load the App.vue component. I can mount whatever component I want to test using Vue and Quasar like I normally do, using that component in a spec file. Use that object to access, vuex store, router, and the underlying Vue instance for the Quasar app., etc. It's like working in the actual app.

I think something can come out of this combined with @asaloff mini App, for independent component testing.

[Rolanddoda]

In May:

No problem, if you find some time go for it. Otherwise I'll try to work on this when I get back into testing scope later this month :)

@IlCallo any update on this please ?

[IlCallo]

There's a PR from the community to add it, gonna merge as soon as possible: #185

[lavoscore]

Very excited with the PR. Until it comes up it may be useful to post how I made CCT work:

I followed @asaloff's code for the webpack config, but for some reason it wasn't working until I changed the return to this:

  // return quasarConfFile.webpackConf; <-- not working
  const { splitWebpackConfig } = require('@quasar/app/lib/webpack/symbols');
  const configEntries = splitWebpackConfig(quasarConfFile.webpackConf, 'spa');

  return configEntries[0].webpack;

And in my specialized support/component.ts:

// CSS
import 'quasar/src/css/index.sass';
import 'src/css/app.scss';
import '@quasar/extras/material-icons/material-icons.css';
import '@quasar/extras/fontawesome-v5/fontawesome-v5.css';

// injects Quasar plugin to VTU's globals
import { config } from '@vue/test-utils';
import { Quasar } from 'quasar';
config.global.plugins = [Quasar];

The css is being manually added of course. It is not clear to me whether the PR tackles this, but I think the css could be inferred from quasar.conf settings - e.g. automatically add FontAwesome.

I don't use Vuex or Vue-18n and I haven't tested with router, but since CCT uses VTU under the cover it should be trivial to add them under globals/plugins. or in a custom factory mount (?)