Error in preprocessor sourcemapping: "Cannot read property 'length' of undefined"

[benmccann]

Describe the bug
The new preprocessor sourcemapping feature seems to sometimes cause errors (CC @halfnelson @milahu)

Logs

> sapper-postcss-template@2020.11.18 build /Users/babichjacob/Repositories/sapper-postcss-template
> cross-env NODE_ENV=production sapper build --legacy

> Building...

[!] (plugin svelte) TypeError: Cannot read property 'length' of undefined
/Users/babichjacob/Repositories/sapper-postcss-template/src/routes/index.svelte
TypeError: Cannot read property 'length' of undefined
    at sourcemap_add_offset (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:22087:23)
    at get_replacement (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28575:10)
    at /Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28635:21
    at async Promise.all (index 0)
    at async replace_async (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28551:52)
    at async preprocess_tag_content (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28618:22)
    at async preprocess (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/svelte/compiler.js:28644:10)
    at async Object.transform (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/rollup-plugin-svelte/index.js:100:23)
    at async ModuleLoader.addModuleSource (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/rollup/dist/shared/rollup.js:18289:30)
    at async ModuleLoader.fetchModule (/Users/babichjacob/Repositories/sapper-postcss-template/node_modules/rollup/dist/shared/rollup.js:18345:9

To Reproduce
git clone git@github.com:babichjacob/sapper-postcss-template.git

Upgrade to svelte 3.30 and rollup-plugin-svelte 7 following the upgrade guide (example of that here: sveltejs/sapper-template#289)

sourcemap = false -> no error
sourcemap = "inline" -> that error
sourcemap = true -> that error

Severity
Several people have reported this issue on Discord. It seems to happen relatively frequently

[dimfeld]

My analysis so far:
The PostCSS processor in svelte-preprocess is returning a SourceMapGenerator instead of whatever the svelte compiler is expecting it to return. This SourceMapGenerator is coming directly from the postcss package. I'm not sure where exactly the SourceMapGenerator is supposed to turn into an actual source map, but that seems to be the missing step AFAICT.

Update:
Looks like just changing the postcss transformer isn't sufficient.

This is certainly not the exact right fix, but adding this in the Svelte compiler.js at line 28636, just before the call to get_replacement allows everything to build without errors:

if (processed.map?._mappings) {
    processed.map = processed.map.toString();
}

omg thank you

My analysis so far:
The PostCSS processor in svelte-preprocess is returning a SourceMapGenerator instead of whatever the svelte compiler is expecting it to return. This SourceMapGenerator is coming directly from the postcss package. I'm not sure where exactly the SourceMapGenerator is supposed to turn into an actual source map, but that seems to be the missing step AFAICT.

Update:
Looks like just changing the postcss transformer isn't sufficient.

This is certainly not the exact right fix, but adding this in the Svelte compiler.js at line 28636, just before the call to get_replacement allows everything to build without errors:

if (processed.map?._mappings) {
    processed.map = processed.map.toString();
}

it's giving me UnhandledPromiseRejectionWarning: /Users/jt/dev/code-challenges-sapper/node_modules/svelte/compiler.js:28636 if (processed.map?._mappings) { SyntaxError: Unexpected token '.'

[dimfeld]

Then your Node version is too old to support that syntax. You're probably better off just using Svelte 3.29 for now.

[nickreese]

Phew, glad I'm not alone. Also seems to be originating from svelte-preprocess.

Then your Node version is too old to support that syntax. You're probably better off just using Svelte 3.29 for now.

I'm using Svelte 3.17.3 ......

[dimfeld]

If you're seeing this bug, you're using Svelte 3.30.

https://bytearcher.com/articles/semver-explained-why-theres-a-caret-in-my-package-json/

[benmccann]

It was explained: https://bytearcher.com/articles/semver-explained-why-theres-a-caret-in-my-package-json/

That package.json specifies ^3.17.3 which means get the latest 3.x release, which is 3.30. The actual version used is specified in the package-lock.json. Try grep -r -A2 svelte package-lock.json

The method this bug refers to did not exist until 3.30, which means you're either using 3.30 or encountering a different issue.

It was explained: https://bytearcher.com/articles/semver-explained-why-theres-a-caret-in-my-package-json/

That package.json specifies ^3.17.3 which means get the latest 3.x release, which is 3.30. The actual version used is specified in the package-lock.json. Try grep -r -A2 svelte package-lock.json

The method this bug refers to did not exist until 3.30, which means you're either using 3.30 or encountering a different issue.

Damn I didn't known it did that, thank you and sorry for being arrogant :/
It worked with the ’~‘, thank you

[milahu]

thanks for the bug ; )

The PostCSS processor in svelte-preprocess is returning a SourceMapGenerator instead of whatever the svelte compiler is expecting it to return. This SourceMapGenerator is coming directly from the postcss package.

postcss is using this one
https://github.com/mozilla/source-map/blob/master/lib/source-map-generator.js

problem is, this one cannot produce "decoded" mappings
as the library remapping can consume

This is certainly not the exact right fix, but adding this in the Svelte compiler.js at line 28636, just before the call to get_replacement allows everything to build without errors:

if (processed.map?._mappings) {
   processed.map = processed.map.toString();
}

yes, that will fix the problem, at the cost of a slightly slower compile
(unnessesary encode + decode steps)
a little less bad is

processed.map = processed.map.toJSON();

the fastest solution is to fork the SourceMapGenerator._serializeMappings() method
and make it produce "decoded" mappings
(and ideally push that feature to upstream ....)

Details

// not tested but "im feeling lucky"

import { compareByGeneratedPositionsInflated } from 'source-map/lib/util';

if (processed.map.constructor.name == 'SourceMapGenerator') {

  /**
   * patch function _serializeMappings()
   * to return mappings in "decoded" format
   */
  processed.map._serializeMappings = function () {
    let previousGeneratedLine = 1;
    let result = [[]];
    let resultLine;
    let resultSegment;
    let mapping;

    // optimized
    const sourceIdx = this._sources.toArray().reduce((acc, val, idx) => (acc[val] = idx, acc), {});
    const nameIdx = this._names.toArray().reduce((acc, val, idx) => (acc[val] = idx, acc), {});

    const mappings = this._mappings.toArray();
    resultLine = result[0];

    for (let i = 0, len = mappings.length; i < len; i++) {
      mapping = mappings[i];

      if (mapping.generatedLine > previousGeneratedLine) {
        while (mapping.generatedLine > previousGeneratedLine) {
          result.push([]);
          previousGeneratedLine++;
        }
        resultLine = result[mapping.generatedLine - 1]; // line is one-based
      } else if (i > 0) {
        if (
          !compareByGeneratedPositionsInflated(mapping, mappings[i - 1])
        ) {
          continue;
        }
      }

      resultLine.push([mapping.generatedColumn]);
      resultSegment = resultLine[resultLine.length - 1];

      if (mapping.source != null) {
        resultSegment.push(...[
          //this._sources.indexOf(mapping.source),
          sourceIdx[mapping.source], // optimized
          mapping.originalLine - 1, // line is one-based
          mapping.originalColumn
        ]);
        if (mapping.name != null) {
          resultSegment.push(
            //this._names.indexOf(mapping.name)
            nameIdx[mapping.name] // optimized
          );
        }
      }
    }
    return result;
  }

  // call the patched function to get a "decoded" sourcemap
  processed.map = processed.map.toJSON();
}

now who said monkey-patching is bad? ; )

edit: optimized indexOf to cache-object
edit: bugfix for line numbers
edit: add .toArray()

[halfnelson]

I'm guessing the correct fix here is probably to detect malformed source maps from preprocessors and log an appropriate error for the user.

[tanhauhau]

agree with @halfnelson, where svelte should handle malformed sourcemaps better with an appropriate error,

and an actual fix the sourcemap would have to come from the upstream, postcss svelte preprocessor

[dummdidumm]

While this is true, I think we should also be pragmatic and handle this specific case because I don't think upstream will just change this , others may even rely on this specific behavior.