atob in base64: "Can't find variable" or "doesn't exist"

[trentlarson]

Ethers Version

5.7.0, 5.0.9

Search Terms

atob, base64

Describe the Problem

When starting my react-native project, I get one of the following errors (depending on environment):

• ReferenceError: Can't find variable: atob
• ReferenceError: Property 'atob' doesn't exist

Here are the full lines from the Metro console:

• From an existing project: [Sun Oct 23 2022 13:44:49.144] ERROR ReferenceError: Can't find variable: atob
• From a new project: ERROR ReferenceError: Property 'atob' doesn't exist, js engine: hermes

After that line, there are other errors that result from this, eg. Invariant Violation: "EndorserMobile" has not been registered. or Invariant Violation: Failed to call into JavaScript module method AppRegistry.runApplication().

Here is the error that shows in an Android simulator:

I have not yet determined if it's a problem in iOS.

The most recent version I've tested is v 5.7.0 because that's the version available on npmjs.com; I'll invest more time and testing if this is a valid bug.

The code in the snippet shows the problem in a totally new environment, using asdf to set up the language tooling. (I also get it after cloning and starting this project, but that uses older versions of these libraries.)

Code Snippet

asdf global ruby ruby 2.7.5
npx react-native init testEthers --template react-native-template-typescript --npm
cd testEthers
asdf local java zulu-11.60.19
npm run android
// The app works in this state.

npm install @ethersproject/base64

// Now insert JavaScript code using base64 into App.tsx after the other imports, for example the following which is the first code snippet from https://www.npmjs.com/package/@ethersproject/base64

import * as base64 from "@ethersproject/base64";
let encodedData = "...";
let data = base64.decode(encodedData);
console.log(data);

// Then hit 'r' in the Metro window to reload the app and see the error.

Contract ABI

No response

Errors

No response

Environment

node.js (v12 or newer), React Native/Expo/JavaScriptCore

Environment (Other)

No response

[trentlarson]

The following code fixes the problem for me in these react-native environments. If it looks good then I'll do more testing and submit it as a pull-request; I haven't yet because I'm unsure of the consequences for non-node environments -- and I've heard I should "start an issue before beginning a pull request" anyway. :-)

Some discussion may be warranted because the problem is happening in the file browser-base64.ts and maybe there's some other condition that should make it invoke base64.ts instead; I don't think react-native counts as a browser environment. Also, I don't know if this works in browsers and all the other environments.

Anyway, here's my solution: insert the following on line 4 in @ethersproject/base64/src.ts/browser-base64.ts (which you can insert into the node_modules/@ethersproject/base64/lib/browser-base64.js file to get it picked up by the app, eg. to fix the code snippet above):

var Buffer = require('buffer/').Buffer;
function atob(str) {
  return Buffer.from(str, 'base64').toString('binary');
}
function btoa(str) {
  return Buffer.from(str, 'binary').toString('base64');
}

This code may not be 100% precise or comprehensive, but it mirrors many public implementations for these functions, including the popular atob and btoa libraries. If we proceed with this solution we may want to add test cases.

Note that the first line defining "Buffer" is not necessary for some of my installations, but without that line and with the reference snippet from the issue I get the error: ReferenceError: Property 'Buffer' doesn't exist.

Note that the 'binary' encoding works, and -- although that's "misleading" and the docs recommend 'utf8' -- I got other errors when trying 'utf8'.

[trentlarson]

Another historical note: I've been running v 5.0.9 in my project for almost 2 years now with inclusion of @ethersproject/hdnode and I've never seen this problem before a week ago. I didn't make any intentional change, but I have started using a new computer and it's possible yarn 2 has done something I don't understand.

I do see that yarn.lock now has reference to v 5.1.0 and also 5.7.0 which is unexpected because I haven't changed my version of hdnode... I guess I don't understand what it means to have multiple references in there. That line changed from this:
"@ethersproject/base64@^5.0.7", "@ethersproject/base64@^5.1.0", "@ethersproject/base64@^5.5.0"
... to this:
"@ethersproject/base64@^5.1.0", "@ethersproject/base64@^5.7.0"
... just this month, so that may be related.

[ricmoo]

There has been no change to the base64 package (I’m pretty sure) since 5.0.

Did you change your version of react native, nodejs, JavaScriptCore, etc? Something that would influence the built-ins? If you switched computers, it’s quite possible you changed to a version of node you weren’t using before…

[trentlarson]

Heh... I was just going to mention that I saw no change in that library, so that's not suspect. It's very possible that it's some other tooling.

But I felt it's worth reporting since it's something that can be duplicated in a new react-native project.

[ricmoo]

Oh! Also, if you switch from cjs to esm, you will likely get this problem (I.E. if you add type: "module" to your package.json).

Ethers out of the box for node is designed for node commonjs. The ESM is for browsers. But now that modern node supports ESM, you would need to tweak the build for node ESM.

This goes away in v6, since the exports let the build tools figure out what you want.

[jl111]

failed Building static HTML for pages - 0.655s

 ERROR #95313 

Building static HTML failed

See our docs page for more info on this error:
https://gatsby.dev/debug-html


  2 | import { arrayify } from "@ethersproject/bytes";
  3 | export function decode(textData) {
> 4 |     textData = atob(textData);
    | ^
  5 |     const data = [];
  6 |     for (let i = 0; i < textData.length; i++) {
  7 |         data.push(textData.charCodeAt(i));


  WebpackError: ReferenceError: atob is not defined
  
  - base64.js:4 
    node_modules/@ethersproject/base64/lib.esm/base64.js:4:1
  
  - include.js:32 
    node_modules/@ethersproject/hash/lib.esm/ens-normalize/i
    nclude.js:32:42
  
  - lib.js:31 
    node_modules/@ethersproject/hash/lib.esm/ens-normalize/l
    ib.js:31:18
  
  - namehash.js:1 
    node_modules/@ethersproject/hash/lib.esm/namehash.js:1:1
  
  - index.js:1 
    node_modules/@ethersproject/hash/lib.esm/index.js:1:1
  
  - interface.js:1 
    node_modules/@ethersproject/abi/lib.esm/interface.js:1:1
  
  - index.js:1 
    node_modules/@ethersproject/abi/lib.esm/index.js:1:1
  
  - index.js:1 
    node_modules/@ethersproject/contracts/lib.esm/index.js:1
    :1
  
  - ethers.js:1 
    node_modules/ethers/lib.esm/ethers.js:1:1
  

I think i'm seeing a similar issue. Using gatsby.

[ricmoo]

atob is definitely defined in all browsers. Is it complaining at compile time? Does it work against the typescript directly? In that case make sure you include dom in the tsconfig. If it is trying to tree-shake, you might need to add atob and btoa in the external names; I’m not familiar with Gatsby, but would have expected this to be the default when exporting a browser build…

[trentlarson]

In my case, it is not complaining at compile time; it only happens as the App.tsx tries to run the decode function (from screenshot).

I don't know enough to say whether it works "against the typescript directly"... not sure what that means or how to test that.

[ricmoo]

Sorry if I’ve already asked, but you’ve followed the instructions here?

[hexcowboy]

Sorry if I’ve already asked, but you’ve followed the instructions here?

I'm also running into this exact same issue even with the shim installed.'

Also I want to note this issue occurs on iOS as well.

[ricmoo]

Also, keep in mind that if you are using ESM, that imports are async, so any shimmed object won't be available in the top scope (I think?).

What happens if you wrap your code in a setTimeout(() => { codeHere(); })?