003-url.md

Title	Implement WHATWG URL Spec
Author	@jasnell
Status	DRAFT
Date	2016-08-11T10:00:00-07:00

Description

The WHATWG URL Standard specifies updated syntax, parsing and serialization of URLs as currently implemented by the main Web Browsers. The existing Node.js url module parsing and serialization implementation currently does not support the URL standard and fails to pass about 160 of the WHATWG URL parsing tests.

This proposal is to implement the WHATWG URL Standard by modifying the existing url module to provide an implementation of the URL object and associated APIs as defined by the WHATWG URL Parsing specification. Doing so improves the robustness of URL parsing, provides consistency with browser JS code, and can eventually allow the reduction of node.js specific APIs.

Initially, the implementation would be introduced as an undocumented experimental feature exposed via a new URL property in the url module.

Note: In Browser implementations, the URL constructor is a global. In the initial Node.js implementation, the URL constructor would not be a global; instead, it would be accessible via the url module (e.g. const URL = require('url').URL).

Because the existing require('url') module remains otherwise unmodified, there should be no backwards compatibility concerns. Once the performance of the new URL implementation is on par with the existing url.parse() implementation, url.parse() will go through a normal deprecation cycle and the new URL implementation will become a fully documentation and supported feature.

The current implementation can be found at:

nodejs/node#7448

Example

const URL = require('url').URL;
const url = new URL('http://user:pass@example.org:1234/p/a/t/h?xyz=abc#hash');

console.log(url.protocol);      // http:
console.log(url.username);     // user
console.log(url.password);     // password
console.log(url.host);         // example.org:1234
console.log(url.hostname);     // example.org
console.log(url.port);         // 1234
console.log(url.pathname);     // /p/a/t/h
console.log(url.search);       // ?xyz=abc
console.log(url.searchParams); // SearchParams object
console.log(url.hash);         // hash

// The SearchParams object is defined by the WHATWG spec also
url.searchParams.append('key', 'value');

console.log(url);
 // http://user:pass@example.org:1234/p/a/t/h?xyz=abc&key=value#hash


// Example using a base URL
const url2 = new URL('/foo', url);
console.log(url.protocol);      // http:
console.log(url.username);     // user
console.log(url.password);     // password
console.log(url.host);         // example.org:1234
console.log(url.hostname);     // example.org
console.log(url.port);         // 1234
console.log(url.pathname);     // /foo
console.log(url.search);       // ''
console.log(url.searchParams); // SearchParams object
console.log(url.hash);         // ''

APIs

The public API would be as defined by the WHATWG spec.

new URL(href, base)

The constructor implements the WHATWG basic parsing algorithm. Accessible via const URL = require('url').URL

• href is a string containing the URL to parse.
• base is either a string or a URL that contains the base URL to resolve against while parsing.

See https://url.spec.whatwg.org/#urlutils-members for detail on the properties of the URL object.

url.protocol

• Returns a string
• Getter/Setter

url.username

• Returns a string
• Getter/Setter

url.password

• Returns a string
• Getter/Setter

url.host

• Returns a string
• Getter/Setter

url.hostname

• Returns a string
• Getter/Setter

url.port

• Returns an integer
• Getter/Setter

url.pathname

• Returns a string
• Getter/Setter

url.search

• Returns a string
• Getter/Setter

url.searchParams

• Returns a URLSearchParams object
• Getter

url.hash

• Returns a string
• Getter/Setter

url.origin

• Returns a string
• Getter

url.href

• Returns a string
• Getter

url.toString()

• Returns a string (same as url.href)

URLSearchParams

Returned by the url.searchParams property and provides access read and write access to the search params. It is defined at https://url.spec.whatwg.org/#interface-urlsearchparams.

Accessible via require('url').URLSearchParams

searchParams.append(name, value)

searchParams.delete(name)

searchParams.get(name)

searchParams.getAll(name)

searchParams.has(name)

searchParams.set(name, value)

searchParams.*[Symbol.iterator]()

searchParams.toString()

URL.domainToASCII(domain)

URL.domainToUnicode(domain)