URL Percent Encoding and Unicode

By spec, some characters in url must be percent encoded. For example, [] in url should be %5B%5D

Browser's URL field automatically do this transformation of encoding when you paste a url and visit the site. However, different browser do not agree which characters should be encoded.

This page discuss this issue.

Summary of Browser's Behaviors on URL Percent Encoding

Here is some summary of the behavior.

• Firefox (v 3.6.3), is the most aggressive in turning characters in URL into the percent encoded form.
• Google Chrome (4.1.249.1064 (45376)) will change Unicode chars into percent encoded form, but not parenthesis chars.
• Safari (4.0.5 (531.22.7)) does convert some percent encoded chars into plain Unicode char, but not all.
• Opera (v 10.10, build 1893) is the best, it shows Unicode and paren and en-dash as is.
• IE (8.0.6001.18904), seems to take the approach that it doesn't do anything to the URL. Whatever you pasted in, remains unchanged.

Conclusion

There are several issues going on about this.

First, just what characters in URL needs to be encoded?

URL, by spec, is just a sequence of characters. Originally, it's supposed to just allow a subset of ASCII Characters only, and certain ASCII character are not allowed. The not allowed characters are those encoded by the JavaScript function encodeURI .

Now, what browsers do in the URL field, arguably has nothing to do with the URL spec. Because it's just a user interface appearance issue. Any encoding, if necessary, can be done when browser actually tries to transmit that URL or do whatever. This is why, we see different behaviors in browsers.

Further, IRI (Internationalized Resource Identifier) became popular since say 2010. IRI basically is just URL except it allows NON-ASCII characters to be in URL as is, sans encoding.

Further more, a separate issue that adds to the confusion is that, if the URL is inside a HTML doc, then the ampersand character U+26: AMPERSAND and U+3C: LESS-THAN SIGN and U+3E: GREATER-THAN SIGN all needs to be encoded in another way, called ampersand encoding.

see URL Percent Encoding and Ampersand Char

Reference

• An Introduction to Multilingual Web Addresses At http://www.w3.org/International/articles/idn-and-iri/
• Internationalized Resource Identifier
• XRI
• Internationalized domain name
• Punycode