UTF-16 Encoding

This section provides a quick introduction of the UTF-16 (Unicode Transformation Format - 16-bit) encoding for Unicode character set. Paired surrogates are used for characters in the U+10000...0x10FFFF range.

Once we learned how to convert Unicode code points in the U+10000...0x10FFFF range into paired surrogates, we are ready to learn how UTF-16 encoding works.

UTF-16: A character encoding that maps code points of Unicode character set to a sequence of 2 bytes (16 bits). UTF-16 stands for Unicode Transformation Format - 16-bit.

Here is my understanding of the UTF-16 specification. When UTF-16 encoding is used to encode (serialize) Unicode characters into a byte stream for communication or storage, there are 3 valid optional formats:

For example, all 3 encoding streams list below are valid UTF-16 encoded streams for 3 Unicode characters, U+004D, U+0061 and U+10000:

When UTF-16 encoding is used to decode (deserialize) a byte stream into Unicode characters, the following logic should be used:

Table of Contents

 About This Book

 Character Sets and Encodings

 ASCII Character Set and Encoding

 GB2312 Character Set and Encoding

 GB18030 Character Set and Encoding

 JIS X0208 Character Set and Encodings

 Unicode Character Set

 UTF-8 (Unicode Transformation Format - 8-Bit)

UTF-16, UTF-16BE and UTF-16LE Encodings

 What Are Paired Surrogates

UTF-16 Encoding

 UTF-16BE Encoding

 UTF-16LE Encoding

 UTF-32, UTF-32BE and UTF-32LE Encodings

 Python Language and Unicode Characters

 Java Language and Unicode Characters

 Character Encoding in Java

 Character Set Encoding Maps

 Encoding Conversion Programs for Encoded Text Files

 Using Notepad as a Unicode Text Editor

 Using Microsoft Word as a Unicode Text Editor

 Using Microsoft Excel as a Unicode Text Editor

 Unicode Fonts

 Archived Tutorials


 Full Version in PDF/EPUB