This section provides a tutorial example on how to write a sample program, EncodingConverter.java, to convert text files from one character set encoding to another.
With the help of HexWriter.java, I know that file hello.utf-16be stores strings of characters
in UTF-16BE encoding.
Now I want to write a sample program, EncodingConverter.java, to convert text files from one
character set encoding to another:
/**
* EncodingConverter.java
* Copyright (c) 2002 by Dr. Herong Yang
*
* This program allows you to convert a text file in one encoding
* to another file in a different encoding.
*/
import java.io.*;
class EncodingConverter {
public static void main(String[] a) {
String inFile = a[0];
String inCharsetName = a[1];
String outFile = a[2];
String outCharsetName = a[3];
try {
InputStreamReader in = new InputStreamReader(
new FileInputStream(inFile), inCharsetName);
OutputStreamWriter out = new OutputStreamWriter(
new FileOutputStream(outFile), outCharsetName);
int c = in.read();
int n = 0;
while (c!=-1) {
out.write(c);
n++;
c = in.read();
}
in.close();
out.close();
System.out.println("Number of characters: "+n);
System.out.println("Number of input bytes: "
+(new File(inFile)).length());
System.out.println("Number of output bytes: "
+(new File(outFile)).length());
} catch (IOException e) {
System.out.println(e.toString());
}
}
}
Compile this program and use it to convert our hello message file into several
encodings:
By observing the output files, you should notice that:
hello.ascii - In this file, only the English message is good, because it contains only
ASCII characters. Both Simplified Chinese and Traditional Chinese messages
are not good. Characters in these messages are replaced by 0x3F, an indication
of invalid code.
hello.iso-8859-1 - This is identical to hello.ascii, because there is no
characters in the 0x80 - 0xFF range.
hello.utf-8 - This file contains all messages with no damages. The ASCII
characters are stored as one-byte characters as expected.
hello.gbk - In this file, the Simplified Chinese message is good. In fact,
characters in the Simplified Chinese message are stored as code values in
GBK character set standard. The English message is also good, because GBK is
ASCII backward compatible. We are lucky with the Traditional Chinese message,
because the Big5 characters used in the message are also valid in GBK standard.
If you use some Big5 special characters, the result could be different.
hello.big5 - In this file, the Traditional Chinese message is good. In fact,
characters in the Traditional Chinese message are stored as code values in
Big5 character set standard. The English message is also good, because Big5 is
ASCII backward compatible. We are not lucky with the Simplified Chinese message,
two GB characters used in the message are not valid in Big5 standard. 0x3F was
stored for those characters.
hello.shift_jis - In this file, the English message is still good. Some of the
characters from both Simplified and Traditional Chinese messages are invalid,
replaced by 0x3F placeholders. Some of the Chinese characters are still valid
in Shift_JIS character set. This is not so surprising, because there are many
shared characters in Chinese and Japanese.
