Changeset 6849

Timestamp:

2004-02-19T14:16:19+13:00 (20 years ago)

Author:

mdewsnip

Message:

Fix to the weird "half-good, half-bad" Unicode display on Windows. Turns out Java was kindly marking some characters as "undisplayable", and replacing them with question marks -- great, thanks very much.

File:

• trunk/gli/src/org/greenstone/gatherer/Dictionary.java (modified) (1 diff)

trunk/gli/src/org/greenstone/gatherer/Dictionary.java

@@ -226 +226 @@
         complete = complete + initial;
 
-        String unicode = null;
+        String unicode = "";
         try {
-        unicode = new String(complete.getBytes(), "UTF-8");
+        // This "ISO-8859-1" looks out of place, but it is very important.
+        // It is essential to call getBytes with an 8-bit encoding, otherwise
+        // Java kindly deems some characters "undisplayable", and replaces
+        // them with question marks. This is NOT good.
+        unicode = new String(complete.getBytes("ISO-8859-1"), "UTF-8");
         }
         catch (Exception ex) {