Changeset 22857 for main/trunk/greenstone2/perllib/plugins/HTMLPlugin.pm

Timestamp:

2010-09-06T19:08:11+12:00 (14 years ago)

Author:

davidb

Message:

Further adjustments to our reading in of text files/data to be Unicode aware. These changes address the issue of when the data being read in is in a different encoding. The key 'problem' was that the (now) Unicode aware was being erroneously being re-encoded within HTMLPlugin

File:

• main/trunk/greenstone2/perllib/plugins/HTMLPlugin.pm (modified) (1 diff)

main/trunk/greenstone2/perllib/plugins/HTMLPlugin.pm

@@ -1296 +1296 @@
 
     $$text =~ s/\s+/ /g; # remove \n's
+
+    # At this point $$text is a binary byte string
+    # => turn it into a Unicode aware string, so full
+    # Unicode aware pattern matching can be used.
+    # For instance: 's/\x{0101}//g' or '[[:upper:]]'
+    # 
+
+    $$text = decode("utf8",$$text);
 }