Changeset 30593 for main/trunk/greenstone2/perllib/ghtml.pm

Timestamp:

2016-06-27T18:11:27+12:00 (7 years ago)

Author:

ak19

Message:

Dr Bainbridge found another point in the code where the UTF-16 Surrogate pairs (that lead to malformed UTF-8 character errors) are encountered in HTMLPlugin. This part of the code is encountered when the PDFPlugin has the pdfbox_conversion set. PDFBox would have produced the HTML containing entities that represent characters not considered valid in UTF-8 and this then failed on Diego's test PDF until Dr Bainbridge's bugfix.

File:

• main/trunk/greenstone2/perllib/ghtml.pm (modified) (1 diff)

main/trunk/greenstone2/perllib/ghtml.pm

@@ -219 +219 @@
     }
 
+
     if (defined $code) {
+    
+    # malformed UTF-8 character used in UTF-16
+    if($code >= 0xD800 && $code <= 0xDFFF) {
+        print STDERR "Warning: encountered the HTML entity \&#$code; which represents part of a UTF-16 surrogate pair, which is not supported in ghtml::getcharequiv(). Replacing with '?'.\n";
+        $code = ord("?");
+    }
+
     # non-standard Microsoft breakage, as usual
     if ($code < 0x9f) { # code page 1252 uses reserved bytes