Changeset 32089 for gs2-extensions

Timestamp:

2017-12-08T19:16:12+13:00 (6 years ago)

Author:

ak19

Message:

1. Attempted fix by Kathy and me for Diego's problem of PDFBox's handling of a PDF. When it was set to convert_to_html, it built fine, but convert_to_text produced something that was invalid XML in doc.XML and build failed. Diego reasoned correctly that building ought to succeed in both cases if it succeeded in one case. Kathy found the correct fix for escaping the ampersand character (it wasn't & to & that I'd attempted, nor did using HTML::Entities' encode work either). 2. The fix needed to read and write files, so introducing readUTF8File() and writeUTF8File() into FileUtils.pm for reusability. Need to still contact John Thompson to ask him if and how these functions need to be modified to support parallel processing, for which FileUtils was written.

File:

• gs2-extensions/pdf-box/trunk/java/perllib/plugins/PDFBoxConverter.pm (modified) (2 diffs)

gs2-extensions/pdf-box/trunk/java/perllib/plugins/PDFBoxConverter.pm

@@ -32 +32 @@
 no strict 'subs'; # allow barewords (eg STDERR) as function arguments
 
+#use HTML::Entities; # for encoding characters into their HTML entities when PDFBox converts to text
+
 use gsprintf 'gsprintf';
+use FileUtils;
 
 # these two variables mustn't be initialised here or they will get stuck 
@@ -257 +260 @@
     #print STDERR "**** item file: $target_file_path\n";
     }
-    
+    elsif ($self->{'converted_to'} eq "text") {
+    # ensure html entities are doubly escaped for pdfbox to text conversion: & -> &
+    # conversion to html does it automatically, but conversion to text doesn't
+    # and this results in illegal characters in doc.xml
+
+    my $fulltext = &FileUtils::readUTF8File($target_file_path);
+    #$fulltext = &HTML::Entities::encode($fulltext); # doesn't seem to help
+    $fulltext =~ s@&@&@sg; # Kathy's fix to ensure doc contents don't break XML
+    &FileUtils::writeUTF8File($target_file_path, \$fulltext);
+    }
+
     if ($had_error) {
     return (0, $result,$target_file_path);