Changeset 34220 for main

Timestamp:

2020-06-29T23:54:16+12:00 (4 years ago)

Author:

ak19

Message:

1. TextPlugin takes care to preserve whitespace formatting when converting txt to html, by nesting text in pre tags. In a recent carefully tabspaced txt file, the final document produced by GS had lost all these tabs. It turns out that this was done to allow XMLParser not to choke on control chars. Have encoded tabs as entities as they're going into doc.xml and ultimately html context instead of tabs being destructively removed. 2. TextPlugin now skips opening punctuation too, not just spaces before setting the title meta to the first non-newline sequence of content.

Location:

main/trunk/greenstone2/perllib

Files:

• docprint.pm (modified) (1 diff)
• plugins/TextPlugin.pm (modified) (1 diff)

main/trunk/greenstone2/perllib/docprint.pm

@@ -103 +103 @@
     # (XML::Parser will barf on anything it doesn't consider to be
     # valid UTF-8 text, including things like \c@, \cC etc.)
-    $all_text =~ s/[\x00-\x09\x0B\x0C\x0E-\x1F]//g;
-
+    # Will treat tab chars, \x09, as a special case right after this
+    $all_text =~ s/[\x00-\x08\x0B\x0C\x0E-\x1F]//g;
+    
+    # $all_text gets written out into an xml context and represents the html version of a doc,
+    # allowing the use of html entities for the tab character (	)
+    # Tabs (ASCII \x09) may be meaningful spacing in such cases whether the html emanated from a
+    # text file, original html or other doc. Particularly when tabs are nested in <pre> tags.
+    # Instead of removing tabs, replacing tabs with their entity reference will allow <pre> tags
+    # to continue preserving any tabs in the final html display.
+    # Hopefully with this, XML::Parser will not choke on tabs, and we get tab stop spaces preserved
+    # in the html output.
+    # This may be the best location to do this replacement and not in TextPlugin, because an html
+    # source doc may contain <pre> elements with tab stops, so then HTMLPlugin would have to do the
+    # replacement too.
+    $all_text =~ s/\x09/&#09;/g;
+    
     return $all_text;
 }

main/trunk/greenstone2/perllib/plugins/TextPlugin.pm

@@ -111 +111 @@
     $title =~ s/$self->{'title_sub'}//;
     }
-    $title =~ /^\s*([^\n]*)/s; $title=$1;
+    # A series of spaces and/or punctuation too can be skipped to get at a meaningful title?
+    # https://www.geeksforgeeks.org/perl-special-character-classes-in-regular-expressions/
+    $title =~ /^[\s|[:punct:]]*([^\n]*)/s; $title=$1;
     $title =~ s/\t/ /g;
     $title =~ s/\r?\n?$//s; # remove any carriage returns and/or line feeds at line end,