Changeset 28836 for main/trunk/greenstone2/perllib/plugins

Timestamp:

2014-02-12T18:03:06+13:00 (10 years ago)

Author:

ak19

Message:

A question on the mailing list involved accented characters in custom metadata set names (not metadata set values). This exposed an issue in greenstone that could not cope with utf8 characters in metaset names. The cause was the sub Char { use bytes; ... lines when reading XML. These needed to be commented out in both MetadataXMLPlugin and ReadXMLFile (as GreenstoneXMLPlugin inherits from ReadXMLFile). Doing so showed that extra Encode::decode() operations to decode strings read in from XML into utf8 were no longer needed. As a result MetaXMLPlug and GreenstoneXMLPlug no longer call decode on the metadaname name and value read in from XML, or for the full-text, since GreenstoneXMLPlugin in entirety now no longer does the 'use bytes' part. Tested with text and html collections where metadata set nanes created in custom .mds files, their assigned metadata values and a document's full-text all contained the utf-8 specific character of a-macron.

Location:

main/trunk/greenstone2/perllib/plugins

Files:

• GreenstoneXMLPlugin.pm (modified) (2 diffs)
• MetadataXMLPlugin.pm (modified) (2 diffs)
• ReadXMLFile.pm (modified) (1 diff)

main/trunk/greenstone2/perllib/plugins/GreenstoneXMLPlugin.pm

@@ -224 +224 @@
 
 
-    my $metadata_name = decode("utf-8",$self->{'metadata_name'});
-    my $metadata_value = decode("utf-8",$self->{'metadata_value'});
+    my $metadata_name = $self->{'metadata_name'};
+    my $metadata_value = $self->{'metadata_value'};
+    #my $metadata_name = decode("utf-8",$self->{'metadata_name'});
+    #my $metadata_value = decode("utf-8",$self->{'metadata_value'});
 
     $self->{'doc_obj'}->add_utf8_metadata($self->{'section'}, 
@@ -271 +273 @@
     # text read in by XML::Parser is in Perl's binary byte value
     # form ... need to explicitly make it UTF-8
-    my $content = decode("utf-8",$self->{'content'});
+    #my $content = decode("utf-8",$self->{'content'});
+    my $content = $self->{'content'};
 
     $self->{'doc_obj'}->add_utf8_text($self->{'section'}, $content);

main/trunk/greenstone2/perllib/plugins/MetadataXMLPlugin.pm

@@ -364 +364 @@
     # form ... need to explicitly make it UTF-8
     
-    my $metadata_name = decode("utf-8",$self->{'metadata_name'});
-    my $metadata_value = decode("utf-8",$self->{'metadata_value'});
+    my $metadata_name = $self->{'metadata_name'};
+    my $metadata_value = $self->{'metadata_value'};
+    #my $metadata_name = decode("utf-8",$self->{'metadata_name'});
+    #my $metadata_value = decode("utf-8",$self->{'metadata_value'});
     
     &metadatautil::store_saved_metadata($self,
@@ -392 +394 @@
 # things down significantly in some cases.
 sub Char {
-    use bytes;  # Necessary to prevent encoding issues with XML::Parser 2.31+ 
+#    use bytes;  # Necessary to prevent encoding issues with XML::Parser 2.31+ 
 
 #    if ($]<5.008) {

main/trunk/greenstone2/perllib/plugins/ReadXMLFile.pm

@@ -291 +291 @@
 # things down significantly in some cases.
 sub Char {
-    use bytes;  # Necessary to prevent encoding issues with XML::Parser 2.31+
+#    use bytes;  # Necessary to prevent encoding issues with XML::Parser 2.31+
     $_[0]->{'Text'} .= $_[1];
     return undef;