Changeset 28836 for main/trunk/greenstone2/perllib/plugins/GreenstoneXMLPlugin.pm

Timestamp:

2014-02-12T18:03:06+13:00 (10 years ago)

Author:

ak19

Message:

A question on the mailing list involved accented characters in custom metadata set names (not metadata set values). This exposed an issue in greenstone that could not cope with utf8 characters in metaset names. The cause was the sub Char { use bytes; ... lines when reading XML. These needed to be commented out in both MetadataXMLPlugin and ReadXMLFile (as GreenstoneXMLPlugin inherits from ReadXMLFile). Doing so showed that extra Encode::decode() operations to decode strings read in from XML into utf8 were no longer needed. As a result MetaXMLPlug and GreenstoneXMLPlug no longer call decode on the metadaname name and value read in from XML, or for the full-text, since GreenstoneXMLPlugin in entirety now no longer does the 'use bytes' part. Tested with text and html collections where metadata set nanes created in custom .mds files, their assigned metadata values and a document's full-text all contained the utf-8 specific character of a-macron.

File:

• main/trunk/greenstone2/perllib/plugins/GreenstoneXMLPlugin.pm (modified) (2 diffs)

main/trunk/greenstone2/perllib/plugins/GreenstoneXMLPlugin.pm

@@ -224 +224 @@
 
 
-    my $metadata_name = decode("utf-8",$self->{'metadata_name'});
-    my $metadata_value = decode("utf-8",$self->{'metadata_value'});
+    my $metadata_name = $self->{'metadata_name'};
+    my $metadata_value = $self->{'metadata_value'};
+    #my $metadata_name = decode("utf-8",$self->{'metadata_name'});
+    #my $metadata_value = decode("utf-8",$self->{'metadata_value'});
 
     $self->{'doc_obj'}->add_utf8_metadata($self->{'section'}, 
@@ -271 +273 @@
     # text read in by XML::Parser is in Perl's binary byte value
     # form ... need to explicitly make it UTF-8
-    my $content = decode("utf-8",$self->{'content'});
+    #my $content = decode("utf-8",$self->{'content'});
+    my $content = $self->{'content'};
 
     $self->{'doc_obj'}->add_utf8_text($self->{'section'}, $content);