Changeset 32640 for main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm

Timestamp:

2018-11-29T21:23:49+13:00 (5 years ago)

Author:

ak19

Message:

Important changes (and commented out debugging statements) to get charset encodings (utf8 or specifically utf8mb4 in perl mysql case) to work with GS SQL Plugs on Windows. First, Strawberry Perl 5.22 was required and hence committed before this since Strawberry Perl 5.18 came with older mysql DBD/DBI packages that didn't decode utf8 content in the database when content was retrieved. Strawberry Perl 5.22 came with newer versions of DBD and DBI that do this automatically, as has been the case in Linux testing where Ubuntu had Perl 5.22 with sufficiently new versions of the DBI/DBD mysql perl packages. The newer Perl and specifically the newer MySQL DBD/DBI packages required some important changes to the gsmysql.pm code in the way charset encoding stuff is configured. This should work on Linux too, as that already allowed 2 ways to configure DB encoding stuff. I chose the single-line version on Linux, no longer supported with the DBI/DBD upgrade that comes with our new Strawberry Perl 5.22, so opting for the 2 line version to setup the DB encoding stuff which works on Windows and should continue to work on Linux too (where I had tested it before settling on the single-line variant).

File:

• main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm (modified) (3 diffs)

main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm

@@ -38 +38 @@
 use DBI; # the central package for this plugout
 
+#use unicode;
 
 # This plugout does not output the metadata and/or fulltxt xml to a file,
@@ -304 +305 @@
     if($proc_mode eq "all" || $proc_mode eq "meta_only" ) {
     
+    ##binmode(STDERR, ":utf8"); # shouldn't be necessary as we call &unicode::utf8decomp() to avoid wide-character warnings by printing wide chars as unicode codepoints
+    #print STDERR "###### dc.Title: ".&unicode::utf8decomp($doc_obj->get_metadata_element($doc_obj->get_top_section(), "dc.Title"))."\n";
+    
     foreach my $data (@{$section_ptr->{'metadata'}}) {
         my $meta_name = $data->[0];
@@ -310 +314 @@
         # going into it, unlike with doc(sql).xml
         my $meta_value = $data->[1];
-        
+        
         # Write out the current section's meta to collection db's METADATA table