Changeset 27321 for main/trunk

Timestamp:

2013-05-09T16:33:05+12:00 (11 years ago)

Author:

ak19

Message:

Two bugfixes: 1. Handling of quotes not just the CSV fields containing commas, but around all CSV fields, can happen when a CSV file is exported from OpenOffice's Calc spreadsheet program. 2. The second bug was when 2 pdfs, called one.pdf and two.pdf have metadata assigned in meta.csv. Then the metadata gets duplicated for two.pdf (2 dc.Title, 2 dc.Author). If the 2 pdfs were called 1.pdf and 2.pdf, the meta was duplicated for both files. Thanks to Kathy who found that this had something to do with the order of the documents and meta.csv getting processed when the EmbeddedMetadataPlugin was also in the list. She also found a different bug: that while EmbeddedMetaPlug merged its own extrameta with existing extrameta, MetaCSVPlug did not merge but overwrote all meta with its own. After adding in merging of extrameta into MetaCSVPlug, the initial bug of duplicate assignment of the meta in the CSV file was resolved too.

File:

• main/trunk/greenstone2/perllib/plugins/MetadataCSVPlugin.pm (modified) (3 diffs)

main/trunk/greenstone2/perllib/plugins/MetadataCSVPlugin.pm

@@ -145 +145 @@
     my $found_filename_field = 0;
     for (my $i = 0; $i < scalar(@csv_file_fields); $i++) {
-    # Remove any spaces from the field names
+    # Remove any spaces from the field names, and surrounding quotes too
     $csv_file_fields[$i] =~ s/ //g;
+    $csv_file_fields[$i] =~ s/^"//;
+    $csv_file_fields[$i] =~ s/"$//;
+
     if ($csv_file_fields[$i] eq "Filename") {
         $found_filename_field = 1;
@@ -183 +186 @@
             $csv_line_metadata{$csv_file_fields[$i]} = [];
             }
-            push (@{$csv_line_metadata{$csv_file_fields[$i]}}, $1);
+            # remove any surrounding quotes. (When exporting to CSV, some spreadsheet
+            # programs add quotes even around field values that don't contain commas.)
+            my $value = $1;
+            $value =~ s/^"//;
+            $value =~ s/"$//;
+            push (@{$csv_line_metadata{$csv_file_fields[$i]}}, $value);
         }
         }
@@ -212 +220 @@
     $csv_line_filename = &util::filename_to_regex($csv_line_filename);
 
-    &extrametautil::setmetadata($extrametadata, $csv_line_filename, \%csv_line_metadata);
-    &extrametautil::addmetakey($extrametakeys, $csv_line_filename);
+    if (defined &extrametautil::getmetadata($extrametadata, $csv_line_filename)) { # merge with existing meta    
+
+        my $file_metadata_table = &extrametautil::getmetadata($extrametadata, $csv_line_filename);
+        
+        foreach my $metaname (keys %csv_line_metadata) {
+        # will create new entry if one does not already exist
+        push(@{$file_metadata_table->{$metaname}}, @{$csv_line_metadata{$metaname}});       
+        }
+        
+        # no need to push $file on to $extrametakeys as it is already in the list
+    } else { # add as new meta
+        
+        &extrametautil::setmetadata($extrametadata, $csv_line_filename, \%csv_line_metadata);
+        &extrametautil::addmetakey($extrametakeys, $csv_line_filename);
+    }
     # record which file the metadata came from 
     if (!defined &extrametautil::getmetafile($extrametafile, $csv_line_filename)) {