Changeset 37047

Timestamp:

2022-12-23T10:24:09+13:00 (16 months ago)

Author:

davidb

Message:

Introduction of 'metadata_separate_fields', a plugin option that controls which fields get the value separation split applied to. By default all fields get split when the value split character is specified, however you can get situations where you want to split on (say) ',' for a Keyword field but not in a Abstract field that happens to use commas

Location:

main/trunk/greenstone2/perllib

Files:

• plugins/CSVFieldSeparator.pm (modified) (1 diff)
• plugins/CSVPlugin.pm (modified) (2 diffs)
• strings.properties (modified) (1 diff)

main/trunk/greenstone2/perllib/plugins/CSVFieldSeparator.pm

@@ -46 +46 @@
       { 'name' => "metadata_value_separator",
     'desc' => "{CSVFieldSeparator.metadata_value_separator}",
+    'type' => "string",
+    'deft' => "",
+        'reqd' => "no" },
+      { 'name' => "metadata_separate_fields",
+    'desc' => "{CSVFieldSeparator.metadata_separate_fields}",
     'type' => "string",
     'deft' => "",

main/trunk/greenstone2/perllib/plugins/CSVPlugin.pm

@@ -193 +193 @@
     }
 
+    my $md_sep_fields = $self->{'metadata_separate_fields'};
+    undef $md_sep_fields if ($md_sep_fields eq "");
+
+    my $md_sep_fields_lookup = undef;
+    if (defined $md_sep_fields) {
+    $md_sep_fields_lookup = {};
+
+    my @md_fields = split(/\s*,\s*/,$md_sep_fields);
+
+    for my $md_field (@md_fields) {
+        $md_sep_fields_lookup->{$md_field} = 1;
+    }
+    }
+
     my $csv = Text::CSV->new();
     $csv->sep_char($separate_char);
@@ -247 +261 @@
         my $md_name = $csv_file_fields[$i];
         $csv_line_metadata{$md_name} = [];
-        if (defined $md_val_sep) {          
+
+        my $needs_md_val_sep = 0;
+        if (defined $md_val_sep) {
+            # Default coming in is 'no' (0)
+            # => Check to see if any conditions met to turn this into a 'yes' (1)
             
+            # check to see if md_sep_fields is in play, and if it is
+            # => determine if this $md_name is one of the ones in $md_sep_fields_lookup
+            
+            if (defined $md_sep_fields_lookup) {
+            if ($md_sep_fields_lookup->{$md_name}) {
+                $needs_md_val_sep = 1;
+            }
+            }
+            else {
+            # if not set, then we apply the md_val_sep to all metadata fields
+            $needs_md_val_sep = 1;
+            }
+        }
+        
+        if ($needs_md_val_sep) {            
+            
             my @within_md_vals = split(/${md_val_sep}/,$md_val);
             

main/trunk/greenstone2/perllib/strings.properties

@@ -879 +879 @@
 ConvertToRogPlugin.desc:A plugin that inherits from RogPlugin.
 
-CSVFieldSeparator.csv_field_separator: The character you've consistently used to seperate each cell of a row in your csv spreadsheet file. CSV stands for comma separated values, however you can specify the csv_field_separator character you used in your csv files here. If you leave this option on auto, the Plugin will try to autodetect your csv field separator character.
-
-CSVFieldSeparator.metadata_value_separator: The character you've consistently used to separate multiple metadata values for a single metadata field within a cell of the csv spreadsheet. If you used the vertical bar as the separator character, then set metadata_value_separator to \| (backslash vertical bar).
+CSVFieldSeparator.csv_field_separator:The character you've consistently used to seperate each cell of a row in your csv spreadsheet file. CSV stands for comma separated values, however you can specify the csv_field_separator character you used in your csv files here. If you leave this option on auto, the Plugin will try to autodetect your csv field separator character.
+
+CSVFieldSeparator.metadata_value_separator:The character you've consistently used to separate multiple metadata values for a single metadata field within a cell of the csv spreadsheet. If you used the vertical bar as the separator character, then set metadata_value_separator to \| (backslash vertical bar).
+
+CSVFieldSeparator.metadata_separate_fields:A comma separated list of metadata fields that the metadata_value_separator is to be applied to.  If left blank then metadata_value_separator is applied to all the metadata fields in the CSV file.
 
 CSVPlugin.desc:A plugin for files in comma-separated value format. Metadata can be assigned to source documents (specified in the Filename field), or new documents created for each line of the file.