Changeset 31975 for main

Timestamp:

2017-09-19T18:59:51+12:00 (7 years ago)

Author:

ak19

Message:

Another bugfix to downloading. Downloading over OAI wasn't working. Dr Bainbridge discovered that this was because OAIDownload.pm was still doublequoting filepaths and URLs too, whereas open3() launching the wget cmd can't handle quotes in its arguments. WgetDownload used split to convert the cmd string into a cmd array. A clean solution was not passing WgetDownload::useWget() methods an array of cmd parameters (too involved and error prone to change all the calling code constructing the parameter cmd string), but to use the quotewords() method in place of split. This will preserve spaces in double quoted params in the cmd string, while splitting on spaces outside quoted strings. Then it also removes double quotes (and unescapes double backslashes). Tested on Mac: both OAI and Web downloading now work.

File:

• main/trunk/greenstone2/perllib/downloaders/WgetDownload.pm (modified) (3 diffs)

main/trunk/greenstone2/perllib/downloaders/WgetDownload.pm

@@ -40 +40 @@
 use IO::Select;
 use IO::Socket;
+use Text::ParseWords; # part of Core modules. Needed to use quotewords() subroutine
 
 #use IO::Select qw( );
@@ -368 +369 @@
     # http://www.perlmonks.org/?node_id=394709 and that ends up causing problems in terminating wget, as 2 processes
     # got launched then which don't have parent-child pid relationship (so that terminating one doesn't terminate the other).
-    my @commandargs = split(' ', $cmdWget);
-    unshift(@commandargs, $wget_file_path);
-    $command = "$wget_file_path $cmdWget";
-#    print STDOUT "Command is: $command\n"; # displayed in GLI output
-#    print STDERR "Command is: $command\n"; # goes into ServerInfoDialog
+
+    # remove leading and trailing spaces, https://stackoverflow.com/questions/4597937/perl-function-to-trim-string-leading-and-trailing-whitespace
+    $cmdWget =~ s/^\s+//;
+    $cmdWget =~ s/\s+$//;
+
+    # split on "words"
+    #my @commandargs = split(' ', $cmdWget);
+    # quotewords: to split on spaces except within quotes, then removes quotes and unescapes double backslash too 
+      # https://stackoverflow.com/questions/19762412/regex-to-split-key-value-pairs-ignoring-space-in-double-quotes
+      # https://docstore.mik.ua/orelly/perl/perlnut/c08_389.htm
+    my @commandargs = quotewords('\s+', 0, $cmdWget);
+    unshift(@commandargs, $wget_file_path); # prepend the wget cmd
+    #print STDERR "Command is: ".join(",", @commandargs) . "\n"; # goes into ServerInfoDialog
     
     # Wget's output needs to be monitored to find out when it has naturally terminated.
@@ -639 +648 @@
     my $wget_file_path = &FileUtils::filenameConcatenate($ENV{'GSDLHOME'}, "bin", $ENV{'GSDLOS'}, "wget");
     # compose the command as an array for open3, to preserve spaces in any filepath
-    my @commandargs = split(' ', $cmdWget);
-    unshift(@commandargs, $wget_file_path);
-    my $command = "$wget_file_path $cmdWget";
-    #print STDOUT "Command is: $command\n";
+    # Do so by removing leading and trailing spaces, then splitting on "words" (preserving spaces in quoted words and removing quotes)
+    $cmdWget =~ s/^\s+//;
+    $cmdWget =~ s/\s+$//;
+    my @commandargs = quotewords('\s+', 0, $cmdWget);
+    unshift(@commandargs, $wget_file_path); # prepend wget cmd to the command array 
+    #print STDOUT "Command is: ".join(",", @commandargs) . "\n";
 
     eval {     # see p.568 of Perl Cookbook