Changeset 24379 for main/trunk/greenstone2/perllib/downloaders/Z3950Download.pm

Timestamp:

2011-08-09T22:21:36+12:00 (13 years ago)

Author:

ak19

Message:

Fixed several bugs in Z3950 download. The main bug had to do with how the Library of Congress is returning records in XML format, instead of whichever format other sources have been returning and for which the code so far was set up to work with. Now it works with the XML returned, however, the MARC record returned (which is in XML) does not explode properly. Other bugs have to do with the Server Information button in GLI failing for Z3950 and an ugly 2-level nested folder structure being created to store the files containing records returned: we don't want a folder called http.

File:

• main/trunk/greenstone2/perllib/downloaders/Z3950Download.pm (modified) (4 diffs)

main/trunk/greenstone2/perllib/downloaders/Z3950Download.pm

@@ -251 +251 @@
         $strShow = "show $intStartNumber+$intRecordsLeft";
         $intRecordsLeft = 0;
-       
-           }
+        
+    }
     
     $strResponse .= $self->get($strShow,$numRecords);
-         
+
     if ($strResponse eq ""){
         print STDERR "<<ERROR: failed to get $numRecords records>>\n";
@@ -276 +276 @@
     my $strFileName = $self->generateFileName($intMaxRecords);
 
-    $strOutputDir  =~ s/"//g; #"
-
-    my $strOutputFile = &util::filename_cat($strOutputDir,$self->{'host'},"$strFileName.marc");
+    $strOutputDir =~ s/"//g; #"
+
+    # remove any http:// prefix from the hostname to generate folder-name to store records in
+    my $foldername = $self->{'host'};
+    $foldername =~ s@^http:\/\/(.*)@$1@;
+
+    my $strOutputFile = &util::filename_cat($strOutputDir,$foldername,"$strFileName.marc");
      # prepare subdirectory for record (if needed)
     my ($strSubDirPath,$unused) = $self->dirFileSplit($strOutputFile);
@@ -316 +320 @@
    while (my $strLine = <$output>)
    {
-   
        if ($strLine =~ m/Records: ([\d]*)/i ){ 
        $readRecord = 1;
        next;  
+       } elsif ($strLine =~ m/<record/i){ # XML output such as from Library of Congress
+       $strFullOutput .= $strLine;
+       $readRecord = 1;
+       next;
        }
       
-      return $strFullOutput if ($strLine =~ m/nextResultSetPosition|Not connected/i);
+       if ($strLine =~ m/\<\/record\>/i) { # end of XML, include the closing tag and then terminate below
+       $strFullOutput .= $strLine;
+       }
+
+      return $strFullOutput if ($strLine =~ m/nextResultSetPosition|Not connected|\<\/record\>/i);
         
       next if(!$readRecord);
@@ -389 +400 @@
    $url = $self->{'url'} unless defined $url;
 
-   my $strOpen = $self->start_yaz();
-
-   $strOpen = $self->run_command_with_output("open $url","^Options");  
+   my $strOpen = $self->start_yaz($url);
+
+   $strOpen = $self->run_command_with_output("open $url","^Options|Connecting...OK.");  
 
    $strOpen =~ s/Z> //g;