Changeset 32536 for main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm

Timestamp:

2018-10-24T20:11:33+13:00 (6 years ago)

Author:

ak19

Message:

First commit to do with reading back in from the SQL DB. This commit introduces the new GreenstoneSQLPlugin for this purpose, which should ideally only be used during buildcol (but its init(), deinit() and read() methods are also called on import.pl). The new plugin works with GreenstoneSQLPlugout which wrote meta and txt to the SQL DB. Lots of TODOs and questions still here, some debug statements too. Also have to run some decisions by Dr Bainbridge. There are many hardcoded values which still have to be parameterised (not always completely sure how) and still have to test the 2 cases of sending just meta and just fulltxt to db. Next commit will tidy some things up.

File:

• main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm (modified) (5 diffs)

main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm

@@ -40 +40 @@
 
 # TODO: SIGTERM rollback and disconnect?
-
+# TODO Q: what about verbosity for debugging
 
 # this plugout does not output xml to a file, but outputs rows into a mysql table
@@ -146 +146 @@
     if(!$gs_sql->load_db_and_tables($db_name, $build_mode)) {
     
-    # This is fatal for the plugout, let's terminate here
-    # PrintError would already have displayed the warning message on connection fail    
-    die("Could not use db or prepare its tables. Can't proceed.\n");
+    # This is fatal for the plugout, let's terminate here after disconnecting again
+    # PrintError would already have displayed the warning message on load fail
+    $gs_sql->disconnect_from_db()
+        || warn("Unable to disconnect from database " . $self->{'site_name'} . "\n");
+    die("Could not use db $db_name and/or prepare its tables. Can't proceed.\n");
     }
 
@@ -184 +186 @@
     $self->{'gs_sql'}->disconnect_from_db() || warn("Unable to disconnect from database " . $self->{'site_name'} . "\n"); # disconnect_from_db() will also issue a warning, but this may be clearer
 }
- 
-# produce files called docsql.xml instead of doc.xml
-sub get_short_doc_file {
+
+# produce files called docsql-<OID>.xml instead of doc.xml
+sub get_doc_xml_filename {
     my $self = shift (@_);
-    my ($doc_dir) = @_;
-    return &FileUtils::filenameConcatenate($doc_dir, "docsql.xml");
-}
-
-# produce files called docsql.xml instead of doc.xml
-sub get_output_file {
-    my $self = shift (@_);
-    my ($doc_dir) = @_;
-    return &FileUtils::filenameConcatenate($self->{'output_dir'}, $doc_dir, "docsql.xml");
+    my ($doc_obj) = @_;
+    
+    my $doc_id = $doc_obj->get_OID();
+    my $filename = $doc_id ? "docsql-".$doc_id.".xml" : "docsql.xml";
+    return $filename;
 }
    
@@ -204 +202 @@
     my ($doc_obj, $doc_dir) = @_;
 
-    print STDERR "\n\n@@@ In saveas\n\n";
+#    print STDERR "\n\n@@@ In saveas\n\n";
     
     my $proc_mode = $self->{'process_mode'};
@@ -291 +289 @@
     foreach my $data (@{$section_ptr->{'metadata'}}) {
         my $meta_name = $data->[0];
+        # TODO: does it need to be stored escaped, as it requires unescaping when read back in
+        # from db (unlike for reading back in from doc.xml)
         my $escaped_meta_value = &docprint::escape_text($data->[1]);