Changeset 32582 for main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm

Timestamp:

2018-11-07T20:44:34+13:00 (5 years ago)

Author:

ak19

Message:

Now that previous commit(s) put sig handlers in place in gs_sql, have been able to add in Undo on build/import Cancel for the GS SQL Plugs. This utilizes AutoCommit vs Transaction (rollback/commit) behaviour. On cancel, a sig handler is triggered (SIGINT) and, if AutoCommit is off, does a rollback before die() which calls object destructor and disconnects from db. On regular program execution running to normal termination, the last finish() call on gs_sql that will trigger the disconnect, will now first do a commit(), if AutoCommit is off, before disconnecting. For now, the default for both GreenstoneSQLPlugs is to support Undo (i.e. transactions), which turns AutoCommit off. Not sure whether this will be robust: what if transactions take place in memory, we could be dealing with millions of docs of large full-txt. Another issue is that the SQL DB may be out of sync with archives and index folder on Cancel: archives and index just terminate and are in an intermediate state depending on when cancel was pressed. Whereas the GS SQL DB is in a rolled back state as if the import or build never took place. A third issue is that during buildcol (perhaps specifically during buildcol's doc processing phase), pressing Cancel still continues buildcol: the current perl process is cancelled but the next one continues, rather than terminating buildcol in entirety. What happens with the GS SQL DB is that any 'transaction' until then is rolled back, perhaps a transaction regarding one doc if the Cancel affects on a doc basis, and the next process (next doc processing?) continues and allows for further transactions that are all committed at the end on natural termination of buildcol. Need to whether Undo behavious is really what we want. But it's available now and we can simply change the default to not support Undo if we want the old behaviour again.

File:

• main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm (modified) (6 diffs)

main/trunk/greenstone2/perllib/plugouts/GreenstoneSQLPlugout.pm

@@ -38 +38 @@
 use DBI; # the central package for this plugout
 
+
+# This entire class is called only during import.pl
 
 # TODO: SIGTERM rollback and disconnect?
@@ -69 +71 @@
         'desc' => "{GreenstoneSQLPlug.process_mode.all}" } ];
 
+my $rollback_on_cancel_list =
+    [ { 'name' => "true",
+        'desc' => "{GreenstoneSQLPlug.rollback_on_cancel}" },      
+      { 'name' => "false",
+        'desc' => "{GreenstoneSQLPlug.rollbacl_on_cancel}" } ];
+
 # The following are the saveas.options:
 my $arguments = [ 
@@ -76 +84 @@
       'list' => $process_mode_list,
       'deft' => "all",
+      'reqd' => "no",
+      'hiddengli' => "no"},
+    { 'name' => "rollback_on_cancel", 
+      'desc' => "{GreenstoneSQLPlug.rollback_on_cancel}",
+      'type' => "enum",
+      'list' => $rollback_on_cancel_list,
+      'deft' => "true", # TODO Q: what's the better default? If "true", any memory concerns?
       'reqd' => "no",
       'hiddengli' => "no"},
@@ -146 +161 @@
     'collection_name' => $ENV{'GSDLCOLLECTION'},
     'verbosity' => $self->{'verbosity'} || 0
+    
     };
 
     my $gs_sql = new gssql($db_params);
+
+    # if autocommit is set, there's no rollback support
+    my $autocommit = ($self->{'rollback_on_cancel'} eq "false") ? 1 : 0;
     
     # try connecting to the mysql db, die if that fails
@@ -156 +175 @@
     'db_client_user' => $self->{'db_client_user'},
     'db_client_pwd' => $self->{'db_client_pwd'},
-    'db_host' => $self->{'db_host'}
+    'db_host' => $self->{'db_host'},
+    'autocommit' => $autocommit
                    })
     )
@@ -164 +184 @@
     die("Could not connect to db. Can't proceed.\n");
     }
+
+    #die("@@@@ TEST. Connected successfully. Testing gssql::destructor.\n"); # WORKS
     
     my $db_name = $self->{'site_name'} || "greenstone2"; # one database per GS3 site, for GS2 the db is called greenstone2