Changeset 32582 for main/trunk/greenstone2/perllib/plugins/GreenstoneSQLPlugin.pm

Timestamp:

2018-11-07T20:44:34+13:00 (5 years ago)

Author:

ak19

Message:

Now that previous commit(s) put sig handlers in place in gs_sql, have been able to add in Undo on build/import Cancel for the GS SQL Plugs. This utilizes AutoCommit vs Transaction (rollback/commit) behaviour. On cancel, a sig handler is triggered (SIGINT) and, if AutoCommit is off, does a rollback before die() which calls object destructor and disconnects from db. On regular program execution running to normal termination, the last finish() call on gs_sql that will trigger the disconnect, will now first do a commit(), if AutoCommit is off, before disconnecting. For now, the default for both GreenstoneSQLPlugs is to support Undo (i.e. transactions), which turns AutoCommit off. Not sure whether this will be robust: what if transactions take place in memory, we could be dealing with millions of docs of large full-txt. Another issue is that the SQL DB may be out of sync with archives and index folder on Cancel: archives and index just terminate and are in an intermediate state depending on when cancel was pressed. Whereas the GS SQL DB is in a rolled back state as if the import or build never took place. A third issue is that during buildcol (perhaps specifically during buildcol's doc processing phase), pressing Cancel still continues buildcol: the current perl process is cancelled but the next one continues, rather than terminating buildcol in entirety. What happens with the GS SQL DB is that any 'transaction' until then is rolled back, perhaps a transaction regarding one doc if the Cancel affects on a doc basis, and the next process (next doc processing?) continues and allows for further transactions that are all committed at the end on natural termination of buildcol. Need to whether Undo behavious is really what we want. But it's available now and we can simply change the default to not support Undo if we want the old behaviour again.

File:

• main/trunk/greenstone2/perllib/plugins/GreenstoneSQLPlugin.pm (modified) (8 diffs)

main/trunk/greenstone2/perllib/plugins/GreenstoneSQLPlugin.pm

@@ -137 +137 @@
         'desc' => "{GreenstoneSQLPlug.process_mode.all}" } ];
 
+my $rollback_on_cancel_list =
+    [ { 'name' => "true",
+        'desc' => "{GreenstoneSQLPlug.rollback_on_cancel}" },      
+      { 'name' => "false",
+        'desc' => "{GreenstoneSQLPlug.rollbacl_on_cancel}" } ];
+
 my $arguments =
     [ { 'name' => "process_exp",
@@ -149 +155 @@
     'deft' => "all",
     'reqd' => "no"},
+      { 'name' => "rollback_on_cancel", 
+    'desc' => "{GreenstoneSQLPlug.rollback_on_cancel}",
+    'type' => "enum",
+    'list' => $rollback_on_cancel_list,
+    'deft' => "true", # TODO Q: what's the better default? If "true", any memory concerns?
+    'reqd' => "no",
+    'hiddengli' => "no"},
       { 'name' => "db_driver", 
     'desc' => "{GreenstoneSQLPlug.db_driver}",
@@ -204 +217 @@
 }
 
+###### Called during import.pl
+
 # This is called once if removeold is set with import.pl. Most plugins will do
 # nothing but if a plugin does any stuff outside of creating doc obj, then 
@@ -230 +245 @@
     $gs_sql->ensure_fulltxt_table_exists();
     }
-}
-
-# This is called per document for docs that have been deleted from the 
+
+    # UNNECESSARY
+    # The removeold related DB transaction (deleting collection tables) is complete
+    # Don't let GS SQL PlugIN interfere with GS SQL PlugOUT's database transactions
+    # during import.pl hereafter. Finish up.
+    #$gs_sql->do_commit_if_on();
+}
+
+# This is called during import.pl per document for docs that have been deleted from the 
 # collection. Most plugins will do nothing
 # but if a plugin does any stuff outside of creating doc obj, then it may need
@@ -273 +294 @@
 }
 
+#### Called during buildcol
 
 sub xml_start_tag {
@@ -395 +417 @@
 }
 
+#### Called during buildcol and import
 
 # GS SQL Plugin::init() (and deinit()) is called by import.pl and also by buildcol.pl
@@ -429 +452 @@
     'verbosity' => $self->{'verbosity'} || 0
                });
+    
+    # if autocommit is set, there's no rollback support
+    my $autocommit = ($self->{'rollback_on_cancel'} eq "false") ? 1 : 0;
 
     # try connecting to the mysql db, die if that fails
@@ -435 +461 @@
     'db_client_user' => $self->{'db_client_user'},
     'db_client_pwd' => $self->{'db_client_pwd'},
-    'db_host' => $self->{'db_host'}
+    'db_host' => $self->{'db_host'},
+    'autocommit' => $autocommit
                    })
     )