source: gs2-extensions/parallel-building

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @27207   11 years jmt12 Classify had been edited to do a better job of searching for …
(edit) @27206   11 years jmt12 The only real change in doc.pm was the replacement of a -e with a call …
(edit) @27205   11 years jmt12 Unchanged from base perllib version
(edit) @27204   11 years jmt12 Unchanged from base perllib version
(edit) @27203   11 years jmt12 SQLite support for Write Ahead Logging (offering basic parallel …
(edit) @27202   11 years jmt12 Unchanged from base perllib version
(edit) @27201   11 years jmt12 Should never have been checked in as I made no changes here - caused …
(edit) @27200   11 years jmt12 A subclass of basebuilder that provides a placeholder function for the …
(edit) @27195   11 years davidb Not changed for this extension
(edit) @27194   11 years davidb Not changed for this extension
(edit) @27193   11 years davidb Not changed for this extension
(edit) @27184   11 years davidb Removing 'downloaders' from parallel importing extension, as there are …
(edit) @27179   11 years davidb Mods to allow code to run with Greenstone3
(edit) @27177   11 years davidb Test for and create (if needed) the 'tmp' directory
(edit) @27126   11 years jmt12 Extra clean up commands (like removing cached versions of video …
(edit) @27125   11 years jmt12 A script to try and flush all caches - I'm certain it's flushing disk …
(edit) @27124   11 years jmt12 Use the new perl version script to extract the version number - so as …
(edit) @27119   11 years jmt12 Merging version finder from Medusa with the one lurking on Karearea
(edit) @27102   11 years jmt12 Adding in a progress reporter, as the set timeout to 10 hours trick …
(edit) @27058   11 years jmt12 Adding data locality report generation to Hadoop greenstone imports
(edit) @27052   11 years jmt12 Turns out the Perl on Medusa doesn't support $V, so I've had to …
(edit) @27051   11 years jmt12 Trying to force Sort::Key to install to a sensible location on Medusa …
(edit) @27048   11 years jmt12 Ignoring untarred Hadoop and Sort::Key directories
(edit) @27047   11 years jmt12 Script to allow the Perl module Sort::Key to be compiled/installed as …
(edit) @27046   11 years jmt12 Adding in Sort::Key cascade makefile
(edit) @27045   11 years jmt12 Ignoring untarred Hadoop and Sort::Key directories
(edit) @27044   11 years jmt12 The Hadoop reporting stuff needs natural sorting
(edit) @27043   11 years jmt12 Sort appears to be platform specific, so I'll need to include it as a …
(edit) @27042   11 years jmt12 Sort appears to be platform specific, so I'll need to include it as a …
(edit) @27041   11 years jmt12 INC path now includes the installed extensions perl path (including …
(edit) @27040   11 years jmt12 A simple script that returns just the version number of Perl
(edit) @27039   11 years jmt12 Adding natural sort for Perl. Shared object files appear to be ignored …
(edit) @27038   11 years jmt12 Allow natural sorting in Perl
(edit) @27037   11 years jmt12 Allow natural sorting in Perl
(edit) @27036   11 years jmt12 A script to extract data locality and other task information from the …
(edit) @27006   11 years jmt12 A companion script to stop-hadoop-processes that just reports running …
(edit) @27005   11 years jmt12 Similar to stop-impt.pl, this script uses kill to stop runaway Hadoop …
(edit) @27004   11 years jmt12 A script to stop (using kill) a runaway import process and any related …
(edit) @27003   11 years jmt12 Adding some Hadoopy stuff to setup.bash - will need to be configured …
(edit) @27002   11 years jmt12 Fixing bug in maxdocs processing where I compared to 'file_count' …
(edit) @27001   11 years jmt12 Passing more environment variables (HADOOPPREFIX, HDFSHOST, HDFSPORT) …
(edit) @26999   11 years jmt12 Ensuring MPI binds to correct interface, and passing through …
(edit) @26998   11 years jmt12 Adding maxdocs variable, lots of debug comments, added some tests for …
(edit) @26987   11 years jmt12 Removing some debug comments I was using to track down why TDB support …
(edit) @26986   11 years jmt12 Commenting out some debug comments
(edit) @26985   11 years jmt12 Removing a couple of debuggy type flags to MPI just to make logs a …
(edit) @26984   11 years jmt12 Removing some debug comments
(edit) @26983   11 years jmt12 Changed the prefix in a log filename to a 'W' for worker rather than a …
(edit) @26982   11 years jmt12 Altered environment setup for subprocesses (due to Hadoop now moving …
(edit) @26981   11 years jmt12 ignoring compiled java classes (hgi_classes)
(edit) @26980   11 years jmt12 setting svnignore
(edit) @26979   11 years jmt12 Fixed up lingering environment variables from when I was going to make …
(edit) @26965   11 years jmt12 Makes use of util library functions (rather than perl built-in file …
(edit) @26964   11 years jmt12 Altered to use util functions rather than built-in perl file tests. …
(edit) @26963   11 years jmt12 Altered to use functions in util library when opening file handles to …
(edit) @26962   11 years jmt12 Altering to use functions in util library so as to support HDFS …
(edit) @26961   11 years jmt12 Several file handling functions extended to be aware of and support …
(edit) @26960   11 years jmt12 Allow plugouts from be loaded from within extensions (assuming no …
(edit) @26959   11 years jmt12 Adding the ability to include a version attribute in manifest files to …
(edit) @26958   11 years jmt12 Using the util library instead of built-in perl file tests
(edit) @26954   11 years jmt12 checking in Java code from Hadoop parallel processing projects... phew
(edit) @26953   11 years jmt12 Checking in the script rather than a symbolic link to the script :P
(edit) @26952   11 years jmt12 Accidentally checked in symbolic link rather than script
(edit) @26951   11 years jmt12 Adding in required enviroment details for Hadoop
(edit) @26950   11 years jmt12 The version of Hadoop on which Hadoop parallel processing work is …
(edit) @26949   11 years jmt12 Parallel import using Hadoop
(edit) @26947   11 years jmt12 empty directory removed
(edit) @26946   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26945   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26944   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26943   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26942   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26941   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26940   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26939   11 years jmt12 Replacing uses of TextFile with TextPlugin... don't know how the …
(edit) @26938   11 years jmt12 Identical to those in default perllib - should never have been checked-in
(edit) @26937   11 years jmt12 Identical to files in default perllib - should never have been checked-in
(edit) @26935   11 years jmt12 Identical to those in default perllib - should never have checked in
(edit) @26934   11 years jmt12 Renamed to gdbml to avoid confusion with non-locking version of GDBM …
(edit) @26933   11 years jmt12 Renamed to gdbml to avoid confusion with non-locking version of GDBM …
(edit) @26932   11 years jmt12 Altered all calls to built-in perl file tests to instead use util …
(edit) @26930   11 years jmt12 Randomized order of files, and added the ability to specify a maximum …
(edit) @26929   11 years jmt12 A script to comprehensively clean up a collection between imports... …
(edit) @26928   11 years jmt12 Ignoring the compiled executable
(edit) @26926   11 years jmt12 Adding the executable file to the ignore set
(edit) @26923   11 years jmt12 Generates a specficied-size subset of a larger import directory
(edit) @26922   11 years jmt12 [bugfix] missing double quote in call to greenstone import.pl
(edit) @26242   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26240   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26239   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26237   12 years jmt12 Allows the specification of a maxfiles - limiting the number of files …
(edit) @26194   12 years jmt12 Fixed bug where I wasn't reseting the number of files handled when …
(edit) @26188   12 years jmt12 Fixing bug where I passed the wrong number of arguments to a sprintf …
(edit) @26187   12 years jmt12 Adding the rest of parallel processing support for Terrier into SVN. …
(edit) @26072   12 years jmt12 Extended to notice and load mpi.conf file on clusters and to use full …
(edit) @26071   12 years jmt12 Randomizing the order files are imported and restoring debugging flags …
(edit) @26070   12 years jmt12 ignoring files using svnignore
(edit) @26069   12 years jmt12 Initial checkin of Terrier parallel processing code
(edit) @26068   12 years jmt12 cascade make files for the new terrier parallel processing executable
(edit) @25945   12 years jmt12 more debugging and a few tweaks - and between this and changes to the …
Note: See TracRevisionLog for help on using the revision log.