source: gs2-extensions/parallel-building/trunk/src/perllib

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @30354   8 years jmt12 Extending manifest v2 support to allow for directories to be listed in …
(edit) @30353   8 years jmt12 Replacing old file util calls with new FileUtil ones - including …
(edit) @30352   8 years jmt12 Adding in a function to test whether a File driver has a certain …
(edit) @30351   8 years jmt12 Restructured readDirectory to not die if directory isn't readable
(edit) @30350   8 years jmt12 Initial commit of rewrite so DBDrivers are object-oriented like …
(edit) @30305   9 years jmt12 replacing deprecated function calls to newer ones in FileUtils
(edit) @30295   9 years jmt12 Using the proper environment variable, GSDL3SRCHOME, rather than GSDL3HOME
(edit) @30292   9 years jmt12 Removing reference to debugging module Devel::Peek
(edit) @30289   9 years jmt12 Significant changes to read() function - essentially split in half …
(edit) @30288   9 years jmt12 No longer different that the vanilla Greenstone version
(edit) @30287   9 years jmt12 Extending error messages a bit to differentiate between linking that …
(edit) @30286   9 years jmt12 Adding a customized version of inexport.pm allowing us to handle …
(edit) @29660   9 years jmt12 making the debug variable global... can't remember why though
(edit) @29276   10 years jmt12 I need to measure the time spent on generating the initial manifest, …
(edit) @29261   10 years jmt12 Removing some of the extraneous IO from high cpu importing... altering …
(edit) @29260   10 years jmt12 Replacing the obsolete call to util::file_lastmodified() with the …
(edit) @29259   10 years jmt12 Kea override allowing for fixed processor affinity if necessary …
(edit) @29258   10 years jmt12 Initial checkin of a new TDB infodb that allows each worker thread in …
(edit) @29257   10 years jmt12 Allow for collection configuration to be passed down to parallel …
(edit) @29243   10 years jmt12 Allowing for file linking to be disabled
(edit) @29161   10 years jmt12 Some modules aren't availalbe on cluster... add test and include path …
(edit) @28779   10 years jmt12 Making timing message all sorts of purty
(edit) @28778   10 years jmt12 Typo - underscore where I meant hyphen
(edit) @28777   10 years jmt12 Need to include path to mpiimport on Medusa
(edit) @28771   10 years jmt12 A version of BasePlugout where the RSS feed update attempts to write …
(edit) @28770   10 years jmt12 Adding microtiming... a little tricky what with TDBServer taking …
(edit) @28654   10 years jmt12 Removed recordEarliestDatestamp() function as that no lurks in the …
(edit) @28653   10 years jmt12 Changed the way a require was 'eval'd - but I have no idea why
(edit) @28649   10 years jmt12 A version of a Textfile reading plugin that has a configurable load …
(edit) @28187   11 years jmt12 A customized version of Kea.pm that looks in the correct place for …
(edit) @28001   11 years jmt12 Write datestamp using dbutil if applicable
(edit) @27995   11 years jmt12 Just adding some code comments
(edit) @27915   11 years jmt12 A new PlugOut that doesn't write any intermediate files (bar those …
(edit) @27682   11 years jmt12 Copying makeAllDirectories() from vanilla FileUtils.pm
(edit) @27653   11 years jmt12 Forgot to pull self off the head of arguments
(edit) @27652   11 years jmt12 Changing buffer to 128K (slightly faster) and adding a comment …
(edit) @27638   11 years jmt12 Change it so failure to open a filehandle isn't fatal - leave it up to …
(edit) @27591   11 years jmt12 Ensure Thrift will, be default, attempt to connect to the local …
(edit) @27570   11 years jmt12 Make the warning about binmode() not being applicable more meaningful, …
(edit) @27569   11 years jmt12 Trying to streamline the error messages from failing to link …
(edit) @27568   11 years jmt12 Testing on Medusa suggests optimal buffer size around 128K
(edit) @27567   11 years jmt12 Found a printWarning that I handed changed to use the FileUtils version
(edit) @27547   11 years jmt12 Rejigging some processing comments
(edit) @27532   11 years jmt12 Add the ability to configure the Thrift connector using a …
(edit) @27531   11 years jmt12 Only output the message about using copy instead of hard/soft link once
(edit) @27526   11 years jmt12 Adding in a 'isHDFS()' function so that some plugins (SimpleVideoPlug) …
(edit) @27525   11 years jmt12 Adding in a 'isHDFS()' function so that some plugins (SimpleVideoPlug) …
(edit) @27514   11 years jmt12 Altering code to allow configurable length of read/write buffer when …
(edit) @27496   11 years jmt12 Replacing a smelly old util::file_exists() with a snazzy new …
(edit) @27493   11 years jmt12 No longer required - not that sure why it was required in the first place
(edit) @27492   11 years jmt12 Some versions of Hadoop add host and protocol information into paths - …
(edit) @27491   11 years jmt12 Repairing three bugs in makeAllDirectories - incorrect pattern meant …
(edit) @27490   11 years jmt12 No longer requires
(edit) @27481   11 years jmt12 Adding makeAllDirectories() (which I'd only implemented in LocalFS) to …
(edit) @27479   11 years jmt12 Remove time parsing as DateTime is a fricking nightmare to install …
(edit) @27478   11 years jmt12 Be a bit smarter about locating Perl version if not provided (rather …
(edit) @27424   11 years jmt12 Adding canRead() and isAbsolute() functions, and some more debugging …
(edit) @27423   11 years jmt12 Adding canRead() and isAbsolute() functions
(edit) @27422   11 years jmt12 Adding canRead() and isAbsolute() functions
(edit) @27421   11 years jmt12 Fixing bug in filePutContents (wasn't shifting parameters off front of …
(edit) @27386   11 years jmt12 Forgot these were just symbolic links to my Dropbox folder - adding in …
(edit) @27385   11 years jmt12 removing soft links to my dropbox folder - sorry
(edit) @27384   11 years jmt12 An overriding version of FileUtils that provides several drivers …
(edit) @27383   11 years jmt12 Adding a perl module for the slightly more efficient Base91 encoding …
(edit) @27377   11 years jmt12 Updating calls to intermediate util functions to the new FileUtils
(edit) @27373   11 years jmt12 No longer needs a custom util.pm - file utility functions now moved …
(edit) @27372   11 years jmt12 Fixing where I'd used uppercase letters in a package name that didn't …
(edit) @27371   11 years jmt12 Replacing with a proper class inheriting from gdbm and just overriding …
(edit) @27370   11 years jmt12 fixing minor bug in inheritence tree
(edit) @27280   11 years jmt12 A subclass of buildcolutils.pm with added functionality for parallel …
(edit) @27279   11 years jmt12 A subclass of inexport.pm with added functionality for parallel …
(edit) @27278   11 years jmt12 Renamed so it can be found in an automated fashion (prefixed by the …
(edit) @27237   11 years jmt12 Moved several generally applicable changes (manifest version, …
(edit) @27236   11 years jmt12 extended loader (to support extensions) and conditional additions to …
(edit) @27235   11 years jmt12 Moving the conditional additions to INC into the main trunk's version
(edit) @27234   11 years jmt12 renamed prepare_build_recipe() to the more meaningful …
(edit) @27233   11 years jmt12 Recipe generation code moved into subclass of mgbuilder called …
(edit) @27232   11 years jmt12 Recipe generation code moved into subclass of mgppbuilder called …
(edit) @27231   11 years jmt12 Recipe generation code moved into subclass of mgppbuilder called …
(edit) @27230   11 years jmt12 Recipe generation code moved into subclass of mgbuilder called …
(edit) @27229   11 years jmt12 A subclass of basebuilder that includes an 'abstract' function for the …
(edit) @27228   11 years jmt12 A subclass of lucenebuilder that includes a function to generate a …
(edit) @27227   11 years jmt12 The indexlevel processing code has been moved into the main trunk's …
(edit) @27226   11 years jmt12 Moved the version property into the main trunk's version
(edit) @27216   11 years jmt12 renaming (away from Java/PHP conventions) to make my life easier when …
(edit) @27215   11 years jmt12 Moved the conditional INC and PATH addition code into the main trunk
(edit) @27214   11 years jmt12 Copied the two new strings (buildcol.indexname and …
(edit) @27213   11 years jmt12 Unchanged from version in main trunk
(edit) @27212   11 years jmt12 No difference from version in main trunk (apart from some debugging …
(edit) @27211   11 years jmt12 The parallel processing changes (pretty much just the placeholder …
(edit) @27210   11 years jmt12 DavidB already ported the few minor changes (complexmeta, indexname, …
(edit) @27209   11 years jmt12 No difference from version in trunk
(edit) @27208   11 years jmt12 The only change was to ensure that Greenstone's 'perllib' and …
(edit) @27207   11 years jmt12 Classify had been edited to do a better job of searching for …
(edit) @27206   11 years jmt12 The only real change in doc.pm was the replacement of a -e with a call …
(edit) @27205   11 years jmt12 Unchanged from base perllib version
(edit) @27204   11 years jmt12 Unchanged from base perllib version
(edit) @27203   11 years jmt12 SQLite support for Write Ahead Logging (offering basic parallel …
(edit) @27202   11 years jmt12 Unchanged from base perllib version
(edit) @27201   11 years jmt12 Should never have been checked in as I made no changes here - caused …
Note: See TracRevisionLog for help on using the revision log.