source: gs2-extensions/parallel-building/trunk/src/src/java

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @30354   8 years jmt12 Extending manifest v2 support to allow for directories to be listed in …
(edit) @28312   11 years jmt12 Working on finer control over data locality - so I can configure a run …
(edit) @28192   11 years jmt12 Need to still output Greenstone messages to log otherwise I can't …
(edit) @28012   11 years jmt12 Express start time as a double as well
(edit) @28011   11 years jmt12 Turn off debugging in the copy in SVN
(edit) @28010   11 years jmt12 Correctly set up the environment for calls to txt2tdb and also replace …
(edit) @27911   11 years jmt12 Modified the compilation to include the new ingester and its co-requisites
(edit) @27910   11 years jmt12 Extended the existing HadoopGreenstoneIngest with proper Reduce phase …
(edit) @27654   11 years jmt12 Add the ability to stagger the starting of Mappers by placing a …
(edit) @27641   11 years jmt12 Altered order of arguments and allow archives dir to be passed as …
(edit) @27571   11 years jmt12 increase timeout to 4 hours per map
(edit) @27566   11 years jmt12 Making the getcpu optional - as it isn't available on Medusa (but then …
(edit) @27558   11 years jmt12 Forgot that Hadoop Map processes no longer have the environment …
(edit) @27546   11 years jmt12 Adding the ability for the Hadoop Mapper to determine what CPU number …
(edit) @27494   11 years jmt12 Fixing a truncated comment - or maybe I never wrote an end to it…
(edit) @27487   11 years jmt12 Ensure Parallel Building path in environment (for ThriftFS) and that …
(edit) @27433   11 years jmt12 Adding a few more details to the Hadoop log to assist with later work
(edit) @27416   11 years jmt12 Forgot to ensure the manifests generated here are also version 2
(edit) @27413   11 years jmt12 Making the Hadoop processing aware that there are two different …
(edit) @27102   11 years jmt12 Adding in a progress reporter, as the set timeout to 10 hours trick …
(edit) @26982   11 years jmt12 Altered environment setup for subprocesses (due to Hadoop now moving …
(edit) @26981   11 years jmt12 ignoring compiled java classes (hgi_classes)
(edit) @26980   11 years jmt12 setting svnignore
(edit) @26979   11 years jmt12 Fixed up lingering environment variables from when I was going to make …
(add) @26954   11 years jmt12 checking in Java code from Hadoop parallel processing projects... phew
Note: See TracRevisionLog for help on using the revision log.