|
|
@28666
|
11 years |
jmt12 |
A script to transform a strace.out into a Tab separated file worthy of …
|
|
|
@28665
|
11 years |
jmt12 |
Latest changes to workaround resumed syscalls massive duration problem
|
|
|
@28654
|
11 years |
jmt12 |
Removed recordEarliestDatestamp() function as that no lurks in the …
|
|
|
@28653
|
11 years |
jmt12 |
Changed the way a require was 'eval'd - but I have no idea why
|
|
|
@28652
|
11 years |
jmt12 |
Changes to support running the reports over logs produced from …
|
|
|
@28649
|
11 years |
jmt12 |
A version of a Textfile reading plugin that has a configurable load …
|
|
|
@28648
|
11 years |
jmt12 |
Adding a short delay after writing to the flush_cache file just to …
|
|
|
@28647
|
11 years |
jmt12 |
Adding progress messages and making a debug message optional
|
|
|
@28646
|
11 years |
jmt12 |
A script that uses strace to produce IO metrics of a Greenstone import
|
|
|
@28645
|
11 years |
jmt12 |
Script to generate a report on data locality from GreenstoneHadoop logs
|
|
|
@28358
|
11 years |
jmt12 |
Replacing my earlier decision to only have data locality information …
|
|
|
@28357
|
11 years |
jmt12 |
used to update the data_locality.csv file in the case where other …
|
|
|
@28356
|
11 years |
jmt12 |
Support the legacy version of taskno in the data_locality.csv file (we …
|
|
|
@28312
|
11 years |
jmt12 |
Working on finer control over data locality - so I can configure a run …
|
|
|
@28192
|
11 years |
jmt12 |
Need to still output Greenstone messages to log otherwise I can't …
|
|
|
@28191
|
11 years |
jmt12 |
Removing redundant error stream redirect - this wasn't causing the …
|
|
|
@28190
|
11 years |
jmt12 |
Had accidently hardcoded the max replication number - allow it to be …
|
|
|
@28189
|
11 years |
jmt12 |
Replace the newer (and faster) while(@file) loop with the older (and …
|
|
|
@28188
|
11 years |
jmt12 |
Minor fix to allow for tasks that start in the same second (now each …
|
|
|
@28187
|
11 years |
jmt12 |
A customized version of Kea.pm that looks in the correct place for …
|
|
|
@28186
|
11 years |
jmt12 |
A (failed) attempt to use the unix iotop tool to determine IO percentage
|
|
|
@28018
|
11 years |
jmt12 |
Try really hard to capture the output from 'time' function as Medusa …
|
|
|
@28017
|
11 years |
jmt12 |
Forgot to add processing comment before call to hadoop_import.pl
|
|
|
@28016
|
11 years |
jmt12 |
Allow the hadoop report generator to parse start and end times …
|
|
|
@28015
|
11 years |
jmt12 |
Add an extra option that allows me to pass in the directory to write …
|
|
|
@28014
|
11 years |
jmt12 |
Remove tasks that have had data locality established from the array of …
|
|
|
@28013
|
11 years |
jmt12 |
A new script to run a battery of Hadoop ingests at varying replication …
|
|
|
@28012
|
11 years |
jmt12 |
Express start time as a double as well
|
|
|
@28011
|
11 years |
jmt12 |
Turn off debugging in the copy in SVN
|
|
|
@28010
|
11 years |
jmt12 |
Correctly set up the environment for calls to txt2tdb and also replace …
|
|
|
@28001
|
11 years |
jmt12 |
Write datestamp using dbutil if applicable
|
|
|
@27996
|
11 years |
jmt12 |
A new version of the archive with minor changes to log4j configuration
|
|
|
@27995
|
11 years |
jmt12 |
Just adding some code comments
|
|
|
@27915
|
11 years |
jmt12 |
A new PlugOut that doesn't write any intermediate files (bar those …
|
|
|
@27914
|
11 years |
jmt12 |
Trying to get around a couple of divide-by-zero issues when generating …
|
|
|
@27913
|
11 years |
jmt12 |
Made the ingester to be used (version 1 without reduce phase, or …
|
|
|
@27912
|
11 years |
jmt12 |
Modified the compilation to include the new ingester and its co-requisites.
|
|
|
@27911
|
11 years |
jmt12 |
Modified the compilation to include the new ingester and its co-requisites
|
|
|
@27910
|
11 years |
jmt12 |
Extended the existing HadoopGreenstoneIngest with proper Reduce phase …
|
|
|
@27753
|
11 years |
jmt12 |
Adding Handbrake's percentage complete to report - although this is …
|
|
|
@27752
|
11 years |
jmt12 |
Data locality file not being found is no longer fatal (HDFS-NFS-Proxy …
|
|
|
@27732
|
11 years |
jmt12 |
Nice the copy itself too
|
|
|
@27686
|
11 years |
jmt12 |
A little more progress comments
|
|
|
@27685
|
11 years |
jmt12 |
in the case of multiple attempts you need to retain the information …
|
|
|
@27684
|
11 years |
jmt12 |
Adding natural sorting into report generation - so also needed to add …
|
|
|
@27683
|
11 years |
jmt12 |
moving a few more headings around to help with information block layout
|
|
|
@27682
|
11 years |
jmt12 |
Copying makeAllDirectories() from vanilla FileUtils.pm
|
|
|
@27669
|
11 years |
jmt12 |
Sort compute nodes naturally before labelling them with incremental …
|
|
|
@27654
|
11 years |
jmt12 |
Add the ability to stagger the starting of Mappers by placing a …
|
|
|
@27653
|
11 years |
jmt12 |
Forgot to pull self off the head of arguments
|
|
|
@27652
|
11 years |
jmt12 |
Changing buffer to 128K (slightly faster) and adding a comment …
|
|
|
@27651
|
11 years |
jmt12 |
|
|
|
@27650
|
11 years |
jmt12 |
|
|
|
@27649
|
11 years |
jmt12 |
No longer in SVN control
|
|
|
@27648
|
11 years |
jmt12 |
Template for setup.bash - a user will have to populate Hadoop fields
|
|
|
@27645
|
11 years |
jmt12 |
|
|
|
@27644
|
11 years |
jmt12 |
Extended to support HDFS-access via NFS. This applies to both the call …
|
|
|
@27643
|
11 years |
jmt12 |
Changed the script generator so it can recurse through directories and …
|
|
|
@27642
|
11 years |
jmt12 |
A script I downloaded that successfully splits video files - something …
|
|
|
@27641
|
11 years |
jmt12 |
Altered order of arguments and allow archives dir to be passed as …
|
|
|
@27640
|
11 years |
jmt12 |
|
|
|
@27638
|
11 years |
jmt12 |
Change it so failure to open a filehandle isn't fatal - leave it up to …
|
|
|
@27631
|
11 years |
jmt12 |
A proxy to allow NFS access to HDFS
|
|
|
@27595
|
11 years |
jmt12 |
Updating list of untarred directories to ignore
|
|
|
@27594
|
11 years |
jmt12 |
Extend hadoop_import.pl to be able to start and stop the Thrift server(s)
|
|
|
@27593
|
11 years |
jmt12 |
Need Class Accessor for Thrift client under Rocks
|
|
|
@27592
|
11 years |
jmt12 |
Adding in a script to allow a daemon version of Thrift to be started …
|
|
|
@27591
|
11 years |
jmt12 |
Ensure Thrift will, be default, attempt to connect to the local …
|
|
|
@27590
|
11 years |
jmt12 |
Adding statistics about data locality, and highlighting tasks where …
|
|
|
@27589
|
11 years |
jmt12 |
Fixing up some minor bugs in regex's
|
|
|
@27588
|
11 years |
jmt12 |
Extend parser to support jobs that are split over several logs. Also …
|
|
|
@27587
|
11 years |
jmt12 |
Allow debug mode to be enabled from the command line
|
|
|
@27586
|
11 years |
jmt12 |
Updating script to date date of hadoop job into account when searching …
|
|
|
@27585
|
11 years |
jmt12 |
The perl on Medusa won't let you immediately treat a returned array in …
|
|
|
@27584
|
11 years |
jmt12 |
I wasn't doing -r when attempting to clear directories left in /tmp by …
|
|
|
@27583
|
11 years |
jmt12 |
Adding code to differentiate between workers in a cluster - all of …
|
|
|
@27571
|
11 years |
jmt12 |
increase timeout to 4 hours per map
|
|
|
@27570
|
11 years |
jmt12 |
Make the warning about binmode() not being applicable more meaningful, …
|
|
|
@27569
|
11 years |
jmt12 |
Trying to streamline the error messages from failing to link …
|
|
|
@27568
|
11 years |
jmt12 |
Testing on Medusa suggests optimal buffer size around 128K
|
|
|
@27567
|
11 years |
jmt12 |
Found a printWarning that I handed changed to use the FileUtils version
|
|
|
@27566
|
11 years |
jmt12 |
Making the getcpu optional - as it isn't available on Medusa (but then …
|
|
|
@27561
|
11 years |
jmt12 |
Adding very basic compile file for getcpu - can't be bothered going …
|
|
|
@27560
|
11 years |
jmt12 |
Fixing typo in regexp that meant filenames sometimes ignored
|
|
|
@27559
|
11 years |
jmt12 |
Changed mime-type away from binary - I hope. Meanwhile, generate …
|
|
|
@27558
|
11 years |
jmt12 |
Forgot that Hadoop Map processes no longer have the environment …
|
|
|
@27551
|
11 years |
jmt12 |
Altered so that it expects to be given a CSV containing parallel …
|
|
|
@27550
|
11 years |
jmt12 |
Ensure the hostname is added to the Hadoop logs so we can identify the …
|
|
|
@27549
|
11 years |
jmt12 |
Extract information from the logs generated by parallel Greenstone …
|
|
|
@27548
|
11 years |
jmt12 |
Extract information from the logs generated by parallel Greenstone …
|
|
|
@27547
|
11 years |
jmt12 |
Rejigging some processing comments
|
|
|
@27546
|
11 years |
jmt12 |
Adding the ability for the Hadoop Mapper to determine what CPU number …
|
|
|
@27545
|
11 years |
jmt12 |
Ignoring just the compiled file (for now)
|
|
|
@27544
|
11 years |
jmt12 |
A tiny C script to guesstimate the CPU the calling Process is on
|
|
|
@27543
|
11 years |
jmt12 |
Adding generate_gantt.pl script in its original form - i.e. directly …
|
|
|
@27532
|
11 years |
jmt12 |
Add the ability to configure the Thrift connector using a …
|
|
|
@27531
|
11 years |
jmt12 |
Only output the message about using copy instead of hard/soft link once
|
|
|
@27530
|
11 years |
jmt12 |
Clear out old logs, and adding more comments about what the script is …
|
|
|
@27526
|
11 years |
jmt12 |
Adding in a 'isHDFS()' function so that some plugins (SimpleVideoPlug) …
|
|
|
@27525
|
11 years |
jmt12 |
Adding in a 'isHDFS()' function so that some plugins (SimpleVideoPlug) …
|
|
|