source: other-projects/hathitrust/wcsa

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @31161   6 years davidb Comparison of local disk version with HDFS version
(edit) @31152   6 years davidb Development of script
(edit) @31151   6 years davidb More nuanced version to help finish off the 'big put'
(edit) @31128   6 years davidb Some scripts to help with pushing and monitoring the progress of the …
(edit) @31112   6 years davidb To move out shards saved in /tmp on gsliscluter1 nodes to nema
(edit) @31106   6 years davidb Scripts to help run an rsync'd copy of gslistcluster1 …
(edit) @31105   6 years davidb Additional scripts to help with running solr locally out of /tmp area
(edit) @31104   6 years davidb now configurable to be run from local disk (/tmp)
(edit) @31103   6 years davidb Changes made after testing with 20 solr nodes
(edit) @31102   6 years davidb Command line way of running a Solr test query
(edit) @31101   6 years davidb Correction to collection name
(edit) @31100   6 years davidb Change to using solr-cloud-nodes that include port number
(edit) @31099   6 years davidb Changes resulting from test runs to get Zookeeper and Solr running on …
(edit) @31098   6 years davidb Changes resulting from test runs to get Zookeeper and Solr running on …
(edit) @31097   6 years davidb Changed to .in style namne
(edit) @31096   6 years davidb Only need to create a volume's pages output directory is _output_dir …
(edit) @31095   6 years davidb Introduced num-partitions property
(edit) @31094   6 years davidb Changes triggered by running on gsliscluster1
(edit) @31093   6 years davidb Changes triggered by running on gsliscluster1
(edit) @31092   6 years davidb Minor tweak to spark/hadoop combination downloaded
(edit) @31091   6 years davidb Change of number of core for 'gsliscluster1' machine; commmented out …
(edit) @31090   6 years davidb Memory monitor debugging code, commented out
(edit) @31089   6 years davidb Change in way the JSON file is read in. Motivation was a …
(edit) @31088   6 years davidb Shift to newIstance for FileSystem due to StackOverflow page …
(edit) @31082   6 years davidb Changes in response to testing on gchead
(edit) @31081   6 years davidb Going live with generation of spark slaves file
(edit) @31080   6 years davidb echo formatting tidy up. Fixed some typos
(edit) @31079   6 years davidb Useful get started scripts
(edit) @31078   6 years davidb Some setup files and scripts to make running Spark and Solr easier on …
(edit) @31077   6 years davidb Move up to JDK1.8. Tidy up of Vagrant machine names. Support for YARN. …
(edit) @31065   6 years davidb Additional echo output
(edit) @31062   6 years davidb Added in -W option so check-sum calculation is skipped
(edit) @31058   6 years davidb echo for additional information added
(edit) @31057   6 years davidb Tweak to jps output formatting
(edit) @31053   6 years davidb Addition of second argument, optional, for where to save the files
(edit) @31051   6 years davidb Added in JDK to list of possible packages needed
(edit) @31046   6 years davidb Added property to control how severe a JSON IO problem is
(edit) @31045   6 years davidb More careful treatment of what to do when a JSON file isn't there
(edit) @31044   6 years davidb Fixed up error when output_dir is empty
(edit) @31043   6 years davidb Version for processing full EF set
(edit) @31042   6 years davidb Name changes, preparing the way for FULL-RUN versions
(edit) @31041   6 years davidb Test needs to be more careful if -read-only specified
(edit) @31036   6 years davidb Renaming to prepare way for YARN version of script
(edit) @31035   6 years davidb Changes after testing scripts
(edit) @31034   6 years davidb Development of scripts for working with Full EF dataset
(edit) @31033   6 years davidb Development of scripts for working with Full EF dataset
(edit) @31030   6 years davidb Tweak to some verbosity level 2 printing
(edit) @31029   6 years davidb Newline at end of file added
(edit) @31028   6 years davidb Support for randonly choosing Solr endpoints added in
(edit) @31027   6 years davidb Mixed typo in property name used
(edit) @31026   6 years davidb Corrected flag setting
(edit) @31025   6 years davidb Use property process-json-mode to determine which sort of Spark …
(edit) @31024   6 years davidb Support for Java properties file
(edit) @31022   6 years davidb No longer used
(edit) @31021   6 years davidb Folder restructure to remove 'trunk' part
(edit) @31020   6 years davidb No longer used
(edit) @31019   6 years davidb Part 2 or two-step folder restructure
(edit) @31018   6 years davidb Part 1 or two-step folder restructure
(edit) @31017   6 years davidb Moved to correct position
(add) @31015   6 years davidb Restructuring of projects into one
Note: See TracRevisionLog for help on using the revision log.