source: other-projects

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @31106   5 years davidb Scripts to help run an rsync'd copy of gslistcluster1 …
(edit) @31105   5 years davidb Additional scripts to help with running solr locally out of /tmp area
(edit) @31104   5 years davidb now configurable to be run from local disk (/tmp)
(edit) @31103   5 years davidb Changes made after testing with 20 solr nodes
(edit) @31102   5 years davidb Command line way of running a Solr test query
(edit) @31101   5 years davidb Correction to collection name
(edit) @31100   5 years davidb Change to using solr-cloud-nodes that include port number
(edit) @31099   5 years davidb Changes resulting from test runs to get Zookeeper and Solr running on …
(edit) @31098   5 years davidb Changes resulting from test runs to get Zookeeper and Solr running on …
(edit) @31097   5 years davidb Changed to .in style namne
(edit) @31096   5 years davidb Only need to create a volume's pages output directory is _output_dir …
(edit) @31095   5 years davidb Introduced num-partitions property
(edit) @31094   5 years davidb Changes triggered by running on gsliscluster1
(edit) @31093   5 years davidb Changes triggered by running on gsliscluster1
(edit) @31092   5 years davidb Minor tweak to spark/hadoop combination downloaded
(edit) @31091   5 years davidb Change of number of core for 'gsliscluster1' machine; commmented out …
(edit) @31090   5 years davidb Memory monitor debugging code, commented out
(edit) @31089   5 years davidb Change in way the JSON file is read in. Motivation was a …
(edit) @31088   5 years davidb Shift to newIstance for FileSystem due to StackOverflow page …
(edit) @31082   5 years davidb Changes in response to testing on gchead
(edit) @31081   5 years davidb Going live with generation of spark slaves file
(edit) @31080   5 years davidb echo formatting tidy up. Fixed some typos
(edit) @31079   5 years davidb Useful get started scripts
(edit) @31078   5 years davidb Some setup files and scripts to make running Spark and Solr easier on …
(edit) @31077   5 years davidb Move up to JDK1.8. Tidy up of Vagrant machine names. Support for YARN. …
(edit) @31065   5 years davidb Additional echo output
(edit) @31062   5 years davidb Added in -W option so check-sum calculation is skipped
(edit) @31058   5 years davidb echo for additional information added
(edit) @31057   5 years davidb Tweak to jps output formatting
(edit) @31053   5 years davidb Addition of second argument, optional, for where to save the files
(edit) @31051   5 years davidb Added in JDK to list of possible packages needed
(edit) @31046   5 years davidb Added property to control how severe a JSON IO problem is
(edit) @31045   5 years davidb More careful treatment of what to do when a JSON file isn't there
(edit) @31044   5 years davidb Fixed up error when output_dir is empty
(edit) @31043   5 years davidb Version for processing full EF set
(edit) @31042   5 years davidb Name changes, preparing the way for FULL-RUN versions
(edit) @31041   5 years davidb Test needs to be more careful if -read-only specified
(edit) @31036   5 years davidb Renaming to prepare way for YARN version of script
(edit) @31035   5 years davidb Changes after testing scripts
(edit) @31034   5 years davidb Development of scripts for working with Full EF dataset
(edit) @31033   5 years davidb Development of scripts for working with Full EF dataset
(edit) @31030   5 years davidb Tweak to some verbosity level 2 printing
(edit) @31029   5 years davidb Newline at end of file added
(edit) @31028   5 years davidb Support for randonly choosing Solr endpoints added in
(edit) @31027   5 years davidb Mixed typo in property name used
(edit) @31026   5 years davidb Corrected flag setting
(edit) @31025   5 years davidb Use property process-json-mode to determine which sort of Spark …
(edit) @31024   5 years davidb Support for Java properties file
(edit) @31022   5 years davidb No longer used
(edit) @31021   5 years davidb Folder restructure to remove 'trunk' part
(edit) @31020   5 years davidb No longer used
(edit) @31019   5 years davidb Part 2 or two-step folder restructure
(edit) @31018   5 years davidb Part 1 or two-step folder restructure
(edit) @31017   5 years davidb Moved to correct position
(edit) @31016   5 years davidb No longer used
(edit) @31015   5 years davidb Restructuring of projects into one
(edit) @31013   5 years davidb Accumulator for PerPageMap
(edit) @31011   5 years davidb Further RDD flatMap/map restructuring and refactoring, for per-page
(edit) @31010   5 years davidb Tidy up on generating Spark App name
(edit) @31009   5 years davidb Adjustments after latest fresh 'vagrant up' trial
(edit) @31008   5 years davidb Additional detail added into Spark app name
(edit) @31007   5 years davidb Class name refactoring
(edit) @31006   5 years davidb Further reversal of Base class. Switch to PerPage
(edit) @31005   5 years davidb Reversal of Base class in PerVolumeJSON
(edit) @31004   5 years davidb added debug
(edit) @31003   5 years davidb Explicity default constructors added
(edit) @31002   5 years davidb Need to separate flatMap and foreach calls in PagedJSON
(edit) @31001   5 years davidb Code to work per-volume and per-page
(edit) @31000   5 years davidb Class name refactoring
(edit) @30999   5 years davidb Class name refactoring
(edit) @30998   5 years davidb Class name refactoring
(edit) @30997   5 years davidb Verbosity control over printing
(edit) @30996   5 years davidb Code refactoring
(edit) @30995   5 years davidb Adjustment of NUM_PARTITIONS to be based on Spark recommended calculation
(edit) @30994   5 years davidb Additional useful links. Links open in new tab
(edit) @30993   5 years davidb Placeholder page to provide useful links to hadoop and solr cluster …
(edit) @30992   5 years davidb Additional adjustments after test run on cluster
(edit) @30991   5 years davidb Inital cut at README notes, and supporting links
(edit) @30990   5 years davidb opt name change
(edit) @30989   5 years davidb Changes to better suit EF set used with solr
(edit) @30988   5 years davidb Changed flag to 'read-only' and changed the filed name full text saved …
(edit) @30986   5 years davidb Debugging for double accumulator added
(edit) @30985   5 years davidb Changed to run main processing method as action rather than transform. …
(edit) @30984   5 years davidb Introduction of Spark accumulator to measure progress. Output of POST …
(edit) @30983   5 years davidb Useful helper script
(edit) @30982   5 years davidb Fixed to host_name for solr2 and solr3
(edit) @30981   5 years davidb Useful folder for 'on-the-side' packages
(edit) @30980   5 years davidb Code added to read response
(edit) @30979   5 years davidb _solr_url needs to be stored in class!
(edit) @30978   5 years davidb Additional debug statements
(edit) @30977   5 years davidb Only have RDD if an output directory was specified on the command-line …
(edit) @30976   5 years davidb Change to reflect changed order of command-line arguments
(edit) @30975   5 years davidb Introduction of new solr-url command line argument, leading to some …
(edit) @30974   5 years davidb update/add/doc JSON structure needed
(edit) @30973   5 years davidb Changed to saving Solr JSON file for debugging purposes
(edit) @30972   5 years davidb addition of useful command needed before re-running
(edit) @30971   5 years davidb Adding in post to Solr cloud. Changed text_t to _text_
(edit) @30970   5 years davidb Added in mapping of EF-JSON to Solr 'add' JSON format
(edit) @30969   5 years davidb Fine tuning resulting from testing the cloud/cluster
(edit) @30962   5 years davidb Corrections and improvements made after initial testing between …
Note: See TracRevisionLog for help on using the revision log.