# # ChangeLog for other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster # # Generated by Trac 1.4.2 # 2024-04-23T19:58:54+12:00 Fri, 25 Nov 2016 10:00:10 GMT davidb [31161] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FILE-SIZE-CHECK-HDFS.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FILE-SIZE-CHECK.sh (added) Comparison of local disk version with HDFS version Wed, 23 Nov 2016 10:11:16 GMT davidb [31152] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-FOREACH-HDFS-PUT.sh (modified) Development of script Wed, 23 Nov 2016 10:10:20 GMT davidb [31151] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-TL-TARGET-DEPTH1-FOREACH-DEPTH2-HDFS-PUT.sh (added) More nuanced version to help finish off the 'big put' Sun, 20 Nov 2016 09:14:09 GMT davidb [31128] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/CHECK-EF-HDFS.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FULL-EF-HDFS.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-FOREACH-HDFS-PUT.sh (added) Some scripts to help with pushing and monitoring the progress of the ... Tue, 15 Nov 2016 20:24:05 GMT davidb [31112] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-rsync2nema-local-shard-all.sh (added) To move out shards saved in /tmp on gsliscluter1 nodes to nema Fri, 11 Nov 2016 04:27:54 GMT davidb [31106] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-init-full-ef-collection.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-stop-all.sh (added) Scripts to help run an rsync'd copy of gslistcluster1 /tmp/gcX-solr- ... Fri, 11 Nov 2016 03:01:24 GMT davidb [31105] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-check-local-disk-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-check-local-shardsize-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-delete-full-ef-collection.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-setup-local-disk-all.sh (added) Additional scripts to help with running solr locally out of /tmp area Fri, 11 Nov 2016 03:00:37 GMT davidb [31104] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) now configurable to be run from local disk (/tmp) Fri, 11 Nov 2016 00:22:02 GMT davidb [31103] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-full-ef-collection.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-stop-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-spark.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-zookeeper.bash (modified) Changes made after testing with 20 solr nodes Thu, 10 Nov 2016 10:08:48 GMT davidb [31101] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-full-ef-collection.sh (modified) Correction to collection name Thu, 10 Nov 2016 09:43:57 GMT davidb [31099] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-full-ef-collection.sh (added) Changes resulting from test runs to get Zookeeper and Solr running on ... Thu, 10 Nov 2016 09:42:31 GMT davidb [31098] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-stop-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) Changes resulting from test runs to get Zookeeper and Solr running on ... Thu, 10 Nov 2016 09:41:33 GMT davidb [31097] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/zoo.cfg.in (moved) Changed to .in style namne Mon, 07 Nov 2016 21:38:35 GMT davidb [31082] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-stop-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Changes in response to testing on gchead Mon, 07 Nov 2016 10:51:29 GMT davidb [31081] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Going live with generation of spark slaves file Mon, 07 Nov 2016 10:49:42 GMT davidb [31080] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-spark.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-zookeeper.bash (modified) echo formatting tidy up. Fixed some typos Mon, 07 Nov 2016 10:31:48 GMT davidb [31079] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-SOLR.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-SPARK.sh (added) Useful get started scripts Mon, 07 Nov 2016 10:27:52 GMT davidb [31078] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/htrc_configs.tar.gz (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/zoo.cfg (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-stop-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-stop-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-start.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-stop.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-spark.bash (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-zookeeper.bash (added) Some setup files and scripts to make running Spark and Solr easier on ...