# # ChangeLog for other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster # # Generated by Trac 1.4.2 # 2024-04-30T23:50:50+12:00 Tue, 01 May 2018 22:51:29 GMT davidb [32175] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local (deleted) No longer needed Thu, 19 Apr 2018 23:21:49 GMT davidb [32174] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-shard.sh (added) Useful utility Thu, 19 Apr 2018 23:19:24 GMT davidb [32173] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr7.bash (added) Shift to using Solr7 setup Thu, 19 Apr 2018 23:16:28 GMT davidb [32172] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/TOPUP-JETTY.sh (modified) Tweaked to use Solr7 config files with Jetty installation Thu, 19 Apr 2018 23:13:41 GMT davidb [32171] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/htrc_configs.tar.gz (modified) Increased number or max bool clauses Thu, 19 Apr 2018 23:10:25 GMT davidb [32170] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-SOLR7.sh (added) Shifting to newer version of Solr Tue, 06 Feb 2018 10:48:03 GMT davidb [32120] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-setup-local-disk-all.sh (modified) Not entirely sure about this tweak to which SOLR env var is being ... Tue, 06 Feb 2018 10:46:27 GMT davidb [32119] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-healthcheck.sh (added) Another useful script to run with Solr Tue, 06 Feb 2018 10:45:46 GMT davidb [32118] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/jetty-solr7 (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/jetty-solr7/jetty.xml (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/jetty-solr7/jetty.xml.orig (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/jetty-solr7/realm.properties (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/jetty-solr7/web.xml (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/jetty-solr7/web.xml.orig (added) Config files needed to top up Jetty with solr servlet Tue, 06 Feb 2018 08:13:38 GMT davidb [32117] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/TOPUP-JETTY.sh (added) Some changes that are needed to run Jetty with security on admin, and ... Tue, 06 Feb 2018 03:27:25 GMT davidb [32116] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-cli.sh (added) Useful script that makes it easier to recall how to talk to the ... Sat, 08 Jul 2017 07:32:54 GMT davidb [31785] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) Change to allow solr command to optioanlly issue 'restart' instead of ... Fri, 07 Jul 2017 11:13:46 GMT davidb [31782] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/htrc_configs.tar.gz (modified) more careful separation into field types htrcstring and htrcstrings Mon, 06 Mar 2017 10:18:58 GMT davidb [31466] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-delete-collection.sh (modified) Fix to work out solr_host rather than assume it is gc0 Mon, 06 Mar 2017 10:18:24 GMT davidb [31465] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Adjustment to run solr with more memory Mon, 06 Mar 2017 10:10:21 GMT davidb [31464] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-delete-collection.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-collection.sh (added) More general version of script that let's you specify the collection ... Sun, 05 Mar 2017 06:44:56 GMT davidb [31455] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/_remote-solr-delete-full-ef-collection.sh (moved) deprecated Sun, 05 Mar 2017 06:41:35 GMT davidb [31454] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/_remote-solr-init-full-ef-collection.sh (moved) Deprecated Mon, 30 Jan 2017 11:08:35 GMT davidb [31373] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-setup-local-disk-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) Changes made to operate on solr1 and solr2 boxes Mon, 30 Jan 2017 10:30:16 GMT davidb [31370] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/htrc_configs.tar.gz (modified) Fixed incorrect version number. Using htrcstring so field values not ... Sun, 29 Jan 2017 10:39:41 GMT davidb [31367] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-start.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-stop.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Changes to work with solr1 and solr2 Sun, 29 Jan 2017 09:19:47 GMT davidb [31366] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-SOLR.sh (modified) Updated to latest released version of Solr Fri, 20 Jan 2017 05:01:03 GMT davidb [31307] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-stop-all.sh (added) convenience scripts Fri, 20 Jan 2017 04:50:33 GMT davidb [31306] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-add-shards-to-routers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-init-collection.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-stop-routerservers.sh (modified) Final part of the mongodb shard puzzle -- router servers Fri, 20 Jan 2017 01:50:12 GMT davidb [31305] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-init-configservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-init-shardservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-setup-local-disk-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-configservers.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-routerservers.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-shardservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-stop-shardservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Next good commit point. Initial testing of shard replset scripts Thu, 19 Jan 2017 05:10:18 GMT davidb [31304] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-setup-local-disk-all.sh (modified) Changes made whe (it turned out) the real source of the error was an ... Thu, 19 Jan 2017 05:07:02 GMT davidb [31303] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-configservers.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-routerservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-stop-configservers.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-stop-routerservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Adding in support to start and stop router server Thu, 19 Jan 2017 04:25:05 GMT davidb [31302] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-setup-local-disk-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-start-configservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-mongodb-stop-configservers.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-mongodb.bash (modified) Initial commit of scripts, after some testing, and subsequent changes ... Thu, 19 Jan 2017 04:22:54 GMT davidb [31301] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) Fix for gsliscluster1 Thu, 19 Jan 2017 02:59:38 GMT davidb [31300] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-mongodb.bash (modified) Need to use NETWORK not PACKAGE Thu, 19 Jan 2017 02:49:30 GMT davidb [31299] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Additionally setup MongoDB Thu, 19 Jan 2017 02:48:04 GMT davidb [31298] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-mongodb.bash (added) Initial cut at setup file for MongoDB Thu, 19 Jan 2017 02:44:53 GMT davidb [31297] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-MONGODB.sh (added) Thu, 05 Jan 2017 10:44:10 GMT davidb [31275] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Changes to allow gc slave nodes to work with local disk versions of ... Tue, 13 Dec 2016 22:29:31 GMT davidb [31235] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) More fine-grained testing to help nema setup Tue, 13 Dec 2016 22:20:57 GMT davidb [31234] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) More selective control of what to source/setup depending on hostname Tue, 13 Dec 2016 22:12:29 GMT davidb [31233] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Changes to operate on nema as well as gsliscluster1 and gc0-9 Tue, 13 Dec 2016 22:11:00 GMT davidb [31232] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/nema-state.json (added) Hand edited version of state.json from gsliscluster1 suitable for ... Tue, 13 Dec 2016 09:41:54 GMT davidb [31231] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-check-local-shardsize-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-rsync2nema-local-shard-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-setup-local-disk-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) Changes to allow SOLR to run on nodes in /hdfsd05/dbbridge/solr-ef Sat, 03 Dec 2016 01:16:01 GMT davidb [31174] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FULL-EF-HDFS-extra10-njp-missing.sh (added) One of the last scripts developed for getting ef dataset into HDFS Sat, 03 Dec 2016 01:14:20 GMT davidb [31173] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-aeu.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-bc.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-caia.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-chi.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-coo.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-coo1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-dul1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-emu.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-gri.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-hvd.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-iau.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ien.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-inu.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-keio.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ku01.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-loc.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-mcg.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-mdp.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-miua.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-miun.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-mmet.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-mou.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-nc01.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ncs1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-njp.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-nnc1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-nnc2.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-nyp.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-osu.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-psia.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-pst.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-pur1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-txa.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uc1-filename.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uc1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uc2.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ucm.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ucw.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-udel.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ufl1.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-ufl2.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uiuc.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uiug.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uiuo.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uma.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-umn.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-usu.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-uva.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-wau.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-wu.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-yale.txt (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/file-size-local/ef-full-yul.txt (added) individual file sizes per top-level folder Fri, 02 Dec 2016 20:40:17 GMT davidb [31172] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FULL-FILE-SIZE-COUNT.sh (added) to help track down missing files in HDFS copy Thu, 01 Dec 2016 21:15:59 GMT davidb [31171] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FILE-SIZE-CHECK-SUBFOLDERS.pl (added) Util to help find where missing files are Thu, 01 Dec 2016 21:15:25 GMT davidb [31170] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-TL-TARGET-DEPTH2-FOREACH-DEPTH3-HDFS-PUT.sh (added) Targetted sub-dir copy Thu, 01 Dec 2016 21:14:47 GMT davidb [31169] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FILE-SIZE-CHECK.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FULL-EF-HDFS.sh (modified) Improved logic Fri, 25 Nov 2016 10:00:10 GMT davidb [31161] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FILE-SIZE-CHECK-HDFS.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FILE-SIZE-CHECK.sh (added) Comparison of local disk version with HDFS version Wed, 23 Nov 2016 10:11:16 GMT davidb [31152] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-FOREACH-HDFS-PUT.sh (modified) Development of script Wed, 23 Nov 2016 10:10:20 GMT davidb [31151] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-TL-TARGET-DEPTH1-FOREACH-DEPTH2-HDFS-PUT.sh (added) More nuanced version to help finish off the 'big put' Sun, 20 Nov 2016 09:14:09 GMT davidb [31128] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/CHECK-EF-HDFS.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/FULL-EF-HDFS.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/PAIRTREE-FOREACH-HDFS-PUT.sh (added) Some scripts to help with pushing and monitoring the progress of the ... Tue, 15 Nov 2016 20:24:05 GMT davidb [31112] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-rsync2nema-local-shard-all.sh (added) To move out shards saved in /tmp on gsliscluter1 nodes to nema Fri, 11 Nov 2016 04:27:54 GMT davidb [31106] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-init-full-ef-collection.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/nema-solr-stop-all.sh (added) Scripts to help run an rsync'd copy of gslistcluster1 /tmp/gcX-solr- ... Fri, 11 Nov 2016 03:01:24 GMT davidb [31105] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-check-local-disk-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-check-local-shardsize-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-delete-full-ef-collection.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-setup-local-disk-all.sh (added) Additional scripts to help with running solr locally out of /tmp area Fri, 11 Nov 2016 03:00:37 GMT davidb [31104] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) now configurable to be run from local disk (/tmp) Fri, 11 Nov 2016 00:22:02 GMT davidb [31103] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-full-ef-collection.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-stop-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-spark.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-zookeeper.bash (modified) Changes made after testing with 20 solr nodes Thu, 10 Nov 2016 10:08:48 GMT davidb [31101] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-full-ef-collection.sh (modified) Correction to collection name Thu, 10 Nov 2016 09:43:57 GMT davidb [31099] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-init-full-ef-collection.sh (added) Changes resulting from test runs to get Zookeeper and Solr running on ... Thu, 10 Nov 2016 09:42:31 GMT davidb [31098] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-stop-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) Changes resulting from test runs to get Zookeeper and Solr running on ... Thu, 10 Nov 2016 09:41:33 GMT davidb [31097] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/zoo.cfg.in (moved) Changed to .in style namne Mon, 07 Nov 2016 21:38:35 GMT davidb [31082] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-start-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-stop-all.sh (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Changes in response to testing on gchead Mon, 07 Nov 2016 10:51:29 GMT davidb [31081] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) Going live with generation of spark slaves file Mon, 07 Nov 2016 10:49:42 GMT davidb [31080] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-spark.bash (modified) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-zookeeper.bash (modified) echo formatting tidy up. Fixed some typos Mon, 07 Nov 2016 10:31:48 GMT davidb [31079] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-SOLR.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/GET-PACKAGES-SPARK.sh (added) Useful get started scripts Mon, 07 Nov 2016 10:27:52 GMT davidb [31078] * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/htrc_configs.tar.gz (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/CONF/zoo.cfg (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-solr-stop-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-start-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-spark-stop-all.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-start.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SCRIPTS/remote-zookeeper-stop.sh (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP.bash (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-solr.bash (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-spark.bash (added) * other-projects/hathitrust/wcsa/extracted-features-solr/trunk/gslis-cluster/SETUP/setup-zookeeper.bash (added) Some setup files and scripts to make running Spark and Solr easier on ...