Ignore:
Timestamp:
2016-11-10T16:20:02+13:00 (7 years ago)
Author:
davidb
Message:

Changes triggered by running on gsliscluster1

Location:
other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/scripts
Files:
2 edited

Legend:

Unmodified
Added
Removed

  • other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/scripts/FULL-RUN-MASTER-SPARK.sh

    r31043 r31093  
    44shift
    55
    6 input_dir="hdfs://master:9000/user/htrc/full-ef-json-files"
     6input_dir="hdfs://$SPARK_MASTER_HOST:9000/tmp/dbbridge/full-ef-json-files"
     7#input_dir="hdfs://master:9000/user/htrc/full-ef-json-files"
    78#input_dir="hdfs://10.10.0.52:9000/user/htrc/full-ef-json-files"
    89
    910#output_dir=hdfs://master:9000/user/htrc/full-solr-json-files
    10 solr_url="http://10.11.0.53:8983/solr/htrc-full-ef/update"
     11solr_url="http://gc0:8983/solr/htrc-full-ef/update"
    1112
    12 master_opt="--master spark://10.10.0.52:7077"
     13#
     14
     15master_opt="--executor-memory 10g --conf spark.executor.extraJavaOptions=-XX:+HeapDumpOnOutOfMemoryError --master spark://$SPARK_MASTER_HOST:7077"
    1316
    1417. scripts/_RUN.sh

  • other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/scripts/_RUN.sh

    r31057 r31093  
    9797if [ "$run_jps" = "1" ] ; then
    9898  echo "* Monitor progress on Spark cluster through:"
    99   echo "*   http://10.10.0.52:8080/"
     99  echo "*   http://$SPARK_MASTER_HOST:8080/"
    100100  echo "****"
    101101fi
Note: See TracChangeset for help on using the changeset viewer.