source: other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/ef-solr.properties@ 31095

Last change on this file since 31095 was 31095, checked in by davidb, 6 years ago

Introduced num-partitions property

File size: 686 bytes
Line 
1
2#wcsa-ef-ingest.process-ef-json-mode = per-volume
3wcsa-ef-ingest.process-ef-json-mode = per-page
4
5#wcsa-ef-ingest.solr-endpoint-ips = 10.11.0.53,10.11.0.54,10.11.0.55
6wcsa-ef-ingest.solr-endpoint-ips = gc0,gc1,gc2,gc3,gc4,gc5,gc6,gc7,gc8,gc9
7wcsa-ef-ingest.strict-file-io = false
8
9# For guide on number of partitions to use, see "Parallelized collections" section of:
10# https://spark.apache.org/docs/2.0.1/programming-guide.html
11# which suggests 2-4 * num_cores
12#
13# For a more detailed discussion see:
14# http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/
15
16# wcsa-ef-ingest.num-partitions = 12
17wcsa-ef-ingest.num-partitions = 120
18
19
Note: See TracBrowser for help on using the repository browser.