|
Last change
on this file since 31095 was 31095, checked in by davidb, 7 years ago |
|
Introduced num-partitions property
|
|
File size:
686 bytes
|
| Rev | Line | |
|---|
| [31024] | 1 |
|
|---|
| [31028] | 2 | #wcsa-ef-ingest.process-ef-json-mode = per-volume
|
|---|
| 3 | wcsa-ef-ingest.process-ef-json-mode = per-page
|
|---|
| 4 |
|
|---|
| [31094] | 5 | #wcsa-ef-ingest.solr-endpoint-ips = 10.11.0.53,10.11.0.54,10.11.0.55
|
|---|
| 6 | wcsa-ef-ingest.solr-endpoint-ips = gc0,gc1,gc2,gc3,gc4,gc5,gc6,gc7,gc8,gc9
|
|---|
| [31046] | 7 | wcsa-ef-ingest.strict-file-io = false
|
|---|
| [31029] | 8 |
|
|---|
| [31095] | 9 | # For guide on number of partitions to use, see "Parallelized collections" section of:
|
|---|
| 10 | # https://spark.apache.org/docs/2.0.1/programming-guide.html
|
|---|
| 11 | # which suggests 2-4 * num_cores
|
|---|
| 12 | #
|
|---|
| 13 | # For a more detailed discussion see:
|
|---|
| 14 | # http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/
|
|---|
| 15 |
|
|---|
| 16 | # wcsa-ef-ingest.num-partitions = 12
|
|---|
| 17 | wcsa-ef-ingest.num-partitions = 120
|
|---|
| 18 |
|
|---|
| 19 |
|
|---|
Note:
See
TracBrowser
for help on using the repository browser.
