Changeset 30944

Timestamp:

2016-10-26T14:47:29+13:00 (6 years ago)

Author:

davidb

Message:

Forcer higher partition (6) than default, which seems to be 2

File:

• other-projects/hathitrust/solr-extracted-features/trunk/src/main/java/org/hathitrust/PrepareForIngest.java (modified) (1 diff)

other-projects/hathitrust/solr-extracted-features/trunk/src/main/java/org/hathitrust/PrepareForIngest.java

@@ -50 +50 @@
         }
                 
-        JavaRDD<String> json_list_data = jsc.textFile(_json_list_filename).cache();
+        JavaRDD<String> json_list_data = jsc.textFile(_json_list_filename,6).cache();
 
         JavaRDD<String> json_ids = json_list_data.flatMap(new PagedJSON(_input_dir,_verbosity));