Changeset 31363 for other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForCatalogLangCount.java

Timestamp:

2017-01-27T21:24:16+13:00 (7 years ago)

Author:

davidb

Message:

Control num of partitions on sort

File:

• other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForCatalogLangCount.java (modified) (1 diff)

other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForCatalogLangCount.java

@@ -154 +154 @@
         
         JavaPairRDD<Long, String> catalog_lang_counts_swapped_pair_sorted 
-            = catalog_lang_counts_swapped_pair.sortByKey(false);
+            = catalog_lang_counts_swapped_pair.sortByKey(false,20);
         catalog_lang_counts_swapped_pair_sorted.setName("descending-sorted-frequency-catalog-lang");