Changeset 31272 for other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForLangCount.java
- Timestamp:
- 2017-01-04T09:37:56+13:00 (7 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForLangCount.java
r31271 r31272 11 11 import org.apache.spark.api.java.function.Function2; 12 12 import org.apache.spark.api.java.function.PairFunction; 13 import org.apache.spark.storage.StorageLevel; 13 14 import org.apache.spark.util.DoubleAccumulator; 14 15 import scala.Tuple2; … … 91 92 strict_file_io); 92 93 JavaRDD<String> lang_list = json_list_data_rp.flatMap(paged_solr_langfreq_flatmap); 94 lang_list.persist(StorageLevel.MEMORY_AND_DISK()); 93 95 lang_list.setName("lang-stream"); 94 96
Note:
See TracChangeset
for help on using the changeset viewer.