Changeset 31361 for other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForCatalogLangCount.java
- Timestamp:
- 2017-01-27T10:26:16+13:00 (7 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForCatalogLangCount.java
r31359 r31361 7 7 import java.io.Serializable; 8 8 import org.apache.commons.cli.*; 9 9 import org.apache.hadoop.io.Text; 10 10 import org.apache.spark.api.java.*; 11 11 import org.apache.spark.api.java.function.Function2; … … 122 122 String packed_sequence_path = "hdfs:///user/capitanu/data/packed-ef"; 123 123 124 JavaPairRDD<String, String> inputRdd = jsc.sequenceFile(packed_sequence_path, String.class, String.class);125 JavaRDD< String> jsonTextRdd = inputRdd.map(Tuple2::_2);124 JavaPairRDD<String, Text> inputRdd = jsc.sequenceFile(packed_sequence_path, String.class, Text.class); 125 JavaRDD<Text> jsonTextRdd = inputRdd.map(Tuple2::_2); 126 126 127 127 /*
Note:
See TracChangeset
for help on using the changeset viewer.