Changeset 31502 for other-projects/hathitrust/wcsa/extracted-features-solr
- Timestamp:
- 2017-03-13T14:16:15+13:00 (7 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
other-projects/hathitrust/wcsa/extracted-features-solr/trunk/solr-ingest/src/main/java/org/hathitrust/extractedfeatures/ProcessForSolrIngest.java
r31451 r31502 107 107 108 108 JavaPairRDD<Text, Text> input_pair_rdd = jsc.sequenceFile(packed_sequence_path, Text.class, Text.class); 109 109 //JavaPairRDD<String, String> input_pair_rdd = jsc.wholeTextFiles(packed_sequence_path); 110 111 //JavaPairRDD<Text, Text> input_pair_sampled_rdd = input_pair_rdd.sample(false,0.5,42); 112 113 //JavaRDD<Text> json_text_rdd = input_pair_sampled_rdd.map(item -> item._2); 114 //JavaRDD<Text> json_text_rdd = input_pair_rdd.map(item -> new Text(item._2)); 110 115 JavaRDD<Text> json_text_rdd = input_pair_rdd.map(item -> item._2); 111 116
Note:
See TracChangeset
for help on using the changeset viewer.