Changeset 33674 for other-projects/maori-lang-detection/src/org/greenstone/atea/morphia/SentenceInfo.java

Timestamp:

2019-11-15T00:21:31+13:00 (4 years ago)

Author:

ak19

Message:

Changes to support the top 5 predicted langcodes and their confidence values per sentence/overlapping sentence (all 103 made some documents, like of site 00006, too big too go into mongodb). Have re-run the NutchTextDumpToMongDB to send the new form of the docs into mongodb.

File:

• other-projects/maori-lang-detection/src/org/greenstone/atea/morphia/SentenceInfo.java (modified) (2 diffs)

other-projects/maori-lang-detection/src/org/greenstone/atea/morphia/SentenceInfo.java

@@ -1 +1 @@
 package org.greenstone.atea.morphia;
+
+import java.util.Map;
+import java.util.HashMap;
 
 import dev.morphia.annotations.*;
 
+
+@Entity("Sentences")
+public class SentenceInfo {
+
+    public final String sentence;
+    public final Map<String, Double> languageToConfidenceMap;
+    @Embedded
+    public final LanguageInfo[] languagesInfo; // array of langCode and confidence value pairs
+
+    
+    public SentenceInfo(String sentence, LanguageInfo[] languages) {
+    this.sentence = sentence;
+    this.languagesInfo = languages;
+
+    // let's store (langCode -> confidence) lookup in Map:
+    this.languageToConfidenceMap = new HashMap<String, Double>();   
+    for(LanguageInfo li : languages) {
+        String langCode = li.langCode;
+        Double confidence = new Double(li.confidenceLevel);
+        languageToConfidenceMap.put(langCode, confidence);
+    }
+    }
+    
+}
+
+// BACK WHEN WE ONLY STORED THE BEST PREDICTED LANGUAGE META FOR EACH SENTENCE:
+/*
 @Entity("Sentences")
 public class SentenceInfo {
     public final double confidenceLevel;
-    /** 3 letter lang code */
+    // 3 letter lang code
     public final String langCode;
     public final String sentence;
@@ -16 +46 @@
     }
 }
+*/