Changeset 33916
- Timestamp:
- 2020-02-13T17:42:11+13:00 (4 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
other-projects/maori-lang-detection/mongodb-data/ManualShortlisting2_afterMongoDBReingest.txt
r33915 r33916 42 42 = 221 websites 43 43 44 45 Both values are the same. This means that after reingesting into MongoDB, there are no longer any Australian sites with /mi in the URL path. Previously, manual inspection found kiwiproperty.com with geoLocation of Australia which was a genuine site of interest from manual inspection. But since its geolocation upon reingest has changed to US, we no longer have to treat that site and therefore Australian sites with mi in their URL paths specially. 44 46 45 47 … … 52 54 {domain: {$not: /\.nz/}}, 53 55 {numPagesContainingMRI: {$gt: 0}}, 54 {$or: [{geoLocationCountryCode: "AU"}, {urlContainsLangCodeInPath: false}]} 56 {urlContainsLangCodeInPath: false} 55 57 ] 56 58 } … … 67 69 ]); 68 70 69 70 71 /* 1 */ 71 72 { 72 73 "_id" : "us", 73 "count" : 117.0, 74 "domain" : [ 75 "http://daandehn.com", 76 "https://articles.imperialtometric.com", 77 "http://anglican.org", 78 "http://mikebonnice.com", 79 "https://nl.pinterest.com", 74 "count" : 120.0, 75 "domain" : [ 76 "http://lianzaconference2012.blogspot.com", 77 "https://www.pinterest.ca", 78 "http://takethatvacation.com", 79 "https://www.indexmundi.com", 80 "http://ngarangatahi.tripod.com", 81 "http://frontrowphotos.com", 82 "https://www.nccri.ie", 83 "http://niken8media.logdown.com", 84 "https://www.seapixonline.com", 85 "https://www.code-postal.com", 86 "http://www.muhammad.com", 87 "https://static-promote.weebly.com", 88 "http://www.unicode.org", 89 "http://anglicanhistory.org", 90 "http://rangiwewehi.com", 91 "https://wol.jw.org", 92 "http://www.pressreader.com", 93 "http://linkvip.top", 94 "https://www.podrozeady.com", 95 "http://www.thesalmons.org", 96 "http://shangrilapress.net", 97 "http://georgegi.tripod.com", 98 "https://www.terakau.org", 80 99 "http://svenskadress.net", 100 "http://malecek.com", 81 101 "http://word-dialect.blogspot.com", 82 "http://fhr.kiwicelts.com", 102 "https://www.blue-frontiers.com", 103 "http://atopeconlostopes.blogspot.com", 104 "http://dannykahei.tripod.com", 105 "https://www.oemsec.com", 106 "http://wikiedit.org", 107 "https://www.dbnames.net", 108 "http://www.godrules.net", 83 109 "http://www.huapala.org", 84 "http://www.whoisthatr.com", 85 "http://www.precious-testimonies.com", 86 "https://www.oemsec.com", 87 "http://www.godrules.net", 88 "https://www.pinterest.it", 89 "http://www.wikitree.com", 90 "http://ritusehji.blogspot.com", 91 "http://www.frogsonline.com", 92 "https://biblehub.com", 93 "https://www.pinterest.co.uk", 110 "https://www.pinterest.jp", 111 "https://kjohnsonnz.blogspot.com", 112 "http://www.gotquestions.org", 113 "http://tuhua2010.blogspot.com", 114 "http://www.twttoa.com", 94 115 "http://pumanawawhangara.blogspot.com", 95 116 "http://hannas-reiseblog.blogspot.com", 96 "http://frontrowphotos.com", 97 "https://www.pinterest.ca", 98 "http://www.muhammad.com", 99 "https://www.pinterest.jp", 100 "http://www.gotquestions.org", 101 "https://www.dbnames.net", 117 "https://nl.pinterest.com", 118 "https://www.myadsclassified.com", 119 "http://mikebonnice.com", 120 "https://www.webwiki.com", 121 "http://fhr.kiwicelts.com", 122 "https://articles.imperialtometric.com", 123 "http://kiaorahola.blogspot.com", 124 "http://ww25.milfsplease.com", 125 "http://daandehn.com", 126 "http://www.precious-testimonies.com", 127 "https://www.pinterest.it", 128 "https://www.pinterest.co.uk", 129 "http://naturalfatburner.net", 130 "https://www.vaihaunui.net", 131 "http://capsuraotearoa.blogspot.com", 132 "http://m.biblepub.com", 133 "http://shuttersportnelson.photoshelter.com", 134 "http://precious-testimonies.com", 135 "http://wowwars.net", 136 "https://www.breaker.audio", 137 "http://tkrow.tripod.com", 138 "http://ritusehji.blogspot.com", 139 "http://seapixonline.com", 140 "http://www.whoisthatr.com", 141 "https://livestream.com", 142 "https://biblehub.com", 143 "https://www.pipirikiapapatuanuku.org", 144 "http://www.wikitree.com", 145 "http://bahaiprayers.net", 146 "https://phet.colorado.edu", 147 "http://tatai09.blogspot.com", 102 148 "http://www.hudl.com", 103 149 "https://ebible.org", 104 "http://tuhua2010.blogspot.com", 105 "http://ww25.milfsplease.com", 106 "http://www.thesalmons.org", 107 "https://wol.jw.org", 108 "http://georgegi.tripod.com", 109 "http://linkvip.top", 150 "http://rhymebrain.com", 151 "http://tkkpipipaopao.blogspot.com", 152 "http://www.waimate.com", 153 "http://piripi.blogspot.com", 154 "http://burkekm001.tripod.com", 155 "https://www.hidroponia.org.mx", 156 "http://www.v3whois.com", 157 "http://www.the-naked.com", 158 "https://www.pinterest.fr", 159 "http://maaori.com", 160 "http://loquevendra318.com", 161 "http://www.geni.com", 162 "https://maorinews.com", 163 "http://www.frogsonline.com", 164 "https://drive.google.com", 165 "https://in.pinterest.com", 166 "http://www.mkiwi.com", 167 "https://www.kaifineart.com", 168 "http://www.roadsmile.com", 169 "https://png.bible", 170 "http://blogdepasopor.blogspot.com", 171 "http://www.steve-wheeler.co.uk", 172 "http://www.whoisentry.com", 173 "http://anglican.org", 174 "http://www.eyecontactsite.com", 175 "http://aclhokiangarocks.blogspot.com", 176 "http://manateina.blogspot.com", 177 "https://www.knowatom.com", 178 "https://chromium.googlesource.com", 179 "https://za.pinterest.com", 180 "http://mahoraroom8.blogspot.com", 181 "https://www.bible.com", 182 "http://worldradiomap.com", 183 "http://www.hiroa.pf", 184 "http://www.lunar-occultations.com", 110 185 "https://docs.google.com", 111 "http://rangiwewehi.com", 112 "http://anglicanhistory.org", 113 "http://niken8media.logdown.com", 186 "http://www.krassotkin.ru", 187 "http://www.namesdir.com", 188 "https://www.poehalisnami.ua", 189 "http://www.forensicfashion.com", 190 "http://eartheum.com", 191 "http://www.code-postal.com", 114 192 "http://mrshamiltonskoolkidz.blogspot.com", 115 "https://www.vaihaunui.net",116 "http://dannykahei.tripod.com",117 "http://www.lunar-occultations.com",118 "http://seapixonline.com",119 "http://tkrow.tripod.com",120 "https://drive.google.com",121 "http://takethatvacation.com",122 "https://in.pinterest.com",123 "https://www.nccri.ie",124 "https://www.webwiki.com",125 "http://www.unicode.org",126 "http://shangrilapress.net",127 "http://ngarangatahi.tripod.com",128 "https://static-promote.weebly.com",129 "https://www.podrozeady.com",130 "https://www.blue-frontiers.com",131 "https://www.indexmundi.com",132 "http://www.namesdir.com",133 "https://www.bible.com",134 "http://www.krassotkin.ru",135 "http://malecek.com",136 "http://korora.econ.yale.edu",137 "https://www.poehalisnami.ua",138 "http://loquevendra318.com",139 "https://www.terakau.org",140 "https://za.pinterest.com",141 "http://www.mkiwi.com",142 "http://maaori.com",143 "http://atopeconlostopes.blogspot.com",144 "http://worldradiomap.com",145 "http://eartheum.com",146 "http://www.forensicfashion.com",147 "http://www.code-postal.com",148 "http://www.pressreader.com",149 "https://www.seapixonline.com",150 "http://lianzaconference2012.blogspot.com",151 "http://blogdepasopor.blogspot.com",152 "https://www.code-postal.com",153 "http://www.steve-wheeler.co.uk",154 "https://www.knowatom.com",155 "http://bahaiprayers.net",156 "http://www.eyecontactsite.com",157 "http://www.hiroa.pf",158 "http://mahoraroom8.blogspot.com",159 "http://www.roadsmile.com",160 "https://chromium.googlesource.com",161 "http://aclhokiangarocks.blogspot.com",162 "http://wowwars.net",163 "https://www.hidroponia.org.mx",164 "http://tkkpipipaopao.blogspot.com",165 "http://tatai09.blogspot.com",166 "http://kiaorahola.blogspot.com",167 "http://manateina.blogspot.com",168 "http://www.the-naked.com",169 "http://shuttersportnelson.photoshelter.com",170 "http://precious-testimonies.com",171 "https://www.breaker.audio",172 193 "https://www.natekore2018.com", 173 "http://naturalfatburner.net", 174 "https://www.pinterest.fr", 175 "https://www.pipirikiapapatuanuku.org", 176 "http://capsuraotearoa.blogspot.com", 177 "http://m.biblepub.com", 178 "https://phet.colorado.edu", 179 "https://livestream.com", 180 "http://www.geni.com", 181 "https://kjohnsonnz.blogspot.com", 182 "https://maorinews.com", 183 "http://www.twttoa.com", 184 "http://www.whoisentry.com", 185 "http://burkekm001.tripod.com", 186 "http://wikiedit.org", 187 "http://piripi.blogspot.com", 188 "https://www.kaifineart.com", 189 "https://png.bible", 190 "http://rhymebrain.com", 191 "http://www.v3whois.com", 192 "http://www.waimate.com", 193 "https://www.myadsclassified.com" 194 "http://korora.econ.yale.edu" 194 195 ] 195 196 } … … 200 201 "count" : 19.0, 201 202 "domain" : [ 203 "http://vulkane.ch", 204 "http://www.stephe.de", 205 "https://ersatzteile-fachversand.de", 206 "http://etoile-de-lune.net", 207 "https://www.cartogiraffe.com", 208 "https://laskar02cinta.page.tl", 209 "http://www.cartogiraffe.com", 202 210 "http://www.udhr.de", 211 "http://klaaskoehne.de", 203 212 "http://m.distanta.1km.net", 213 "http://insecta.pro", 214 "http://weltderberge.de", 204 215 "http://arts.mythologica.fr", 205 "http://vulkane.ch",206 216 "http://www.behlig.de", 217 "http://svenkirsten.com", 218 "http://etymologie.info", 207 219 "http://www.nierstrasz.org", 208 220 "https://www.tvteile.de", 209 "http://etymologie.info", 210 "https://www.cartogiraffe.com", 211 "https://www.you-fly.com", 212 "http://klaaskoehne.de", 213 "http://weltderberge.de", 214 "http://www.cartogiraffe.com", 215 "http://svenkirsten.com", 216 "https://laskar02cinta.page.tl", 217 "http://etoile-de-lune.net", 218 "https://ersatzteile-fachversand.de", 219 "http://insecta.pro", 220 "http://www.stephe.de" 221 "https://www.you-fly.com" 221 222 ] 222 223 } … … 227 228 "count" : 16.0, 228 229 "domain" : [ 230 "http://www.gif.ovh", 231 "http://pt.city-usa.net", 232 "http://www.maraamusurfskirace.com", 233 "http://rapanui.fr", 234 "http://kihikihi.fr", 235 "http://blueheavenisland.com", 236 "http://www.gototahiti.net", 237 "http://www.gaudry.be", 238 "http://www.rongo-rongo.com", 239 "http://chantsdeluttes.free.fr", 240 "http://www.blueheavenisland.com", 229 241 "http://baladeornithologique.com", 230 "http://chantsdeluttes.free.fr", 231 "http://kihikihi.fr", 232 "http://www.blueheavenisland.com", 233 "http://splaf.free.fr", 242 "http://mahajana.net", 234 243 "https://www.lexilogos.com", 235 244 "https://www.manualscat.com", 236 "http://rapanui.fr", 237 "http://www.gototahiti.net", 238 "http://pt.city-usa.net", 239 "http://blueheavenisland.com", 240 "http://www.gif.ovh", 241 "http://www.gaudry.be", 242 "http://www.maraamusurfskirace.com", 243 "http://mahajana.net", 244 "http://www.rongo-rongo.com" 245 "http://splaf.free.fr" 245 246 ] 246 247 } … … 251 252 "count" : 16.0, 252 253 "domain" : [ 254 "http://gouvernante.info", 255 "http://www.gouvernante.info", 256 "https://arrowheadproject.azurewebsites.net", 257 "http://hidsonphoto.com", 258 "http://skimap.info", 253 259 "http://tetsubo.org", 254 260 "https://arrowhead.eu", 261 "https://www.henrifloor.nl", 262 "http://diverosa.com", 255 263 "https://www.arrowhead.eu", 256 "https://www.henrifloor.nl", 257 "http://hidsonphoto.com", 258 "https://arrowheadproject.azurewebsites.net", 264 "http://wearehomework.com", 259 265 "http://nielsonboutique.co.uk", 260 "http://www.gouvernante.info",261 266 "http://tonhut.nl", 262 "http://diverosa.com",263 267 "http://longhornlaw.net", 264 268 "http://www.nonlinear.demon.nl", 265 "http://wearehomework.com",266 "http://gouvernante.info",267 "http://skimap.info",268 269 "http://www.encyclo.co.uk" 269 270 ] … … 275 276 "count" : 8.0, 276 277 "domain" : [ 278 "http://jazz.ngapuhitelevision.com", 279 "http://komisch.ngapuhitelevision.com", 277 280 "http://powhiri.ngapuhitelevision.com", 278 "http://akona.ngapuhitelevision.com", 281 "http://waiatarangatiratanga.ngapuhitelevision.com", 282 "http://ngapuhiradio.com", 279 283 "http://ngapuhitelevision.com", 280 "http://ngapuhiradio.com",281 "http://komisch.ngapuhitelevision.com",282 284 "http://www.rennertweb.de", 283 "http://jazz.ngapuhitelevision.com", 284 "http://waiatarangatiratanga.ngapuhitelevision.com" 285 "http://akona.ngapuhitelevision.com" 285 286 ] 286 287 } … … 288 289 /* 6 */ 289 290 { 291 "_id" : "cz", 292 "count" : 5.0, 293 "domain" : [ 294 "https://www.fipojobs.com", 295 "http://about.ilikeyou.com", 296 "https://www.viveipcl.com", 297 "http://www.henryklahola.nazory.cz", 298 "http://henryklahola.nazory.cz" 299 ] 300 } 301 302 /* 7 */ 303 { 290 304 "_id" : "ca", 291 "count" : 7.0, 292 "domain" : [ 293 "http://00.gs", 305 "count" : 5.0, 306 "domain" : [ 307 "http://www.myrasplace.net", 308 "http://bcmarina.com", 309 "http://bckayak.com", 294 310 "http://aguadilla.airport-authority.com", 295 "http://bckayak.com", 296 "http://bcmarina.com", 297 "http://www.myrasplace.net" 298 ] 299 } 300 301 /* 7 */ 302 { 303 "_id" : "au", 304 "count" : 5.0, 305 "domain" : [ 306 "https://infogram.com", 307 "https://www.kiwiproperty.com", 308 "http://theunderwaterworld.com", 309 "http://fionajack.net", 310 "https://koreromaori.com" 311 "http://00.gs" 311 312 ] 312 313 } 313 314 314 315 /* 8 */ 315 {316 "_id" : "cz",317 "count" : 4.0,318 "domain" : [319 "http://www.henryklahola.nazory.cz",320 "http://about.ilikeyou.com",321 "http://henryklahola.nazory.cz",322 "https://www.fipojobs.com"323 ]324 }325 326 /* 9 */327 316 { 328 317 "_id" : "gb", 329 318 "count" : 4.0, 330 319 "domain" : [ 331 "http://www.wordsearchfun.com",332 320 "http://www.woolrych.org", 333 321 "https://omniatlas.com", 322 "http://www.wordsearchfun.com", 334 323 "http://mikestephens.co.uk" 335 324 ] 336 325 } 337 326 338 /* 10*/327 /* 9 */ 339 328 { 340 329 "_id" : "es", 341 330 "count" : 4.0, 342 331 "domain" : [ 343 "http://www.info-hoteles.com",344 332 "https://www.uv.es", 333 "http://www.cruceros-princess.mx", 345 334 "https://www.reclamaciondevuelos.com", 346 "http://www.cruceros-princess.mx" 335 "http://www.info-hoteles.com" 336 ] 337 } 338 339 /* 10 */ 340 { 341 "_id" : "au", 342 "count" : 4.0, 343 "domain" : [ 344 "https://koreromaori.com", 345 "http://theunderwaterworld.com", 346 "https://infogram.com", 347 "http://fionajack.net" 347 348 ] 348 349 } 349 350 350 351 /* 11 */ 352 { 353 "_id" : "it", 354 "count" : 3.0, 355 "domain" : [ 356 "http://oipaz.net", 357 "http://www.marcosanti.it", 358 "http://www.pegasoesmicamion.com" 359 ] 360 } 361 362 /* 12 */ 351 363 { 352 364 "_id" : "at", … … 354 366 "domain" : [ 355 367 "http://www.petit-prince.at", 356 "http://www.tmtmm.net", 357 "http://petit-prince.at" 358 ] 359 } 360 361 /* 12 */ 362 { 363 "_id" : "it", 364 "count" : 3.0, 365 "domain" : [ 366 "http://oipaz.net", 367 "http://www.pegasoesmicamion.com", 368 "http://www.marcosanti.it" 368 "http://petit-prince.at", 369 "http://www.tmtmm.net" 369 370 ] 370 371 } 371 372 372 373 /* 13 */ 373 {374 "_id" : "il",375 "count" : 2.0,376 "domain" : [377 "http://www.daat.ac.il",378 "https://www.hitiaotera.com"379 ]380 }381 382 /* 14 */383 374 { 384 375 "_id" : "ch", 385 376 "count" : 2.0, 386 377 "domain" : [ 387 "https:// nicoledidi.ch",388 "https:// photos.axelebert.org"389 ] 390 } 391 392 /* 1 5*/378 "https://photos.axelebert.org", 379 "https://nicoledidi.ch" 380 ] 381 } 382 383 /* 14 */ 393 384 { 394 385 "_id" : "ro", 395 386 "count" : 2.0, 396 387 "domain" : [ 397 "http://www.parohiauceadesus.ro", 398 "http://parohiauceadesus.ro" 388 "http://parohiauceadesus.ro", 389 "http://www.parohiauceadesus.ro" 390 ] 391 } 392 393 /* 15 */ 394 { 395 "_id" : "unknown", 396 "count" : 1.0, 397 "domain" : [ 398 "https://www.hitiaotera.com" 399 399 ] 400 400 } 401 401 402 402 /* 16 */ 403 { 404 "_id" : "fi", 405 "count" : 1.0, 406 "domain" : [ 407 "http://pertti.com" 408 ] 409 } 410 411 /* 17 */ 412 { 413 "_id" : "jp", 414 "count" : 1.0, 415 "domain" : [ 416 "http://yutaka.it-n.jp" 417 ] 418 } 419 420 /* 18 */ 403 421 { 404 422 "_id" : "mx", … … 409 427 } 410 428 411 /* 1 7*/412 { 413 "_id" : " unknown",429 /* 19 */ 430 { 431 "_id" : "ru", 414 432 "count" : 1.0, 415 433 "domain" : [ 416 "https://www. viveipcl.com"417 ] 418 } 419 420 /* 18*/434 "https://www.gismeteo.lv" 435 ] 436 } 437 438 /* 20 */ 421 439 { 422 440 "_id" : "bg", … … 427 445 } 428 446 429 /* 19 */ 447 /* 21 */ 448 { 449 "_id" : "ie", 450 "count" : 1.0, 451 "domain" : [ 452 "https://coggle.it" 453 ] 454 } 455 456 /* 22 */ 430 457 { 431 458 "_id" : "cn", … … 436 463 } 437 464 438 /* 2 0*/465 /* 23 */ 439 466 { 440 467 "_id" : "ir", … … 445 472 } 446 473 447 /* 2 1*/448 { 449 "_id" : " fi",474 /* 24 */ 475 { 476 "_id" : "il", 450 477 "count" : 1.0, 451 478 "domain" : [ 452 "http://pertti.com" 453 ] 454 } 455 456 /* 22 */ 457 { 458 "_id" : "ie", 459 "count" : 1.0, 460 "domain" : [ 461 "https://coggle.it" 462 ] 463 } 464 465 /* 23 */ 466 { 467 "_id" : "ru", 468 "count" : 1.0, 469 "domain" : [ 470 "https://www.gismeteo.lv" 471 ] 472 } 473 474 /* 24 */ 475 { 476 "_id" : "jp", 477 "count" : 1.0, 478 "domain" : [ 479 "http://yutaka.it-n.jp" 479 "http://www.daat.ac.il" 480 480 ] 481 481 } … … 527 527 ~ http://00.gs/Maniapoto;Uriwera;Moriori;Hivaoa;Kumulipo.htm - Maori-Polynesian comparative dictionary words listing 528 528 - aguadilla.airport-authority.com - misidentification 529 - https://articles.imperialtometric.com - misidentification 530 - http://daandehn.com - no more than 1 sentence over multiple files. Appears to be photo captions of NZ placenames 529 [MOVED TO US: - https://articles.imperialtometric.com - misidentification] 530 [MOVED TO US: - http://daandehn.com - no more than 1 sentence over multiple files. Appears to be photo captions of NZ placenames] 531 531 DE: 532 532 - http://etymologie.info/~e/n_/nz-___reg.html - placenames, not meaningful … … 561 561 - ISRAEL: 562 562 http://www.daat.ac.il - misidentification of "no." as MRI, and Hebrew words. 563 https://www.hitiaotera.com/ - misidentifiation of Tahitian pages563 [MOVED TO UNKNOWN: https://www.hitiaotera.com/ - misidentifiation of Tahitian pages] 564 564 - RUSSIA: https://www.gismeteo.lv - misidentification of an email address 565 565 - JAPAN: http://yutaka.it-n.jp - many pages of scientific names of (plants?) which are often misdetected as MRI … … 570 570 !! http://www.henryklahola.nazory.cz/094.Maori.htm and http://henryklahola.nazory.cz variant 571 571 http://about.ilikeyou.com - dating site. Misidentification. 572 GAINED FROM UNKNOWN: https://www.viveipcl.com: tours website, placenames mentioned] 572 573 - SPAIN: 573 574 !! https://www.uv.es/~pla/red.net/intmaori.html … … 582 583 nicoledidi.ch - blog, placenames 583 584 https://photos.axelebert.org - Tahiti related content 584 - UNKNOWN: https://www.viveipcl.com: tours website, placenames mentioned 585 - UNKNOWN: 586 [MOVED TO CZ: https://www.viveipcl.com: tours website, placenames mentioned] 587 GAINED FROM IL: https://www.hitiaotera.com/ - misidentifiation of Tahitian pages 585 588 #- EU: https://www.the-good-stuff-factory.be/mi/ : Autotranslated 586 589 !! - BULGARIA: http://anitra.net/activism/humanrights/UDHR/rrt_print.htm (2 pages) … … 588 591 589 592 AUSTRALIA: 590 !! https://www.kiwiproperty.com - e.g. https://www.kiwiproperty.com/the-base/mi/he-paepaki/ has some actual MRI sentences. [Not autotranslated]593 [MOVED TO US: !! https://www.kiwiproperty.com - e.g. https://www.kiwiproperty.com/the-base/mi/he-paepaki/ has some actual MRI sentences. [Not autotranslated]] 591 594 ? http://fionajack.net - Wellington gallery of artist. A few occurrences of Kia Ora in a title like context (e.g. "Street Party Kia Ora! Kia Ora!") 592 595 X!! https://infogram.com/te-marautanga-o-aotearoa-moe-pld-allocations-2012-1go502ygvn562jd - site of individual pages (like docs.google.com). This one has a relevant infogram image. But it's English with MRI in the image legend and captions. … … 605 608 606 609 US: 607 Done: manually inspected 68/117 sites 608 609 TOTAL US: 4+7+7+4+3=25 610 Done: manually inspected 69/120 sites 611 612 TOTAL US: 1+4+7+7+4+3=26 613 614 615 US GAINED AFTER REINGEST: 616 + anglican.org 617 GAINED FROM CA: - https://articles.imperialtometric.com - misidentification 618 GAINED FROM CA: - http://daandehn.com - no more than 1 sentence over multiple files. Appears to be photo captions of NZ placenames 610 619 611 620 DEFINITELY: … … 775 784 776 785 TOTALS: 777 US: 2 5778 AU: 2786 US: 26 787 AU: 1 779 788 DE: 2 780 789 DK: 2 … … 784 793 FR: 1 785 794 IE: 1 786 TOTAL: 213795 TOTAL: (assuming 176 for NZ) + 36 = 212 787 796 788 797 ------------------------------------------------ … … 807 816 ]) 808 817 809 Also excluding AU, since we dealt with that already in step A1: 810 db.getCollection('Websites').find({$and: [{numPagesContainingMRI: {$gt: 0}},{urlContainsLangCodeInPath: true}, {domain: {$not: /.nz$/}}, {geoLocationCountryCode: {$not: /NZ|AU/}}]}).count() 811 = 471 812 813 db.Websites.aggregate([ 814 { 815 $match: { 816 $and: [{numPagesContainingMRI: {$gt: 0}},{urlContainsLangCodeInPath: true}, {domain: {$not: /.nz$/}}, {geoLocationCountryCode: {$not: /NZ|AU/}}] 817 } 818 }, 819 {$group: {_id: "$geoLocationCountryCode", count: {$sum: 1}, domain: { $addToSet: '$domain' }}}, 820 { $sort : { count : -1} } 821 ]) 822 823 824 /* 1 */ 825 { 826 "_id" : "US", 827 "count" : 305.0, 828 "domain" : [ 829 "http://www.qjfiberglass.com", 830 "http://www.sokenswitch.com", 831 "https://follow3rs.com", 832 "http://www.zjnbzy.com", 833 "http://www.nbbvc.com", 834 "https://mamaclub.info", 835 "https://www.nbwinwinea.com", 836 "https://www.sdspraybooth.com", 837 "https://www.jlextract.com", 838 "http://www.artmetalcn.com", 839 "http://www.steel-in-china.com", 840 "http://www.sunnymaycn.com", 841 "https://www.weld-automation.com", 842 "http://www.homewin88.com", 843 "http://indigenousblogs.com", 844 "https://biblia.gospelprime.com.br", 845 "http://www.jhc-nonwoven.com", 846 "http://www.forever-moving.com", 847 "http://www.eternal-friendship.com", 848 "http://www.conele-mixer.com", 849 "http://binaryoptionsindicators.com", 850 "http://www.mao-shuo.com", 851 "http://www.bmaxmachine.com", 852 "http://www.sdxhhd.com", 853 "http://lingeriefc.com", 854 "http://infomutt.com", 855 "https://jobdescriptionsample.org", 856 "http://mi.hongwugas.com", 857 "http://www.lishin.cc", 858 "http://www.mksmartcard.com", 859 "http://www.toption-ingredients.com", 860 "http://www.sps-squeegee.com", 861 "https://facebook.roseconverter.com", 862 "https://www.bestpvcfence.com", 863 "http://www.tubemillcn.com", 864 "http://www.jindunlaobao.com", 865 "http://www.wavesspring.com", 866 "http://www.restart-industry.com", 867 "https://mi.nyecountdown.com", 868 "http://www.ycautoc.com", 869 "http://www.htwindsolarpower.com", 870 "http://www.joyseaplywood.com", 871 "http://www.teda-hydraulic.com", 872 "http://www.gmk-valve.com", 873 "https://usahello.org", 874 "https://www.datemypet.com", 875 "https://worldstarhiphop.roseconverter.com", 876 "http://www.sdtzgloves.com", 877 "https://www.airpullfilter.com", 878 "http://www.bestwaytowhitenteethguide.org", 879 "https://mi.m.wikipedia.org", 880 "http://www.analiabriz.com", 881 "http://www.xida-electronics.com", 882 "http://www.weldpipemill.com", 883 "http://www.ictctruss.com", 884 "https://www.junschem.com", 885 "https://www.judinwire.net", 886 "http://www.mtpak.com", 887 "http://www.nantaidiesel.com", 888 "https://www.nbkeming.com", 889 "http://www.windsolarchina.com", 890 "http://www.gormeet.com", 891 "http://www.wf-fastener.com", 892 "http://www.pressurelantern.com", 893 "https://www.drickinstruments.com", 894 "http://milfsplease.com", 895 "http://www.risepipe.com", 896 "https://www.yourcloudlibrary.com", 897 "http://portal.smart-project.info", 898 "http://cdn.centrallanguageschool.com", 899 "http://www.shengxinsport.com", 900 "https://www.tianjia-lock.com", 901 "http://www.julongjewelry.cn", 902 "https://www.yogemcasting.com", 903 "http://www.chinaocan.com", 904 "http://www.autosunsoul.com", 905 "https://www.prostepper.com", 906 "https://www.pldyes.com", 907 "http://www.nicerelay.com", 908 "https://www.sinodryair.com", 909 "https://www.risenltd.com", 910 "http://www.albertnovosino.com", 911 "http://www.ttyzfilter.com", 912 "http://www.bst-elecs.com", 913 "http://www.hqftex.com", 914 "http://www.kehengmixing.com", 915 "http://loginmail.online", 916 "http://www.bluekin.com", 917 "https://blondewebcamgirl.com", 918 "https://www.nickel-alloy.net", 919 "https://www.hjfoodmachinery.com", 920 "http://csunplugged.org", 921 "https://www.inpnurseryproducts.com", 922 "http://www.americasportsfloor.com", 923 "http://www.dmdryer.com", 924 "http://www.sindadisplay.com", 925 "http://www.focusway-casting.com", 926 "https://mi.centr-zashity.ru", 927 "https://www.axnewdisplay.com", 928 "https://cycletraderpro.com", 929 "https://vk.roseconverter.com", 930 "https://www.livehoster.com", 931 "http://www.cnsongben.com", 932 "http://www.mytrickstips.com", 933 "http://www.quickcncmachine.com", 934 "http://www.arjextrailerparts.com", 935 "http://www.shshenyong.com", 936 "http://www.pvcroofingtile.com", 937 "http://www.wrdtubemill.com", 938 "http://church-of-christ.org", 939 "https://www.td-casting.com", 940 "https://www.hengweihoseclamp.com", 941 "http://www.wpcline.com", 942 "https://www.kubbamachine.com", 943 "http://www.goethe.de", 944 "https://www.njkeyuda.com", 945 "http://www.prostepper.com", 946 "http://www.cnfeinade.com", 947 "http://www.huamachinery.com", 948 "http://www.damiser.com", 949 "http://www.shanghailangzhiweld.com", 950 "http://www.fanhaopets.com", 951 "https://blockchains.io", 952 "http://www.inpnurseryproducts.com", 953 "http://www.yixinhetrade.com", 954 "http://www.newbaoquan.com", 955 "https://mi.lawyers.cafe", 956 "http://www.shenhe-bearing.com", 957 "http://atoall.com", 958 "http://www.vango-tech.com", 959 "https://www.gigalight.com", 960 "http://www.ladybagcn.com", 961 "http://www.tjcywires.com", 962 "http://www.vigor-industry.com", 963 "http://www.litbright-candles.com", 964 "http://www.nide-industry.com", 965 "http://www.cnfreda.com", 966 "http://www.jbpcba.com.cn", 967 "http://www.qitai-adhesive.com", 968 "http://www.weld-automation.com", 969 "http://www.cnyaonan.com", 970 "http://www.ruifeng-leather.com", 971 "http://www-hotmail-com.email", 972 "http://www.jointcontrols.net", 973 "https://twitter.roseconverter.com", 974 "https://www.aquagem.com.cn", 975 "http://www.seasum.cn", 976 "http://www.steelprotectionpack.com", 977 "http://www.suoxuehuwai.com", 978 "http://www.sunshinebelt.com", 979 "http://www.nyforgedwheels.com", 980 "http://www.amcbox.com", 981 "http://www.livepro-beauty.com", 982 "http://www.nbyobo.com", 983 "http://www.chinacarbonfibre.com", 984 "https://guidebooq.com", 985 "https://www.hello4x4.com", 986 "http://www.zhonghe222.com", 987 "http://www.church-of-christ.org", 988 "https://www.czzhit.com", 989 "https://www.king-pcb.com", 990 "http://www.secondhormone.com", 991 "http://www.sxceramic.com", 992 "http://www.hobbycarbon.com", 993 "http://www.bdknitting.com", 994 "http://www.ntvigourbrush.com", 995 "http://www.china-brewhouse.com", 996 "http://mi.tccasdic.com", 997 "http://www.hzhinew.com", 998 "http://www.silicone-odm.com", 999 "http://www.liweimetal.com", 1000 "http://www.huaxinfurnace.com", 1001 "http://www.envicool.net", 1002 "http://www.cnxh-electric.com", 1003 "http://www.jiejingfactory.com", 1004 "http://www.longda-inc.com", 1005 "http://www.pamaens.com", 1006 "http://www.sdcncrouter.com", 1007 "http://www.tkfanen.com", 1008 "http://www.touchdisplays-tech.com", 1009 "http://www.twtvalvecn.com", 1010 "http://www.weddingfurniture.com", 1011 "https://www.huadongmedical.com", 1012 "http://www.ledecofr.com", 1013 "http://www.rosin-kings.com", 1014 "http://www.aluminum-profiles-supplier.com", 1015 "http://www.cannapresso.com", 1016 "https://www.cz-juteng.com", 1017 "http://www.strongsaw.com", 1018 "http://jobdescriptionsample.org", 1019 "http://www.btmeac.com", 1020 "http://www.nicehut-window.com", 1021 "http://www.accotech.net", 1022 "https://www.dshprecision.com", 1023 "http://www.gemnice.com", 1024 "http://www.richina-tools.com", 1025 "http://www.brushcutterjusen.com", 1026 "http://www.szhaiwang.com", 1027 "https://www.conele-mixer.com", 1028 "https://www.tkthvac.com", 1029 "http://technobuzzer.com", 1030 "https://www.csunplugged.org", 1031 "http://www.ainuogas.com", 1032 "https://policies.oclc.org", 1033 "http://www.xfinsulation.com", 1034 "http://www.lanlinprintech.com", 1035 "http://www.yrkseal.com", 1036 "http://www.jpslurrypump.com", 1037 "http://www.soontruepackaging.com", 1038 "http://www.shengrunqiche.com", 1039 "http://www.luluae.com", 1040 "https://www.judipak.com", 1041 "http://www.cz-juteng.com", 1042 "http://www.jiajiebathmirror.com", 1043 "http://www.bigrollscloth.com", 1044 "http://www.chinatopcnc.com", 1045 "https://drugsinc.eu", 1046 "http://www.wosaicabinet.com", 1047 "http://www.wellfit-sportswear.com", 1048 "http://www.pxbaisheng.com", 1049 "http://www.meihua-wm.com", 1050 "http://www.wzdongyi.com", 1051 "http://www.kd-physicalrehab.com", 1052 "http://www.longs-motor.com", 1053 "https://www.samsungwiremesh.com", 1054 "http://www.wellformpacking.com", 1055 "http://www.hs-stationery.com", 1056 "http://www.allutertech.com", 1057 "http://www.czzhit.com", 1058 "http://www.jlgrating.com", 1059 "http://www.qbd-group.com", 1060 "http://www.evaescort.net", 1061 "https://dwsolo.com", 1062 "http://www.chuamotor.com", 1063 "http://www.ksdoing.com", 1064 "http://mi.broadcastbeat.com", 1065 "http://www.czldfloor.com", 1066 "http://www.qypaperbox.com", 1067 "https://mi.wikipedia.org", 1068 "http://www.houshenshoes.com", 1069 "http://www.xzc9.com", 1070 "http://www.chinacombinerbox.com", 1071 "https://www.everfineplastics.com", 1072 "http://www.sinemagnetic.com", 1073 "http://www.linphos.com", 1074 "https://www.rikoooo.com", 1075 "http://www.ncpcpharma.com", 1076 "http://www.evergrowingcage.com", 1077 "http://www.qxmic.com", 1078 "https://www.fxcc.com", 1079 "http://www.ldsolarpv.com", 1080 "http://mytrickstips.com", 1081 "http://www.linbaymachinery.com", 1082 "http://www.photoprofix.com", 1083 "http://www.supplyfurniture.com", 1084 "http://www.honglu-mining.com", 1085 "http://www.szebo.com", 1086 "http://www.cnrgxy.com", 1087 "http://blicanada.net", 1088 "http://www.homey-tec.com", 1089 "http://www.whties.com", 1090 "http://www.zhenchengscrew.com", 1091 "http://www.ruk-tech.com", 1092 "http://www.longxin-global.com", 1093 "https://www.tymexnetting.com", 1094 "http://www.chinabosun.com", 1095 "http://www.b-packaging.com", 1096 "http://www.ncpcvet.com", 1097 "https://mi.kidspicturedictionary.com", 1098 "http://mi.guoguangelectric.com", 1099 "http://topbitcoincard.com", 1100 "https://atoall.com", 1101 "http://www.acouplefortheroad.com", 1102 "http://www.tongyujiaju.com", 1103 "http://www.chinapipemills.com", 1104 "http://www.infomutt.com", 1105 "http://www.fxctool.com", 1106 "http://www.samewe.net", 1107 "https://www.aquark.com.cn", 1108 "https://www.artiegarden.com", 1109 "http://www.fxpremiere.com", 1110 "http://www.sog-pump.com", 1111 "http://www.omnicnc.com", 1112 "https://www.waterproof-factory.com", 1113 "http://www.wanmaroto.com", 1114 "http://mi.gmpmetalwork.com", 1115 "https://www.webhostingsecretrevealed.net", 1116 "http://www.gecko-kalimba.com", 1117 "https://www.glorystarlaser.com", 1118 "http://www.viairdoormat.com", 1119 "https://vimeo.roseconverter.com", 1120 "https://www.fctele.com", 1121 "http://www.hzzjair.com", 1122 "https://2fish.co", 1123 "http://www.qymachines.com", 1124 "http://www.chinachairtable.com", 1125 "http://www.gfh-electric.com", 1126 "http://www.tangres100.com", 1127 "https://www.valve-pipe-fitting.com", 1128 "http://www.fancyco.com", 1129 "http://www.zhengmaoelec.com", 1130 "http://www.chinagxmy.com", 1131 "https://www.tjshenzhoutong.com", 1132 "https://maxspeedtest.com" 1133 ] 1134 } 1135 1136 /* 2 */ 1137 { 1138 "_id" : "CN", 1139 "count" : 113.0, 1140 "domain" : [ 1141 "https://www.fibereye2.com", 1142 "https://www.outstandingdm.com", 1143 "https://www.szradiant.com", 1144 "http://www.gmmdjx.com", 1145 "http://www.likvchina.com", 1146 "https://www.abdindustrial.com", 1147 "https://www.c-superun.com", 1148 "https://www.slagremoving.com", 1149 "https://www.sino-masterbatch.com", 1150 "http://www.cntiescarf.com", 1151 "https://www.dm-compressor.com", 1152 "https://www.szhtpmart.com", 1153 "https://www.phhydraulic.com", 1154 "https://www.imposalight.com", 1155 "https://www.medke.com", 1156 "http://www.eburn-burner.com", 1157 "https://www.haitungchem.com", 1158 "http://www.medicohongkong.com", 1159 "http://www.koowheel.com", 1160 "https://www.aerial-display.com", 1161 "https://www.cntfsolar.com", 1162 "https://www.aoxinhvacr.com", 1163 "https://www.diamante-tech.com", 1164 "https://www.richest-group.com", 1165 "http://www.world-starter.com", 1166 "http://www.goldenlaser.cc", 1167 "https://www.km-medicine.com", 1168 "https://www.safesworld.com", 1169 "https://www.peptidejymed.com", 1170 "https://www.nbhengchen.com", 1171 "https://www.xinyuesteel.com", 1172 "https://www.charmingmetal.com", 1173 "https://www.lasonparts.com", 1174 "https://www.ngyc.com", 1175 "https://www.pacopower.com", 1176 "https://www.tjtgsteel.com", 1177 "http://www.abdindustrial.com", 1178 "https://www.yangrutingtrade.com", 1179 "http://www.wedacdisplays.com", 1180 "https://www.gaofeng-petro.com", 1181 "https://www.ez-walk.com", 1182 "https://www.szzhsbag.com", 1183 "https://www.simphoenix.com", 1184 "http://www.focuslasersystems.com", 1185 "https://www.fc-med.com", 1186 "http://www.zypackag.com", 1187 "http://www.kavounautoparts.com", 1188 "https://www.foocles.com", 1189 "https://www.jsjlmachinery.com", 1190 "https://www.special-metal.com", 1191 "https://www.bestardoors.com", 1192 "http://www.wenwencf.com", 1193 "https://www.insharevape.com", 1194 "https://www.dghk-buffer.com", 1195 "https://www.n2o2gas.com", 1196 "https://www.changjia-machinery.com", 1197 "https://www.nfyo.com", 1198 "http://www.estarspareparts.com", 1199 "https://www.jsbotanics.com", 1200 "https://www.chinarfidcard.com", 1201 "https://www.sjzhgw.com", 1202 "https://www.study-mandarin.com", 1203 "https://www.qdruidetai.com", 1204 "https://www.zhongxinlighting.com", 1205 "http://www.qjqdvalve.com", 1206 "https://www.painting-machine.com", 1207 "https://www.bescatray.com", 1208 "https://www.tianseoffice.com", 1209 "https://www.herbal-ingredients.com", 1210 "https://www.qlart.com", 1211 "https://www.sehenda-en.com", 1212 "https://www.egbadges.com", 1213 "http://www.eudemonbaby.com", 1214 "http://www.3drambery.com", 1215 "https://www.chinawelken.com", 1216 "http://www.jsbotanics.com", 1217 "https://www.rswires.com", 1218 "https://www.zjyongqi.com", 1219 "https://www.micropreparedslides.com", 1220 "http://www.longtopmining.com", 1221 "https://www.rykay.com", 1222 "https://www.sdtoplit.com", 1223 "https://www.wecare-life.com", 1224 "http://www.wigglewires.com", 1225 "https://www.grandstarcn.com", 1226 "https://www.bailixin.com", 1227 "http://www.refinehotelsupply.com", 1228 "http://www.prius-automatic.com", 1229 "https://www.nbulboy.com", 1230 "https://www.jy-glass.com", 1231 "http://www.ankaicnc.com", 1232 "https://www.band-ss.com", 1233 "https://www.hytokstech.com", 1234 "https://www.goldnard.com", 1235 "http://www.comfortebicycle.com", 1236 "https://www.zengrit.com", 1237 "https://www.3drambery.com", 1238 "https://www.pakite.com", 1239 "https://www.xianglin-plastics.com", 1240 "https://www.inductorchina.com", 1241 "https://www.nbjiatong.com", 1242 "https://www.bofanpc.com", 1243 "https://www.sakysteel.com", 1244 "http://www.coneleqd.com", 1245 "https://www.jewellrylove.com", 1246 "http://www.nbwellrun.com", 1247 "http://www.yulong-cellulose-cmc.com", 1248 "https://www.aootan.com", 1249 "https://www.coffbrewing.com", 1250 "http://www.jetwayamenities.com", 1251 "https://english.taiergroup.com", 1252 "http://www.czhengfa.com", 1253 "https://www.sitzonechair.com" 1254 ] 1255 } 1256 1257 /* 3 */ 1258 { 1259 "_id" : "FR", 1260 "count" : 19.0, 1261 "domain" : [ 1262 "https://www.slotsltd.com", 1263 "https://mi.apicmo.com", 1264 "http://mi.psychicbonus.com", 1265 "http://mi.aasraw.com", 1266 "https://mi.hyperbaric-chamber.com", 1267 "https://mi.usa-casino-online.com", 1268 "https://mi.gem.agency", 1269 "https://mi.hghphuket.com", 1270 "https://mi.mehmetdursun.av.tr", 1271 "https://mi.mhthread.com", 1272 "https://mi.phcoker.com", 1273 "https://www.casino.uk.com", 1274 "https://www.planetkeyboard.com", 1275 "http://mi.outboard-boat-motor-repair.com", 1276 "http://www.gpedia.com", 1277 "http://mi.fitnessrebates.com", 1278 "https://www.expresscasino.com", 1279 "https://mi.petrpikora.com", 1280 "https://mi.isearch.de" 1281 ] 1282 } 1283 1284 /* 4 */ 1285 { 1286 "_id" : "DE", 1287 "count" : 8.0, 1288 "domain" : [ 1289 "https://herocity.de", 1290 "https://traynews.com", 1291 "http://www.almancax.com", 1292 "https://transposh.org", 1293 "http://transposh.org", 1294 "https://mi.vessoft.com", 1295 "https://www.saper-link-news.com", 1296 "https://afrikhepri.org" 1297 ] 1298 } 1299 1300 /* 5 */ 1301 { 1302 "_id" : "NL", 1303 "count" : 6.0, 1304 "domain" : [ 1305 "http://www.martinvrijland.nl", 1306 "https://realtytenerife.com", 1307 "https://www.bitbybitbook.com", 1308 "https://www.emergency-live.com", 1309 "http://www.cbdolievoordelen.nl", 1310 "http://www.spectrumschool.be" 1311 ] 1312 } 1313 1314 /* 6 */ 1315 { 1316 "_id" : "CA", 1317 "count" : 5.0, 1318 "domain" : [ 1319 "https://www.wikiplanet.click", 1320 "https://cloudsfeed.com", 1321 "http://newsrule.com", 1322 "http://dehaut.com", 1323 "https://www.chinanbdb.com" 1324 ] 1325 } 1326 1327 /* 7 */ 1328 { 1329 "_id" : "HK", 1330 "count" : 2.0, 1331 "domain" : [ 1332 "https://www.desunpump.com", 1333 "http://www.10turntables.com" 1334 ] 1335 } 1336 1337 /* 8 */ 1338 { 1339 "_id" : "UA", 1340 "count" : 2.0, 1341 "domain" : [ 1342 "http://ukraine.admission.center", 1343 "http://umsa.admission.center" 1344 ] 1345 } 1346 1347 /* 9 */ 1348 { 1349 "_id" : "GB", 1350 "count" : 2.0, 1351 "domain" : [ 1352 "https://www.centrallanguageschool.com", 1353 "https://www.solasolv.com" 1354 ] 1355 } 1356 1357 /* 10 */ 1358 { 1359 "_id" : "UNKNOWN", 1360 "count" : 2.0, 1361 "domain" : [ 1362 "https://mi.buyaas.com", 1363 "http://en.wiki.wintoflash.com" 1364 ] 1365 } 1366 1367 /* 11 */ 1368 { 1369 "_id" : "ES", 1370 "count" : 1.0, 1371 "domain" : [ 1372 "https://www.torresbus.es" 1373 ] 1374 } 1375 1376 /* 12 */ 1377 { 1378 "_id" : "IE", 1379 "count" : 1.0, 1380 "domain" : [ 1381 "http://netkiosk.co.uk" 1382 ] 1383 } 1384 1385 /* 13 */ 1386 { 1387 "_id" : "RU", 1388 "count" : 1.0, 1389 "domain" : [ 1390 "http://www.treningmozga.com" 1391 ] 1392 } 1393 1394 /* 14 */ 1395 { 1396 "_id" : "SG", 1397 "count" : 1.0, 1398 "domain" : [ 1399 "https://omg-solutions.com" 1400 ] 1401 } 1402 1403 /* 15 */ 1404 { 1405 "_id" : "JP", 1406 "count" : 1.0, 1407 "domain" : [ 1408 "https://forexmania.org" 1409 ] 1410 } 1411 1412 /* 16 */ 1413 { 1414 "_id" : "EU", 1415 "count" : 1.0, 1416 "domain" : [ 1417 "http://www.the-good-stuff-factory.be" 1418 ] 1419 } 1420 1421 /* 17 */ 1422 { 1423 "_id" : "TR", 1424 "count" : 1.0, 1425 "domain" : [ 1426 "https://www.elitedeluxe.com.tr" 1427 ] 1428 } 818 (No longer special handling for AU, as we saw earlier.) 819 820 /* 1 */ 821 { 822 "_id" : "US", 823 "count" : 302.0, 824 "domain" : [ 825 "https://www.tkthvac.com", 826 "http://mi.gmpmetalwork.com", 827 "http://www.huaxinfurnace.com", 828 "http://mi.tccasdic.com", 829 "https://www.waterproof-factory.com", 830 "http://www.omnicnc.com", 831 "http://www.bdknitting.com", 832 "http://www.prostepper.com", 833 "http://www.tkfanen.com", 834 "http://www.sdcncrouter.com", 835 "http://www.china-brewhouse.com", 836 "http://www.twtvalvecn.com", 837 "http://www.zhengmaoelec.com", 838 "http://www.hqftex.com", 839 "https://mi.centr-zashity.ru", 840 "http://www.szebo.com", 841 "http://www.jointcontrols.net", 842 "http://www.hobbycarbon.com", 843 "https://www.nickel-alloy.net", 844 "http://www.10turntables.com", 845 "https://www.inpnurseryproducts.com", 846 "https://www.risenltd.com", 847 "http://www.ncpcpharma.com", 848 "http://www.weld-automation.com", 849 "http://www.gfh-electric.com", 850 "http://www.tongyujiaju.com", 851 "http://www.nide-industry.com", 852 "http://www.nicehut-window.com", 853 "http://www.acouplefortheroad.com", 854 "https://www.aquagem.com.cn", 855 "https://www.tymexnetting.com", 856 "http://www.ruk-tech.com", 857 "http://www.yrkseal.com", 858 "http://www.ainuogas.com", 859 "http://blicanada.net", 860 "http://www.goethe.de", 861 "https://www.njkeyuda.com", 862 "http://topbitcoincard.com", 863 "http://www.fancyco.com", 864 "http://www.chinagxmy.com", 865 "http://www.cnfeinade.com", 866 "http://www.longxin-global.com", 867 "http://www.nyforgedwheels.com", 868 "http://www.sog-pump.com", 869 "http://www.inpnurseryproducts.com", 870 "http://www.wanmaroto.com", 871 "http://www.yixinhetrade.com", 872 "http://www.b-packaging.com", 873 "http://www.bluekin.com", 874 "http://www.ncpcvet.com", 875 "https://www.glorystarlaser.com", 876 "http://www.shengrunqiche.com", 877 "http://www.wellfit-sportswear.com", 878 "http://csunplugged.org", 879 "https://www.kiwiproperty.com", 880 "http://www.infomutt.com", 881 "http://www.photoprofix.com", 882 "https://drugsinc.eu", 883 "http://www.ttyzfilter.com", 884 "http://www.nicerelay.com", 885 "https://www.gigalight.com", 886 "https://www.sinodryair.com", 887 "http://www.ladybagcn.com", 888 "http://www.cnrgxy.com", 889 "http://www.honglu-mining.com", 890 "http://www.kehengmixing.com", 891 "http://www.cnfreda.com", 892 "http://www.longs-motor.com", 893 "http://www.xzc9.com", 894 "http://www.dmdryer.com", 895 "http://www.ksdoing.com", 896 "http://www.mytrickstips.com", 897 "http://www.focusway-casting.com", 898 "http://www.americasportsfloor.com", 899 "https://cycletraderpro.com", 900 "http://www.chinabosun.com", 901 "https://www.everfineplastics.com", 902 "http://mi.guoguangelectric.com", 903 "http://www.albertnovosino.com", 904 "http://www.evergrowingcage.com", 905 "http://www.seasum.cn", 906 "http://www-hotmail-com.email", 907 "http://www.cnyaonan.com", 908 "http://www.ntvigourbrush.com", 909 "http://www.quickcncmachine.com", 910 "https://www.hengweihoseclamp.com", 911 "http://www.sokenswitch.com", 912 "http://www.soontruepackaging.com", 913 "https://www.rikoooo.com", 914 "http://www.cnxh-electric.com", 915 "http://www.teda-hydraulic.com", 916 "http://www.strongsaw.com", 917 "https://www.prostepper.com", 918 "http://www.pressurelantern.com", 919 "http://www.hs-stationery.com", 920 "http://www.nbbvc.com", 921 "http://lingeriefc.com", 922 "http://www.evaescort.net", 923 "http://www.kd-physicalrehab.com", 924 "http://www.chuamotor.com", 925 "http://cdn.centrallanguageschool.com", 926 "https://worldstarhiphop.roseconverter.com", 927 "https://www.csunplugged.org", 928 "http://www.qypaperbox.com", 929 "https://www.junschem.com", 930 "http://www.gormeet.com", 931 "http://www.szhaiwang.com", 932 "http://www.wzdongyi.com", 933 "http://www.jlgrating.com", 934 "http://www.nantaidiesel.com", 935 "http://www.zhenchengscrew.com", 936 "http://www.accotech.net", 937 "https://atoall.com", 938 "https://mi.wikipedia.org", 939 "https://usahello.org", 940 "http://www.gemnice.com", 941 "http://www.richina-tools.com", 942 "http://www.samewe.net", 943 "http://www.liweimetal.com", 944 "http://www.pxbaisheng.com", 945 "http://www.jiejingfactory.com", 946 "http://www.meihua-wm.com", 947 "http://www.jiajiebathmirror.com", 948 "http://www.touchdisplays-tech.com", 949 "http://www.sdtzgloves.com", 950 "http://www.forever-moving.com", 951 "http://www.cannapresso.com", 952 "http://www.aluminum-profiles-supplier.com", 953 "http://indigenousblogs.com", 954 "http://www.btmeac.com", 955 "http://www.longda-inc.com", 956 "http://www.conele-mixer.com", 957 "http://www.brushcutterjusen.com", 958 "https://mi.m.wikipedia.org", 959 "https://www.judinwire.net", 960 "http://www.toption-ingredients.com", 961 "https://www.fctele.com", 962 "http://www.ledecofr.com", 963 "https://www.drickinstruments.com", 964 "https://policies.oclc.org", 965 "http://www.lanlinprintech.com", 966 "http://www.qjfiberglass.com", 967 "https://www.huadongmedical.com", 968 "http://www.hzhinew.com", 969 "http://www.envicool.net", 970 "http://www.steel-in-china.com", 971 "https://mamaclub.info", 972 "https://www.conele-mixer.com", 973 "https://www.jlextract.com", 974 "http://www.chinaocan.com", 975 "http://www.htwindsolarpower.com", 976 "https://mi.nyecountdown.com", 977 "http://www.gecko-kalimba.com", 978 "https://www.tjshenzhoutong.com", 979 "http://www.vigor-industry.com", 980 "https://maxspeedtest.com", 981 "http://www.sunnymaycn.com", 982 "http://www.tangres100.com", 983 "http://www.bst-elecs.com", 984 "https://www.weld-automation.com", 985 "http://www.suoxuehuwai.com", 986 "http://www.steelprotectionpack.com", 987 "https://twitter.roseconverter.com", 988 "http://mytrickstips.com", 989 "http://binaryoptionsindicators.com", 990 "http://www.jhc-nonwoven.com", 991 "http://www.tjcywires.com", 992 "https://www.wikiplanet.click", 993 "http://infomutt.com", 994 "http://www.nbyobo.com", 995 "http://www.amcbox.com", 996 "http://www.fanhaopets.com", 997 "http://www.supplyfurniture.com", 998 "http://www.ruifeng-leather.com", 999 "https://mi.lawyers.cafe", 1000 "http://www.vango-tech.com", 1001 "http://www.viairdoormat.com", 1002 "https://2fish.co", 1003 "http://atoall.com", 1004 "http://www.qymachines.com", 1005 "https://www.aquark.com.cn", 1006 "http://www.church-of-christ.org", 1007 "http://www.litbright-candles.com", 1008 "https://www.nbwinwinea.com", 1009 "https://www.bestpvcfence.com", 1010 "http://www.chinachairtable.com", 1011 "http://www.zhonghe222.com", 1012 "http://church-of-christ.org", 1013 "http://www.lishin.cc", 1014 "https://www.webhostingsecretrevealed.net", 1015 "http://www.damiser.com", 1016 "http://www.hzzjair.com", 1017 "http://www.sxceramic.com", 1018 "http://www.fxctool.com", 1019 "http://www.livepro-beauty.com", 1020 "https://www.pldyes.com", 1021 "https://vimeo.roseconverter.com", 1022 "http://www.chinapipemills.com", 1023 "http://www.shanghailangzhiweld.com", 1024 "https://mi.kidspicturedictionary.com", 1025 "http://www.ldsolarpv.com", 1026 "https://www.fxcc.com", 1027 "https://www.kubbamachine.com", 1028 "http://www.linbaymachinery.com", 1029 "https://www.axnewdisplay.com", 1030 "http://www.whties.com", 1031 "http://www.homey-tec.com", 1032 "http://www.arjextrailerparts.com", 1033 "http://www.julongjewelry.cn", 1034 "https://www.livehoster.com", 1035 "http://www.risepipe.com", 1036 "http://www.wrdtubemill.com", 1037 "http://www.sunshinebelt.com", 1038 "https://www.yourcloudlibrary.com", 1039 "http://loginmail.online", 1040 "http://www.shengxinsport.com", 1041 "http://www.fxpremiere.com", 1042 "https://www.czzhit.com", 1043 "https://www.king-pcb.com", 1044 "http://www.wpcline.com", 1045 "http://portal.smart-project.info", 1046 "http://www.qxmic.com", 1047 "http://www.luluae.com", 1048 "https://www.datemypet.com", 1049 "http://www.gmk-valve.com", 1050 "https://www.sdspraybooth.com", 1051 "http://www.houshenshoes.com", 1052 "http://www.homewin88.com", 1053 "http://www.sdxhhd.com", 1054 "http://www.bmaxmachine.com", 1055 "http://www.bestwaytowhitenteethguide.org", 1056 "http://www.linphos.com", 1057 "http://www.analiabriz.com", 1058 "http://www.joyseaplywood.com", 1059 "http://www.chinatopcnc.com", 1060 "https://blondewebcamgirl.com", 1061 "http://www.czzhit.com", 1062 "https://www.judipak.com", 1063 "http://www.sindadisplay.com", 1064 "http://www.wellformpacking.com", 1065 "http://www.wosaicabinet.com", 1066 "http://www.windsolarchina.com", 1067 "http://www.sinemagnetic.com", 1068 "http://www.ictctruss.com", 1069 "http://www.shshenyong.com", 1070 "http://www.pvcroofingtile.com", 1071 "http://www.mtpak.com", 1072 "http://www.tubemillcn.com", 1073 "http://www.weldpipemill.com", 1074 "http://www.xida-electronics.com", 1075 "http://www.cnsongben.com", 1076 "https://www.nbkeming.com", 1077 "http://www.jpslurrypump.com", 1078 "http://www.cz-juteng.com", 1079 "https://vk.roseconverter.com", 1080 "http://www.sps-squeegee.com", 1081 "http://mi.broadcastbeat.com", 1082 "https://www.td-casting.com", 1083 "http://milfsplease.com", 1084 "http://www.qbd-group.com", 1085 "http://technobuzzer.com", 1086 "https://www.cz-juteng.com", 1087 "http://www.xfinsulation.com", 1088 "http://www.wavesspring.com", 1089 "http://www.bigrollscloth.com", 1090 "http://www.huamachinery.com", 1091 "http://www.restart-industry.com", 1092 "http://www.shenhe-bearing.com", 1093 "http://www.newbaoquan.com", 1094 "https://follow3rs.com", 1095 "https://www.airpullfilter.com", 1096 "http://www.mao-shuo.com", 1097 "http://mi.hongwugas.com", 1098 "http://www.pamaens.com", 1099 "http://www.weddingfurniture.com", 1100 "http://www.mksmartcard.com", 1101 "http://jobdescriptionsample.org", 1102 "http://www.jbpcba.com.cn", 1103 "https://biblia.gospelprime.com.br", 1104 "https://blockchains.io", 1105 "http://www.qitai-adhesive.com", 1106 "http://www.jindunlaobao.com", 1107 "https://jobdescriptionsample.org", 1108 "https://www.samsungwiremesh.com", 1109 "http://www.eternal-friendship.com", 1110 "http://www.rosin-kings.com", 1111 "https://facebook.roseconverter.com", 1112 "https://www.yogemcasting.com", 1113 "http://www.chinacombinerbox.com", 1114 "https://dwsolo.com", 1115 "http://www.autosunsoul.com", 1116 "https://www.hello4x4.com", 1117 "http://www.silicone-odm.com", 1118 "http://www.wf-fastener.com", 1119 "http://www.czldfloor.com", 1120 "http://www.zjnbzy.com", 1121 "http://www.secondhormone.com", 1122 "http://www.artmetalcn.com", 1123 "http://www.ycautoc.com", 1124 "http://www.chinacarbonfibre.com", 1125 "https://guidebooq.com" 1126 ] 1127 } 1128 1129 /* 2 */ 1130 { 1131 "_id" : "CN", 1132 "count" : 118.0, 1133 "domain" : [ 1134 "https://www.qlart.com", 1135 "https://www.grandstarcn.com", 1136 "https://www.valve-pipe-fitting.com", 1137 "http://www.wedacdisplays.com", 1138 "http://www.goldenlaser.cc", 1139 "https://www.cntfsolar.com", 1140 "http://www.abdindustrial.com", 1141 "http://www.koowheel.com", 1142 "https://www.gaofeng-petro.com", 1143 "https://www.nbhengchen.com", 1144 "http://www.jsbotanics.com", 1145 "https://www.simphoenix.com", 1146 "https://www.bestardoors.com", 1147 "https://www.n2o2gas.com", 1148 "https://www.charmingmetal.com", 1149 "https://www.fc-med.com", 1150 "http://www.focuslasersystems.com", 1151 "https://www.nfyo.com", 1152 "http://www.zypackag.com", 1153 "http://www.kavounautoparts.com", 1154 "https://www.jsjlmachinery.com", 1155 "https://www.tjtgsteel.com", 1156 "https://www.yangrutingtrade.com", 1157 "https://www.c-superun.com", 1158 "https://www.lasonparts.com", 1159 "https://www.special-metal.com", 1160 "https://www.szhtpmart.com", 1161 "https://www.chinarfidcard.com", 1162 "https://www.ez-walk.com", 1163 "https://www.diamante-tech.com", 1164 "https://www.sino-masterbatch.com", 1165 "https://www.medke.com", 1166 "https://www.dm-compressor.com", 1167 "https://www.haitungchem.com", 1168 "http://www.wenwencf.com", 1169 "https://www.peptidejymed.com", 1170 "https://www.slagremoving.com", 1171 "https://www.chinanbdb.com", 1172 "http://www.gmmdjx.com", 1173 "https://www.richest-group.com", 1174 "http://www.world-starter.com", 1175 "http://www.medicohongkong.com", 1176 "http://www.jetwayamenities.com", 1177 "https://www.abdindustrial.com", 1178 "https://www.artiegarden.com", 1179 "https://www.outstandingdm.com", 1180 "https://www.aoxinhvacr.com", 1181 "https://www.safesworld.com", 1182 "https://www.ngyc.com", 1183 "https://www.szradiant.com", 1184 "https://www.3drambery.com", 1185 "https://www.xianglin-plastics.com", 1186 "http://www.cntiescarf.com", 1187 "https://www.aerial-display.com", 1188 "https://www.imposalight.com", 1189 "https://www.pacopower.com", 1190 "http://www.eburn-burner.com", 1191 "https://www.szzhsbag.com", 1192 "https://www.phhydraulic.com", 1193 "https://www.bofanpc.com", 1194 "http://www.comfortebicycle.com", 1195 "http://www.3drambery.com", 1196 "https://www.pakite.com", 1197 "https://www.inductorchina.com", 1198 "https://www.aootan.com", 1199 "https://www.micropreparedslides.com", 1200 "https://www.tianjia-lock.com", 1201 "https://english.taiergroup.com", 1202 "https://www.hytokstech.com", 1203 "http://www.czhengfa.com", 1204 "http://www.ankaicnc.com", 1205 "https://www.nbulboy.com", 1206 "http://www.eudemonbaby.com", 1207 "http://www.coneleqd.com", 1208 "https://www.band-ss.com", 1209 "https://www.coffbrewing.com", 1210 "https://www.km-medicine.com", 1211 "https://www.jy-glass.com", 1212 "https://www.changjia-machinery.com", 1213 "https://www.zengrit.com", 1214 "http://www.prius-automatic.com", 1215 "https://www.sitzonechair.com", 1216 "https://www.goldnard.com", 1217 "https://www.bescatray.com", 1218 "http://www.qjqdvalve.com", 1219 "http://www.yulong-cellulose-cmc.com", 1220 "https://www.sakysteel.com", 1221 "https://www.tianseoffice.com", 1222 "http://www.likvchina.com", 1223 "https://www.sehenda-en.com", 1224 "http://www.nbwellrun.com", 1225 "https://www.painting-machine.com", 1226 "https://www.sdtoplit.com", 1227 "https://www.jewellrylove.com", 1228 "https://www.fibereye2.com", 1229 "https://www.dghk-buffer.com", 1230 "https://www.rykay.com", 1231 "https://www.wecare-life.com", 1232 "https://www.foocles.com", 1233 "http://www.estarspareparts.com", 1234 "https://www.study-mandarin.com", 1235 "https://www.dshprecision.com", 1236 "https://www.jsbotanics.com", 1237 "https://www.zhongxinlighting.com", 1238 "http://www.refinehotelsupply.com", 1239 "http://www.longtopmining.com", 1240 "https://www.insharevape.com", 1241 "https://www.xinyuesteel.com", 1242 "https://www.herbal-ingredients.com", 1243 "http://www.wigglewires.com", 1244 "https://www.bailixin.com", 1245 "https://www.egbadges.com", 1246 "https://www.qdruidetai.com", 1247 "https://www.sjzhgw.com", 1248 "https://www.zjyongqi.com", 1249 "https://www.rswires.com", 1250 "https://www.chinawelken.com", 1251 "https://www.nbjiatong.com" 1252 ] 1253 } 1254 1255 /* 3 */ 1256 { 1257 "_id" : "FR", 1258 "count" : 19.0, 1259 "domain" : [ 1260 "https://mi.hyperbaric-chamber.com", 1261 "https://mi.mehmetdursun.av.tr", 1262 "https://www.planetkeyboard.com", 1263 "https://mi.mhthread.com", 1264 "https://mi.gem.agency", 1265 "http://mi.outboard-boat-motor-repair.com", 1266 "https://www.slotsltd.com", 1267 "http://www.gpedia.com", 1268 "http://mi.aasraw.com", 1269 "http://mi.fitnessrebates.com", 1270 "https://mi.petrpikora.com", 1271 "https://mi.phcoker.com", 1272 "https://www.casino.uk.com", 1273 "https://mi.hghphuket.com", 1274 "https://mi.apicmo.com", 1275 "https://mi.isearch.de", 1276 "https://www.expresscasino.com", 1277 "https://mi.usa-casino-online.com", 1278 "http://mi.psychicbonus.com" 1279 ] 1280 } 1281 1282 /* 4 */ 1283 { 1284 "_id" : "DE", 1285 "count" : 7.0, 1286 "domain" : [ 1287 "https://afrikhepri.org", 1288 "https://mi.vessoft.com", 1289 "http://transposh.org", 1290 "https://transposh.org", 1291 "https://www.saper-link-news.com", 1292 "https://herocity.de", 1293 "https://traynews.com" 1294 ] 1295 } 1296 1297 /* 5 */ 1298 { 1299 "_id" : "NL", 1300 "count" : 6.0, 1301 "domain" : [ 1302 "http://www.cbdolievoordelen.nl", 1303 "https://www.emergency-live.com", 1304 "http://www.martinvrijland.nl", 1305 "https://realtytenerife.com", 1306 "https://www.bitbybitbook.com", 1307 "http://www.spectrumschool.be" 1308 ] 1309 } 1310 1311 /* 6 */ 1312 { 1313 "_id" : "UNKNOWN", 1314 "count" : 3.0, 1315 "domain" : [ 1316 "https://mi.buyaas.com", 1317 "https://www.hjfoodmachinery.com", 1318 "https://www.desunpump.com" 1319 ] 1320 } 1321 1322 /* 7 */ 1323 { 1324 "_id" : "CA", 1325 "count" : 3.0, 1326 "domain" : [ 1327 "https://cloudsfeed.com", 1328 "http://dehaut.com", 1329 "http://newsrule.com" 1330 ] 1331 } 1332 1333 /* 8 */ 1334 { 1335 "_id" : "UA", 1336 "count" : 2.0, 1337 "domain" : [ 1338 "http://ukraine.admission.center", 1339 "http://umsa.admission.center" 1340 ] 1341 } 1342 1343 /* 9 */ 1344 { 1345 "_id" : "GB", 1346 "count" : 2.0, 1347 "domain" : [ 1348 "https://www.centrallanguageschool.com", 1349 "https://www.solasolv.com" 1350 ] 1351 } 1352 1353 /* 10 */ 1354 { 1355 "_id" : "AU", 1356 "count" : 1.0, 1357 "domain" : [ 1358 "http://www.almancax.com" 1359 ] 1360 } 1361 1362 /* 11 */ 1363 { 1364 "_id" : "SG", 1365 "count" : 1.0, 1366 "domain" : [ 1367 "https://omg-solutions.com" 1368 ] 1369 } 1370 1371 /* 12 */ 1372 { 1373 "_id" : "EU", 1374 "count" : 1.0, 1375 "domain" : [ 1376 "http://www.the-good-stuff-factory.be" 1377 ] 1378 } 1379 1380 /* 13 */ 1381 { 1382 "_id" : "RU", 1383 "count" : 1.0, 1384 "domain" : [ 1385 "http://www.treningmozga.com" 1386 ] 1387 } 1388 1389 /* 14 */ 1390 { 1391 "_id" : "HK", 1392 "count" : 1.0, 1393 "domain" : [ 1394 "http://www.allutertech.com" 1395 ] 1396 } 1397 1398 /* 15 */ 1399 { 1400 "_id" : "IE", 1401 "count" : 1.0, 1402 "domain" : [ 1403 "http://netkiosk.co.uk" 1404 ] 1405 } 1406 1407 /* 16 */ 1408 { 1409 "_id" : "TR", 1410 "count" : 1.0, 1411 "domain" : [ 1412 "https://www.elitedeluxe.com.tr" 1413 ] 1414 } 1415 1416 /* 17 */ 1417 { 1418 "_id" : "JP", 1419 "count" : 1.0, 1420 "domain" : [ 1421 "https://forexmania.org" 1422 ] 1423 } 1424 1425 /* 18 */ 1426 { 1427 "_id" : "ES", 1428 "count" : 1.0, 1429 "domain" : [ 1430 "https://www.torresbus.es" 1431 ] 1432 } 1433 1434 /* 19 */ 1435 { 1436 "_id" : "SE", 1437 "count" : 1.0, 1438 "domain" : [ 1439 "http://en.wiki.wintoflash.com" 1440 ] 1441 } 1429 1442 1430 1443 … … 1433 1446 Of interest or possible interest remain the following, grouped per country of site origin: 1434 1447 1435 US: 1448 US: 1449 + GAINED FROM AU: https://www.kiwiproperty.com - e.g. https://www.kiwiproperty.com/the-base/mi/he-paepaki/ has some actual MRI sentences. [Not autotranslated] 1450 1436 1451 !! http://indigenousblogs.com [15/18 blogs work] - has one page in Maori (http://indigenousblogs.com/feeds/mi.xml) 1437 1452 X https://biblia.gospelprime.com.br - misdetection (containsMRI) … … 1474 1489 1475 1490 TOTALS: 1476 US: 2 5+4 from US with mi in URL path = 291477 AU: 21491 US: 26+5 from US with mi in URL path = 31 1492 AU: 1 1478 1493 DE: 2 1479 1494 DK: 2 … … 1483 1498 FR: 1 1484 1499 IE: 1 1485 TOTAL: 213+4 from US with mi in URL path = 216 1500 TOTAL: 212+5 from US with mi in URL path = 217 1501 1486 1502 ------------------------------------------------ 1487 1503 B. NEW ZEALAND SITES: NZ origin + .nz TLD SITES … … 2037 2053 IE, Ireland: 1 out of 2 2038 2054 2039 TOTAL: 16 6sites of all the crawled sites where the crawled set of pages per site actually contained at least one sentence in MÄori based on manual inspection.2040 Out of a total of 221+471+176 = 86 8sites that were detected with numPagesContainingMRI > 0 (868 sites containing at least one page with at least one sentence detected in MRI)2055 TOTAL: 167 sites of all the crawled sites where the crawled set of pages per site actually contained at least one sentence in MÄori based on manual inspection. 2056 Out of a total of 221+471+176 = 869 sites that were detected with numPagesContainingMRI > 0 (868 sites containing at least one page with at least one sentence detected in MRI) 2041 2057 2042 2058 ======================================== 2043 Adjusted grand totals in manualShortlisting.txtwith the following.2059 In the 2nd table (immediately above), I've adjusted grand totals with the following. 2044 2060 2045 2061 ----------------------------------------------------------------------
Note:
See TracChangeset
for help on using the changeset viewer.