Changeset 33722

Show
Ignore:
Timestamp:
25.11.2019 21:29:42 (11 days ago)
Author:
ak19
Message:

Adding in additional instructions in mongodb.txt, before I forgot how to do basic things like run the mongo server on the vagrant VM and which vagrant VM it was, and how to connect a client to it. Also instructions on importing a csv file as a new collection into mongodb. And the csv file that Dr Bainbridge found for geo locations associated with country codes.

Location:
other-projects/maori-lang-detection/MoreReading
Files:
1 added
1 modified

Legend:

Unmodified
Added
Removed
  • other-projects/maori-lang-detection/MoreReading/mongodb.txt

    r33710 r33722  
    311311 
    312312----------------------- 
     313Vagrant virtual machine Node1 has the mongodb installed. 
     314 
     315After doing "vagrant up" on node1 to start node1: 
     316 
     317    [anupama@analytics vagrant-hadoop-hive-spark]$ vagrant ssh 
     318    vagrant@node1:~$ mongo 
     319    MongoDB shell version v4.2.1 
     320    connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb 
     321    2019-11-13T09:22:46.996+0000 E  QUERY    [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused : 
     322    connect@src/mongo/shell/mongo.js:341:17 
     323    @(connect):2:6 
     324    2019-11-13T09:22:46.999+0000 F  -        [main] exception: connect failed 
     325    2019-11-13T09:22:46.999+0000 E  -        [main] exiting with code 1 
     326    vagrant@node1:~$ sudo service mongod status 
     327    ● mongod.service - MongoDB Database Server 
     328       Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled) 
     329       Active: inactive (dead) 
     330         Docs: https://docs.mongodb.org/manual 
     331    vagrant@node1:~$ sudo service mongod start 
     332    vagrant@node1:~$ sudo service mongod status 
     333    ● mongod.service - MongoDB Database Server 
     334       Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled) 
     335       Active: active (running) since Wed 2019-11-13 09:24:07 UTC; 2s ago 
     336         Docs: https://docs.mongodb.org/manual 
     337     Main PID: 4383 (mongod) 
     338        Tasks: 32 
     339       Memory: 199.3M 
     340          CPU: 754ms 
     341       CGroup: /system.slice/mongod.service 
     342           â””─4383 /usr/bin/mongod --config /etc/mongod.conf 
     343 
     344    Nov 13 09:24:07 node1 systemd[1]: Started MongoDB Database Server. 
     345    vagrant@node1:~$  
     346 
     347 
     348So now mongodb is running on node1 on localhost:27017. 
     349 
     350Next, in another x-term connected to analytics' node1 Vagrant VM, port forward node1's localhost:27017 to analytics' localhost:27017: 
     351    vagrant ssh -- -L 27017:localhost:27017 
     352 
     353 
     354 
     355Finally, in another x-term, port-forward from analytics:27017 to current machine's 27017: 
     356    ssh -L 27017:localhost:27017 analytics 
     357 
     358 
     359Now can connect Robo-3T running on current machine to localhost:27017. 
     360 
     361Then in a new x-term, can use the client mongo shell to connect (by default to localhost:27017): 
     362     
     363    wharariki:[122]/Scratch/ak19/GS309>mongo --shell 
     364    MongoDB shell version v4.0.13 
     365    connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb 
     366    ... 
     367    > show dbs 
     368    admin          0.000GB 
     369    ateacrawldata  1.532GB 
     370    config         0.000GB 
     371    local          0.000GB 
     372    > use ateacrawldata 
     373 
     374    > show collections 
     375    Webpages 
     376    Websites 
     377    oldwebpages 
     378    oldwebsites 
     379------------------- 
     380 
     381Country code to geolocation CSV file found by Dr Bainbridge: 
     382https://developers.google.com/public-data/docs/canonical/countries_csv 
     383 
     384Import into mongodb with: 
     385https://stackoverflow.com/questions/4686500/how-to-use-mongoimport-to-import-csv 
     386 
     387 
     388 
     389NOTE: mongoimport is a commandline utility and not a command to be run from the mongo shell. See https://jira.mongodb.org/browse/DOCS-11072 
     390This means, in an x-term, DON'T RUN MONGO SHELL/client first. Instead, directly from x-term, run the following to import the countrycodes.csv file: 
     391 
     392 
     393    mongoimport -d ateacrawldata -c countrylocations --type csv --file /Scratch/ak19/maori-lang-detection/MoreReading/countrycodes.csv --headerline 
     394 
     395 
     396------------------------- 
    313397 
    314398MONGODB QUERIES: 
     
    474558        } 
    475559    }, 
    476     { $sort : { count : -1} }, 
    477     { $limit : 100 } 
     560    { $sort : { count : -1} } 
    478561]); 
    479562 
     
    493576        } 
    494577    }, 
    495     { $sort : { count : -1} }, 
    496     { $limit : 100 } 
     578    { $sort : { count : -1} } 
    497579]); 
    498580 
     
    511593        } 
    512594    }, 
    513     { $sort : { count : -1} }, 
    514     { $limit : 100 } 
     595    { $sort : { count : -1} } 
    515596]); 
    516597