Changeset 33722 for other-projects

Timestamp:

2019-11-25T21:29:42+13:00 (4 years ago)

Author:

ak19

Message:

Adding in additional instructions in mongodb.txt, before I forgot how to do basic things like run the mongo server on the vagrant VM and which vagrant VM it was, and how to connect a client to it. Also instructions on importing a csv file as a new collection into mongodb. And the csv file that Dr Bainbridge found for geo locations associated with country codes.

Location:

other-projects/maori-lang-detection/MoreReading

Files:

• countrycodes.csv (added)
• mongodb.txt (modified) (4 diffs)

other-projects/maori-lang-detection/MoreReading/mongodb.txt

@@ -311 +311 @@
 
 -----------------------
+Vagrant virtual machine Node1 has the mongodb installed.
+
+After doing "vagrant up" on node1 to start node1:
+
+    [anupama@analytics vagrant-hadoop-hive-spark]$ vagrant ssh
+    vagrant@node1:~$ mongo
+    MongoDB shell version v4.2.1
+    connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
+    2019-11-13T09:22:46.996+0000 E  QUERY    [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
+    connect@src/mongo/shell/mongo.js:341:17
+    @(connect):2:6
+    2019-11-13T09:22:46.999+0000 F  -        [main] exception: connect failed
+    2019-11-13T09:22:46.999+0000 E  -        [main] exiting with code 1
+    vagrant@node1:~$ sudo service mongod status
+    ● mongod.service - MongoDB Database Server
+       Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
+       Active: inactive (dead)
+         Docs: https://docs.mongodb.org/manual
+    vagrant@node1:~$ sudo service mongod start
+    vagrant@node1:~$ sudo service mongod status
+    ● mongod.service - MongoDB Database Server
+       Loaded: loaded (/lib/systemd/system/mongod.service; disabled; vendor preset: enabled)
+       Active: active (running) since Wed 2019-11-13 09:24:07 UTC; 2s ago
+         Docs: https://docs.mongodb.org/manual
+     Main PID: 4383 (mongod)
+        Tasks: 32
+       Memory: 199.3M
+          CPU: 754ms
+       CGroup: /system.slice/mongod.service
+           └─4383 /usr/bin/mongod --config /etc/mongod.conf
+
+    Nov 13 09:24:07 node1 systemd[1]: Started MongoDB Database Server.
+    vagrant@node1:~$ 
+
+
+So now mongodb is running on node1 on localhost:27017.
+
+Next, in another x-term connected to analytics' node1 Vagrant VM, port forward node1's localhost:27017 to analytics' localhost:27017:
+    vagrant ssh -- -L 27017:localhost:27017
+
+
+
+Finally, in another x-term, port-forward from analytics:27017 to current machine's 27017:
+    ssh -L 27017:localhost:27017 analytics
+
+
+Now can connect Robo-3T running on current machine to localhost:27017.
+
+Then in a new x-term, can use the client mongo shell to connect (by default to localhost:27017):
+    
+    wharariki:[122]/Scratch/ak19/GS309>mongo --shell
+    MongoDB shell version v4.0.13
+    connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
+    ...
+    > show dbs
+    admin          0.000GB
+    ateacrawldata  1.532GB
+    config         0.000GB
+    local          0.000GB
+    > use ateacrawldata
+
+    > show collections
+    Webpages
+    Websites
+    oldwebpages
+    oldwebsites
+-------------------
+
+Country code to geolocation CSV file found by Dr Bainbridge:
+https://developers.google.com/public-data/docs/canonical/countries_csv
+
+Import into mongodb with:
+https://stackoverflow.com/questions/4686500/how-to-use-mongoimport-to-import-csv
+
+
+
+NOTE: mongoimport is a commandline utility and not a command to be run from the mongo shell. See https://jira.mongodb.org/browse/DOCS-11072
+This means, in an x-term, DON'T RUN MONGO SHELL/client first. Instead, directly from x-term, run the following to import the countrycodes.csv file:
+
+
+    mongoimport -d ateacrawldata -c countrylocations --type csv --file /Scratch/ak19/maori-lang-detection/MoreReading/countrycodes.csv --headerline
+
+
+-------------------------
 
 MONGODB QUERIES:
@@ -474 +558 @@
         }
     },
-    { $sort : { count : -1} },
-    { $limit : 100 }
+    { $sort : { count : -1} }
 ]);
 
@@ -493 +576 @@
         }
     },
-    { $sort : { count : -1} },
-    { $limit : 100 }
+    { $sort : { count : -1} }
 ]);
 
@@ -511 +593 @@
         }
     },
-    { $sort : { count : -1} },
-    { $limit : 100 }
+    { $sort : { count : -1} }
 ]);