Changeset 33621

Show
Ignore:
Timestamp:
04.11.2019 20:35:59 (10 days ago)
Author:
ak19
Message:

Comitting jotted down mongodb related instructions from what Dr Bainbridge did today.

Files:
1 modified

Legend:

Unmodified
Added
Removed
  • gs3-extensions/maori-lang-detection/MoreReading/crawling-Nutch.txt

    r33615 r33621  
    351351    http://central.maven.org/maven2/org/mongodb/mongo-java-driver/ 
    352352    https://mongodb.github.io/mongo-java-driver/ 
     353 
     354 
     355INSTALLING THE MONGODB SERVER AND MONGO CLIENT ON LINUX 
     356Need to have sudo and root powers. 
     357 
     358https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/ 
     359http://www.programmersought.com/article/6500308940/ 
     360 
     361   52  sudo apt-get install mongodb-clients 
     362   53  mongo 'mongodb://mongodb.cms.waikato.ac.nz:27017' -u anupama -p 
     363 
     364Failed with 
     365    Error: HostAndPort: host is empty at src/mongo/shell/mongo.js:148 
     366    exception: connect failed 
     367 
     368This is due to a version incompatibility between Client and mongodb Server. 
     369The solution is to follow instructions at http://www.programmersought.com/article/6500308940/ 
     370and then https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/ 
     371as below: 
     372 
     373   54  sudo apt-get purge mongodb-clients 
     374   55  sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4 
     375   56  echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list 
     376   57  sudo apt-get update 
     377   58  sudo apt-get install mongodb-clients 
     378   59  mongo 'mongodb://mongodb.cms.waikato.ac.nz:27017' -u anupama -p 
     379(still doesn't work) 
     380   60  sudo apt-get install -y mongodb-org 
     381The above ensures an up to date mongo client but installs the mongodb server too. Maybe this is the only step that is needed to install up-to-date mongo client and mongodb server? 
     382   72  sudo service mongod status 
     383 
     384  103  sudo service mongod start 
     385"mongod" stands for mongo-daemon. This runs the mongo db server listening for client connections 
     386  104  sudo service mongod status 
     387   88  sudo service mongod stop 
     388 
     389 
     390RUNNING AND USING THE MONGO CLIENT SHELL: 
     391Among the many things you can do with the Mongo client shell, one can use it to find the mongo client version (which is the version of the shell) and the mongo db version. 
     392 
     393To run the mongo client shell WITHOUT loading a db: 
     394 
     395 
     396    wharariki:[880]/Scratch/ak19/gs3-extensions/maori-lang-detection>mongo --shell -nodb 
     397    MongoDB shell version: 2.6.10 <<<<<<<<<-------------------<<<< MONGO CLIENT VERSION 
     398 
     399    type "help" for help 
     400    > help 
     401        db.help()                    help on db methods 
     402        db.mycoll.help()             help on collection methods 
     403        sh.help()                    sharding helpers 
     404        rs.help()                    replica set helpers 
     405        help admin                   administrative help 
     406        help connect                 connecting to a db help 
     407        help keys                    key shortcuts 
     408        help misc                    misc things to know 
     409        help mr                      mapreduce 
     410 
     411        show dbs                     show database names 
     412        show collections             show collections in current database 
     413        show users                   show users in current database 
     414        show profile                 show most recent system.profile entries with time >= 1ms 
     415        show logs                    show the accessible logger names 
     416        show log [name]              prints out the last segment of log in memory, 'global' is default 
     417        use <db_name>                set current database 
     418        db.foo.find()                list objects in collection foo 
     419        db.foo.find( { a : 1 } )     list objects in foo where a == 1 
     420        it                           result of the last line evaluated; use to further iterate 
     421        DBQuery.shellBatchSize = x   set default number of items to display on shell 
     422        exit                         quit the mongo shell 
     423 
     424    > help connect 
     425 
     426    Normally one specifies the server on the mongo shell command line.  Run mongo --help to see those options. 
     427    Additional connections may be opened: 
     428 
     429        var x = new Mongo('host[:port]'); 
     430        var mydb = x.getDB('mydb'); 
     431      or 
     432        var mydb = connect('host[:port]/mydb'); 
     433 
     434    Note: the REPL prompt only auto-reports getLastError() for the shell command line connection. 
     435 
     436    Getting help on connect options:  
     437 
     438    > var x = new Mongo('mongodb.cms.waikato.ac.nz:27017'); 
     439    > var mydb = x.getDB('anupama'); 
     440 
     441    > mydb.connect.help() 
     442    DBCollection help 
     443        db.connect.find().help() - show DBCursor help 
     444        db.connect.count() 
     445        db.connect.copyTo(newColl) - duplicates collection by copying all documents to newColl; no indexes are copied. 
     446        db.connect.convertToCapped(maxBytes) - calls {convertToCapped:'connect', size:maxBytes}} command 
     447        db.connect.dataSize() 
     448        db.connect.distinct( key ) - e.g. db.connect.distinct( 'x' ) 
     449        db.connect.drop() drop the collection 
     450        db.connect.dropIndex(index) - e.g. db.connect.dropIndex( "indexName" ) or db.connect.dropIndex( { "indexKey" : 1 } ) 
     451        db.connect.dropIndexes() 
     452        db.connect.ensureIndex(keypattern[,options]) - options is an object with these possible fields: name, unique, dropDups 
     453        db.connect.reIndex() 
     454        db.connect.find([query],[fields]) - query is an optional query filter. fields is optional set of fields to return. 
     455                                                  e.g. db.connect.find( {x:77} , {name:1, x:1} ) 
     456        db.connect.find(...).count() 
     457        db.connect.find(...).limit(n) 
     458        db.connect.find(...).skip(n) 
     459        db.connect.find(...).sort(...) 
     460        db.connect.findOne([query]) 
     461        db.connect.findAndModify( { update : ... , remove : bool [, query: {}, sort: {}, 'new': false] } ) 
     462        db.connect.getDB() get DB object associated with collection 
     463        db.connect.getPlanCache() get query plan cache associated with collection 
     464        db.connect.getIndexes() 
     465        db.connect.group( { key : ..., initial: ..., reduce : ...[, cond: ...] } ) 
     466        db.connect.insert(obj) 
     467        db.connect.mapReduce( mapFunction , reduceFunction , <optional params> ) 
     468        db.connect.aggregate( [pipeline], <optional params> ) - performs an aggregation on a collection; returns a cursor 
     469        db.connect.remove(query) 
     470        db.connect.renameCollection( newName , <dropTarget> ) renames the collection. 
     471        db.connect.runCommand( name , <options> ) runs a db command with the given name where the first param is the collection name 
     472        db.connect.save(obj) 
     473        db.connect.stats() 
     474        db.connect.storageSize() - includes free space allocated to this collection 
     475        db.connect.totalIndexSize() - size in bytes of all the indexes 
     476        db.connect.totalSize() - storage allocated for all data and indexes 
     477        db.connect.update(query, object[, upsert_bool, multi_bool]) - instead of two flags, you can pass an object with fields: upsert, multi 
     478        db.connect.validate( <full> ) - SLOW 
     479        db.connect.getShardVersion() - only for use with sharding 
     480        db.connect.getShardDistribution() - prints statistics about data distribution in the cluster 
     481        db.connect.getSplitKeysForChunks( <maxChunkSize> ) - calculates split points over all chunks and returns splitter function 
     482        db.connect.getWriteConcern() - returns the write concern used for any operations on this collection, inherited from server/db if set 
     483        db.connect.setWriteConcern( <write concern doc> ) - sets the write concern for writes to the collection 
     484        db.connect.unsetWriteConcern( <write concern doc> ) - unsets the write concern for writes to the collection 
     485    > mydb.version() 
     486    4.0.13 <<<<<<<<<-------------------<<<< MONGODB SERVER VERSION 
     487 
     488(Check Mongo server version: https://stackoverflow.com/questions/38160412/how-to-find-the-exact-version-of-installed-mongodb) 
     489 
     490Finally we now know the mongodb server version: 4.0.13 
     491This version didn't work with our mongo client (shell) version of 2.6.10. And that's we had to upgrade the client. 
     492 
     493 
     494 
     495 
     496MONGO DB ROBO 3T 
     4971. Download "Double Pack" from https://robomongo.org/ 
     4982. Untar its contents. Then untar the tarball in that. 
     4993. Run: 
     500    wharariki:[110]~/Downloads/robo3t-1.3.1-linux-x86_64-7419c406>./bin/robo3t 
     501