This is an assignment in Coursera Big Data course by UCSD.

Part 1 is about MongoDB and Par 2 is about simple analysis with Spark tools.

This hands on is based on Cloudera Quickstart VM.

Start MongoDB server

Start MongoDB client and list existing databases

Choose database “sample” and then list collections

So we took a look at how may records in collection “users” and showed fields and values of the first record.

Next, question 1, how many tweets have location not null?

We can use following commands to figure it out.

Also another way

Question 2, How many people have more followers than friends?

Question 3, Return text of tweets which have the string “http://” ?

Question 4, Return all the tweets which contain text “England” but not “UEFA” ?

Question 5, Get all the tweets from the location “Ireland” and contains the string “UEFA” And then find who has most of the friends.

So we’ll create text index and do search with multiple conditions on it.

Last, let’s export collection users to csv file and use it in part 2 tasks.

A field list has to be specified to export to csv file.