Google Trends Data Mining Using R

Name: Google Trends Data Mining Using R
Rating: 3.9 (41 reviews)

Building an automated solution to download Google Trends data for a large number of terms.

Created bySanjeev Baniwal Ph.D.

Last updated 2/2017

English

English [Auto],

What you'll learn

Set up your own process in R to autonomously download and process large amounts of Google Trends data.
Automate various tasks used in a day to day life of a data scientist through R.
Understand each line of code used in the course and how it fits in the bigger picture of a live project.
Two independent approaches used in the course to access Trends data will reinforce the concept of building R solutions from scratch. In one approach, we will exclusively use Base R methods, and in other, learn how to work with R packages.
You will be able to run various windows commands from R e.g. copy, rename, delete files on hard disc etc
Use Window's Task Scheduler to have desktop run R in batch mode on recurring basis.
Your R toolbox will become much more diverse and applicable to a wide range of future projects.

Course content

4 sections • 14 lectures • 1h 44m total length

Introduction6:31
Introduction to Google Trends as a tool for deriving interesting insights about the world around us.
Understand the layout of Trends data.
Bottlenecks associate with Google Trends website.

Import Export of txt and csv files8:02
Import and Export of text files using read.table and read.csv functions.
Export R data files using write.csv and write.table functions.
Address various commonly faced issues.
Data Frames in R Part 15:54
Create data frames.
Create calculated variables.
Change variable names in an existing data frame.
Data Frames in R Part 29:07
Learn to manipulate data frames in R by subsetting with the subset function and logical conditions (blue cars, price under 10000), and by bracket methods to select columns and rows.
Performing system functions using R12:44
Use systems command to:

Create Directories.
Copy multiple files.
Rename files.
Use wild cards (characters) in these functions to allows selection of files with a particular characteristic.

Reinforce the concepts for lapply and do.call functions to read in multiple files and stack them using rbind function.
Understand lapply and docall functions in R7:10
Create list objects and sub set them.
Distinguish between lapply and do.call functions.
Combine lapply and do.call functions to achieve desired outcomes.
Review Windows Batch processes8:03
Conceptualize the manual steps and how to automate them using R.
Capture the string that is submitted to Google Trends servers from Chrome's 'Inspect Element' Tools menu.
Set up a batch script to open Chrome and download a particular response from a server.
Write a Windows Batch process from R4:52
Use R to write a .bat (batch) file.
Execute batch files from R.
Combinding the power of Batch files with R "For loops"9:50
Explore for loops in R, from simple iterations to complex syntax, and learn to combine them with batch files to generate and submit multiple Google Trends queries, downloading data automatically.
Use of an Anchor search term to enable comparative analysis of Trends data5:14
Understand the concept of Anchor term and its role as 'equalizer' of data across search terms.
learn to identify an Anchor term from the collection of search terms.
Modify an existing batch file to include two search item simultaneously.
Create a Pivot chart in Excel to visualize the data.

Automating a Google Trends data download.8:14
Put together all the functions described above in the context of a bigger project.
Set up a file as a repository for search terms used in a project.
Use for loop to iterate over all the search terms to create and execute batch files specifically created for each search term.
Scheduling Trends Data Download using R script, .bat, and Task Scheduler6:50
Set up your first automated task using Task Scheduler, Batch file, and a R script.
Precautions needed when downloading data for a large number of terms on a slow internet or machine.
Override a scheduled task in Windows Scheduler.

Requirements

No prerequisites, previous exposure to R and windows work environment will be helpful..
You will need access to R, Windows, Google account, Chrome.

Description

Google Trends is a new tool in a Data Scientist or a Marketing Analyst's tool box. It provides data that could be quite useful when assessing trend in general public interest in a particular product or an array of products. Trends data essentially gives us a way to assess how often a particular search was performed on world's most popular search engine.

However, Google Trends only allows a simultaneous search of up to five terms, also it does not provide data through a dedicated API. This course describes two processes in R which could be employed to circumvent both of these issues. First process stitches together R and Windows batch system. Second, utilize gtrendsR package. In both case we set up an automated process.

The course begins by describing Google Trends data itself, and then various R functions relevant to build the automated solution. In the third section, we apply the knowledge from previous sections to build a complete end to end solution.

By the end of the course, we will better understand how R could be employed in more automated manner with minimum human involvement. Additionally, better appreciate the power of Google Trends data to provide a greater understanding of the way the world around us functions.

Who this course is for:

Those wanting to retrieve and explore large amounts of Google Trends data.
Students with beginner level of R understanding and those wanting to join course which is not typical beginner's curriculum. We will stay focused on using R to automate a software process.
We will NOT deal with any statistics here!

Google Trends Data Mining Using R

What you'll learn

Explore related topics

Course content

Introduction to Google Trends web Search engine1 lecture • 7min

Understanding R functions one at a time9 lectures • 1hr 11min

Bringing it all together: Setting up the automated R process.2 lectures • 15min

Section 4: Using gTrendsR package2 lectures • 12min

Requirements

Description

Who this course is for: