VBA Selenium Web Scraping Part 1/2

Name: VBA Selenium Web Scraping Part 1/2
Rating: 4.5 (32 reviews)

WebScraping with VBA Selenium via Google Chrome

Created byajay parmar

Last updated 2/2023

English

What you'll learn

Welcome in Part1 of this VBA Selenium course. Please note what we are covering in this part1
How to use Selenium library inside Excel VBA Editor to scrape websites which supports major browsers like chrome, mozilla, firefox , edge etc.
We are using chrome browser in this series to scrape website elements but you will know how to use other browsers as well.
How to download selenium library and what is the procedure to download browser webdrivers
How to navigate websites using chrome browsers
Learn how to use attributes ID & name or classes to enter a value in search box or text boxes.
Which attribute should you use and why. How to use find function on website for manual search and why it is important to be familiar with this method.
How to click the buttons using attributes like id, name , classname
How to work with tags like A tags and their attribute href.
Also, learn how to use html in notepad and create webpages. It will help you in visualizing the things.
Know how tables are designed on websites and how we can export every table in one go. Know table tags
Know table rows tags and how to work with every or one specific rows and export whatever you need.
Know table headers and table data tags like th and td. Learn how to locate specific td tag
What are collection loops used to scrape tables OR how to scrape one specific table only.
What if user wants to see data for few clients only out of 1000s rows. How to control these tables from excel where user enters the names.
What is the fundamental difference between web element and its collection elements.
Why and where we need SET Keyword and where to avoid it. Proper explanation given.
Using send keys and enter if search button is unable to search data for you.
How to use Header or Rows count property in code to handle them together. Its an amazing approach.
How to use Xpath to enter keywords in input boxes and how we can click on search buttons using xpath
How to read Xpath DOM structure, A proper explanation is given.
How to use Xpath to scrape tables.
How to work with tags like headers - H1,H2,H3 on individual levels and using Xpath method and its benefits.
How to understand VBA Errors if they pop up on screen instead going on google.
How to work with listbox with single property or multiple property.
How to work with drop down using index, text and values property.
Your questions and projects also discussed. Go inside one page drop down and then inside second one.

Course content

1 section • 29 lectures • 5h 57m total length

How to Download Selenium and chrome driver exe files16:55
First and foremost thing. we need to download selenium library and chrome driver. Please follow the instructions and do it. If you will find any problems please let me know though it is a simple process. File is available in next lecture. Pls download from there.
Let us Launch the website and understand the concepts.11:33
After downloading the necessary packages , now we like to launch a website on chrome browser. After few lecures I will educate you about other browsers too like edge or firefox.
Work with ID attribute22:51
Work with Name Attribute10:09
Know the name attribute in html and how we can use it in selenium.
Work with Atags hyperlinks31:01
How to work with hyperlinks on web pages. This is done by web developers using A tag with href as their attribute. Let us see how we can click on any link using collection loops and by knowing its indexes.
Work with Tables - HTML of Tables Behind coding - Part116:34
Let us understand the Table structure first in HTML
Work with Tables - Export every table data into excel -Part213:37
How to loop through each and every table on a web page and export the table data in excel.
Work with Tables - Target the specific countries list from excel sheet - Part330:38
We are taking the inputs from user for country names and same is getting scraped from website. This is super example not only from website scraping perspective but also you will find the smart and advance way of using the loops.
How to target specific td tags in a table - Superb Control - Part49:08
What if you do not want to see every detail in a row of a table. How to hit the right td tag ignoring others.
This is not just a conceptual thing but its a very practical situation and you can be ready for it in advance.
Target specific table among many and then target specific data -Part59:44
How to scrape one particular table when you have 100s tables on a web page. Quite possible. Isnt it?
So, let us see how we can reach to the table we want to scrape and not others and not only this inside that table we can further target specific rows and data.
Work with Table specific rows -Part65:31
How to scrape a table in a reverse order - Part710:35
This one came from your side . Can we scrape the page from bottom to Top.
Learn how to handle Headers and Data in one code -Count property -Part88:31
How to start with Xpath8:52
Please use this file for all rest Xpath chapters.
Xpath is one of my favorite, It can scrape anything and very useful and easy to learn.
Let us see how to obtain xpath from any element and let us understand the path as to how it is written.
Use Xpath to export Tables -Part24:50
Xpath can also help us in exporting the website tables. See this in this lecture
Xpath to refer to specific object by editing -Must know Part310:28
We now know very well that how xpath syntax is formed so why dont we go ahead and edit it to get different results.
Idea is you should know that you can edit the xpath and get different outputs.
How to handle Headers tags along and with Xpath - H1,H2,H3 tags10:52
How to scrape h1,h2,h3 etc tags .
How to scrape them in one go by using Xpath. Its awesome.
Let us deal with dropdown now12:52
We often see websites which bullet points or serial numbers and paragraphs are written. In this lecture we are learning to export the comments or those paragraphs using li tags. Li means list tag.
How to work with unordered list tags and ordered list tags.
Please note: Excel file attached here will be used for next list related lectures too. So, please follow the same till part4
Work with Partial Link text property- Click just by finding the text on website5:35
How to avoid running A TAG loops and directly click on any link by knowing its word only
Partial Link method - How to click website links even if you know few words.9:31
We can click on web links even by finding the partial word instead of exact word. This is amazing knowledge to learn.
Project for you- Load a website with today date page showing information6:17
How to select item in a drop down -Select Webelement Class7:20
This one very simple. How to select an item from a combobox or drop down using text, index and value methods.
Student Project is done here- More confident now .26:19
Work with ListBox - Single select property - Part122:43
How to select listbox values using select element class and why it cannot be webelement class.
Little introduction on methods and property of a class.
Work with listbox which has multi-select property on - Part29:43
Work with Listbox -How to export list box selected or all values -Part313:50
Use Xpath with Variables too - An idea can help in projects5:59
How to make Calendar pop ups disappear - Mouse Click3:46
My other Courses with Links1:56
If you like to learn Basic to super Advance excel , Excel VBA, Bi or PowerPivot or PowerQuery or M code or MS Access and its VBA ,these links will help you a lot. My every course will teach you from very basics and it will have tons of practical examples.

Requirements

You should have good understanding of VBA for next Loops and good understanding of IF functions.

Description

In this Part1 you shall learn below written things. Please go through them.

We are going to learn Webscraping using google chrome and it will be done using selenium library.
Learn how to download selenium and browser drivers to first initially do the setup.
How work with attributes like name, id using findelement methods.
How to work with tags like A tag and its attribute href to click on the links.
How to use class attribute and is it a good approach to use class attribute.
how to work with Xpath to insert the data into textboxes and click on search buttons. Not just use but understand the structure of Xpath as well.
Learn how we can change the xpath elements by their indexes.
How to work with header tags like h1 or h2 or h3 using normal way and using xpath.
How to run the collection loops and why we need it and when we can avoid using collection loops.
What is difference between Web-element and Webelements and why we use set word for collection elements and not for single object.
How to click on links on webpage using link -text and partial link text property. Based on this knowledge one mini project -how to load website by default on current date.
How to scrape tables and its rows and rows data values and table headers.
Collection loops design and execution from very start while working with tables.
Do you like to export all tables in a website or do you want to scrape one specific table only, How to reach out to specific index of a table then.
How to scrape specific rows inside one specific table . Know your "tr' tags.
How to scrape all data insider rows or just one specific data value using td tag. Its awesome when you have so much control.
How to export tables with out running collection loops and what is the difference between direct export and export using loops. You must know.
How to work with li tags inside unordered and ordered lists.
Know how to insert a value in drop down or combo-boxes .
How to select items in list box with single select property or multi select property.
Learn to select or de-select listbox options using collection loops or without the loops.
How to alter the collection elements by referring to one object and second inside the first one and so on.

Who this course is for:

If you are Business analyst and want to learn how to scrape websites using chrome or other major browsers.
If you like to export website tables into excel. If you like to click on website links or fill up a form or download the files etc.