Supercharged Web Scraping with Asyncio and Python
What you'll learn
- Basic Web scraping with Python
- Web scraping with Selenium & Python
- Asynchronous Web scraping with Asyncio
- 30 Days of Python (or similar experience with Python)
- Basic understanding of HTML is helpful
Web scraping is simply automatically opening up any website and grabbing the data you find important on that website. It's fundamental to the internet, search engines, Data Science, automation, machine learning, and much more.
Opening websites and extracting data are only part of what makes web scraping great. It's the parsing of the data that's where the value is.
This project will cover:
Basic web scraping with Python
Web scraping with Selenium
Sync vs Async
Asynchronous Web scraping with Asyncio
But why asynchronous code? What is it? How does it benefit us?
Asynchrounous code is a way to execute multiple functions basically at once. It's not actually at the exact same time but it's close. (They actually run concurrently). This means that we can do more things in less time and, when it comes to mining or scraping data, this time saving is absolutely significant.
Imagine for a moment you're recreating google's search engine. You'd have to scrape trillions (if not more) web pages on a regular interval to help with the search results. Of course you're not going to be scraping all of the trillions of pages at once but the idea is that scraping event 1,000 pages would take a very long time doing it synchronously (like using Python requests and/or just selenium).
If you've done a lot of web scraping before but never used Python's aysncio, this course will help you better understand the fundamentals and bring your scraping game to another level.
Let's get started!
Who this course is for:
- Data Scientists & Aspiring Data Scientists
- Python Developers looking to better understand asynchronous coding within Python
- Anyone interested in Data Mining / Data Scraping
It all started with an idea. I wanted freedom... badly. Freedom from work, freedom from boredom, and, most of all, the freedom to choose. This simple idea grew to define me; it made me become an entrepreneur.
As I strived to gain freedom, overtime I realized that with everything that you do you can either (1) convince someone, somehow, to do it with you or (2) figure out how to do it yourself.
Due to a lack of financial resources (and probably the ability to convince people to do high quality work for free), I decided to learn. Then learn some more. Then some more. My path of learning website design started a long time ago. And yes, it was out of need not desire. I believed I needed a website for a company that I started. So I learned how to do it. The company died, my skills lived on... and got better and better.
It took me a while after learning web design (html/css) to actually start learning programming (web application, storing "data", user logins, etc). I tinkered with Wordpress, believing it could be a "user" site, but I was mistaken. Sure there are/were hacks for that, but they were hacks/work-arounds and simply not-what-wordpress-was-indended-to-be. Wordpress is for blogs/content. Plain and simple.
I wanted more. I had a web application idea that I thought would change the way restaurants hire their service staff. I tested it with my basic html/css skills, had great initial results, and found a technical (programmer) cofounder as a result. He was awesome. We were featured on CNN. Things looked great.
Until... cash-flow was a no-flow. Business? I think not. More like an avid hobby. We had the idea for a business just no business. Naturally, my partner had to find a means of income so I was left with the idea on its own.
Then, I tried Python. I was hooked. It was so easy. So simple. So elegant.
Then, I tried Django. Even more hooked. Made from python & made for web applications. It powers Instagram & Pinterest (two of the hottest web apps right now?).
Then, I tried Bootstrap. Simple and easy front-end design (html & css) that is super easy to use, mobile-ready, and overall... incredible.
Python, Django, and Bootstrap are truly changing the way the world builds web applications. I believe it's because of the simplicity to learn, the sheer power behind them, and, most of all, the plethora of resources to aid anyone in building their web projects (from packages to tutorials to q&a sites).
I relaunched my original venture with my new found skills. That wasn't enough. It didn't compel me as it once had. I started imagining all the possibilities of all the ideas I've always wanted to implement. Now I could. Which one to start with? There were so many good ideas...
Then another idea, a new & fresh idea, started brewing. I started to believe in the power of learning these skills. What would it mean if other non-technical entrepreneurs could learn? What would it mean if ideas were executed quickly, revenue models proven, all prior to approaching the highly sought-after programmers? What would it mean if entrepreneurs became coders?
And so. Coding for Entrepreneurs was born.
Here are some bio highlights:
Adjunct Professor of Entrepreneurship at the Lloyd Greif Center for Entrepreneurial Studies in the Marshall School of Business at the University of Southern California
Bestselling instructor on Udemy
Funded creator on Kickstarter
Founder of Coding For Entrepreneurs
Cohost of Backer Radio