Python Web Scraping Tutorial. Use BeautifulSoup & Requests to scrape & crawl Craigslist directory with Python
Automated Web Scraping, When the webpage structure is so complicated, making it difficult to extract specific pieces of data. Or when you need to open so many pages to extract data from each of them, the manual process can become boring and time-wasting, and that is when automated Web Scraping can make the process more efficient and effective.
What you’ll learn
- Installing Python Web Scraping libraries, BeautifulSoup & Requests.
- Extracting URLs from a webpage.
- Scraping text data from a webpage.
- Crawling multiple webpages and scraping data from each of them.
- Handling navigation links and move to next pages.
- Saving scraped data into a CSV file with Pandas.
- Overview about other popular web scraping frameworks.
Course main points
Web Scraping with Python BeautifulSoup & Request
- Web Scraping Course Overview.
- Installing BeautifulSoup & Requests. Learn how to install BeautifulSoup and Requests libraries.
- URL Extraction. How to extract the job URLs from Craigslist.
- Web Scraping Craigslist – Titles. How to scrape job titles.
- Web Scraping Craigslist – Job Details Wrapper. how to deal with HTML wrappers to be able to extract job details from each job section. will learn how to follow the job URL, open the job description page and scrape it.
- Web Scraping Craigslist – Job Description Page. follow the job URL, open the job description page and scrape it.
- Crawling & Scraping Next Pages. Crawl next pages and scrape all the job lists
- Saving Output to CSV File. How to save your scraped output to a CSV file using Pandas.
Other Web Scraping Python Frameworks Overview
- Scrapy vs. Other Python Web Scraping Libraries. Learn about other web scraping frameworks such as Scrapy and Selenium and how they are compared to BeautifulSoup and Requests.
Who this course is for
- Python developers and students who are new to Web Scraping
Requirements
- Basic knowledge of Python programming
Rating:
4.3 ( 1,602 ratings) 36,767 students enrolled
| Author | GoTrained Academy |
| Categories | eLearning Professionals |
| Instructor Rating | 4.2 |
| Reviews | 5,135 |
| Students | 58,969 |
| Courses | 9 |
