~Get Your Files Here !

  1. Introduction

   1. Introduction.srt

   2. Scrapy installation.html

  2. Xpath first steps

   1. Xpath 101 node types.srt

   2. Xpath 102 basic syntax.mp4

   2.1 XPath 102 Cheat Sheet.pdf

   3. XPath 103 Axes (Node Relations).srt

   4. Revisiting our real-estate web scraping example.mp4

  3. Hello Scrapy

   1. What is a web bot Is it ethical.srt

   2. The Scrapy Shell.srt

   3. Creating your first Scrapy project.srt

   4. Creating your first Scrapy spider.mp4

   4.1 Create your own Scrapy spider.html

   5. Handling combined queries using the getall() method.srt

   6. Data cleansing using Item Loaders.mp4

   6.1 Item Loaders.html

   7. Pagination and link-following using Crawl Spiders.mp4

   7.1 Crawl Spiders.html

  4. Scrapy web-scraping scenarios

   1. Login to websites.srt

   2. Changing the user-agent.mp4

   3. Handling AJAX requests 1.mp4

   3.1 Handling AJAX requests.html

   4. Handling AJAX requests 2.srt

   5. Handling AJAX requests 3.mp4

   5.1 Handling AJAX requests.html

   6. Caching responses.srt

   7. Image harvesting.srt

   8. Scraped images storage in FTP and AWS S3.srt

  5. Data transformation using Scrapy Pipelines

   1. Introduction and sample project (classifieds ads scraping).srt

   2. Removing ads with duplicate titles.mp4

   2.1 Remove duplicates pipeline.html

   3. Removing ads with no phone numbers.mp4

   3.1 Dropping Ads with no phones pipeline.html

  6. Data loading (storage) using Scrapy's pipelines

   1. Storing scraped data in MongoDB.srt

   2. Storing scraped data in MySQL.mp4

   2.1 MySQL Pipeline.html

   3. Using Vault to sore sensitive Scrapy settings.srt

   4. Storing data to AWS S3 bucket.mp4

   4.1 S3 Pipeline.html

   5. Using Amazon Glue and Athena to query the data from S3 (extra lecture).srt

  7. Scrapy Middleware (or how to avoid getting banned)

   1. Phone-models project and spider rate-limiting.srt

   2. Rotating user-agents middleware.mp4

   2.1 Rotating user-agents project.html

   3. Rotating proxies middleware.srt

  8. Handling JavaScript websites using Splash

   1. What is Splash.srt

   2. Introduction to Docker (optional).srt

   3. Test-driving Splash.srt

   4. Integrating Scrapy with Splash.srt

   5. Dealing with infinitely-scrolling pages using Splash.mp4

   5.1 Handling scrolling pages with Splash.html

  9. Browser automation using Selenium and Scrapy

   1. What is Selenium.srt

   2. Revisiting infinitely-scrolling pages (medium.com).srt

   2.2 Revisiting infinitely-scrolling pages (medium.com).html

   3. Clicking buttons (Yahoo Finance).srt

  Bonus Resources.txt