# Web Scraping

{{tag>Web Scraping}}


### Different Techniques for Scraping

  - [[Requests]]
  - [[BeautifulSoup]]
  - [[Scrapy]]
  - [Scrapy Tutorial](https://docs.scrapy.org/en/latest/intro/tutorial.html)
  - [[Portia]]
  - [[Scrapely]]


### Challenges while Scraping at Scale

- https://github.com/scrapinghub/frontera
- https://github.com/rmax/scrapy-redis


### Scraping Guidelines/ Best Practices





## 출처

- https://towardsdatascience.com/web-scraping-with-scrapy-theoretical-understanding-f8639a25d9cd
- https://medium.com/velotio-perspectives/web-scraping-introduction-best-practices-caveats-9cbf4acc8d0f
- [Spiders](https://doc.scrapy.org/en/latest/topics/spiders.html)