A collection of awesome web crawler,spider in different languages
-
Updated
Jun 16, 2024
A collection of awesome web crawler,spider in different languages
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
A list of practical knowledge-building projects.
Web Scraper in Go, similar to BeautifulSoup
Generate and download e-books from online sources.
A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama
Faster requests on Python 3
A web crawler and scraper for Rust
🚀 Stealth - Secure, Peer-to-Peer, Private and Automateable Web Browser/Scraper/Proxy
scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
Bulk download your favourite anime episodes from your favourite anime websites
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
CLI tool for saving a faithful copy of a complete web page in a single HTML file (based on SingleFile)
A list of scrapers from around the web.
Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.
A universal web-util for PHP.
A framework for creating semi-automatic web content extractors
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Add a description, image, and links to the web-scraper topic page so that developers can more easily learn about it.
To associate your repository with the web-scraper topic, visit your repo's landing page and select "manage topics."