apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.
Language:TypeScript
Total stars: 9760
Stars trend:
#typescript
#apify, #automation, #crawler, #crawling, #headless, #headlesschrome, #javascript, #nodejs, #npm, #playwright, #puppeteer, #scraper, #scraping, #typescript, #webcrawler, #webcrawling, #webscraping
Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.
Language:TypeScript
Total stars: 9760
Stars trend:
22 Nov 2023
12am █▍ +11
1am ▊ +6
2am █▏ +9
3am ▍ +3
4am ▎ +2
5am ▍ +3
6am ▎ +2
7am ▋ +5
8am ████▋ +37
9am ████▉ +39
10am ███▏ +25
11am ██ +16#typescript
#apify, #automation, #crawler, #crawling, #headless, #headlesschrome, #javascript, #nodejs, #npm, #playwright, #puppeteer, #scraper, #scraping, #typescript, #webcrawler, #webcrawling, #webscraping
mendableai/firecrawl
🔥 Turn entire websites into LLM-ready markdown
Language:TypeScript
Total stars: 106
Stars trend:
#typescript
#ai, #crawler, #data, #htmltomarkdown, #llm, #markdown, #rag, #scraper, #scraping, #webcrawler
🔥 Turn entire websites into LLM-ready markdown
Language:TypeScript
Total stars: 106
Stars trend:
16 Apr 2024
5pm ▋ +5
6pm ██▍ +19
7pm ██▎ +18
8pm █▊ +14
9pm █▎ +10
10pm █ +8
11pm ▎ +2
17 Apr 2024
12am ██▎ +18#typescript
#ai, #crawler, #data, #htmltomarkdown, #llm, #markdown, #rag, #scraper, #scraping, #webcrawler
adithya-s-k/omniparse
Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks
Language:Python
Total stars: 109
Stars trend:
#python
#ingestionapi, #ocr, #omniparser, #parseserver, #parserlibrary, #visiontransformer, #webcrawler, #whisperapi
Ingest, parse, and optimize any data format ➡️ from documents to multimedia ➡️ for enhanced compatibility with GenAI frameworks
Language:Python
Total stars: 109
Stars trend:
28 Jun 2024
12am █▊ +14
1am █▏ +9
2am ▉ +7
3am ▊ +6
4am ▊ +6
5am █▍ +11
6am █▌ +12
7am ▋ +5
8am ▋ +5
9am ▌ +4
10am ▍ +3
11am ▊ +6#python
#ingestionapi, #ocr, #omniparser, #parseserver, #parserlibrary, #visiontransformer, #webcrawler, #whisperapi
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Language:Python
Total stars: 208
Stars trend:
#python
#apify, #automation, #beautifulsoup, #crawler, #crawling, #headless, #headlesschrome, #pip, #playwright, #python, #scraper, #scraping, #webcrawler, #webcrawling, #webscraping
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Language:Python
Total stars: 208
Stars trend:
9 Jul 2024
7am ██▏ +17
8am █▍ +11
9am █▎ +10
10am ███▏ +25
11am ██▊ +22
12pm ███ +24
1pm ███▍ +27#python
#apify, #automation, #beautifulsoup, #crawler, #crawling, #headless, #headlesschrome, #pip, #playwright, #python, #scraper, #scraping, #webcrawler, #webcrawling, #webscraping
jasonxtn/Argus
The Ultimate Information Gathering Toolkit
Language:Python
Total stars: 748
Stars trend:
#python
#cmsdetection, #directoryfinder, #dnslookup, #emailharvester, #informationgathering, #osint, #pastebinmonitoring, #recontools, #reconnaissance, #serverinfo, #sslanalitcs, #txtrecords, #virustotal, #webcrawler, #whoislookup
The Ultimate Information Gathering Toolkit
Language:Python
Total stars: 748
Stars trend:
24 Oct 2024
9am ▍ +3
10am ▍ +3
11am ▎ +2
12pm ▎ +2
1pm ▏ +1
2pm █ +8
3pm ▎ +2
4pm ▉ +7
5pm ▊ +6
6pm █ +8
7pm ███▍ +27
8pm ██▌ +20#python
#cmsdetection, #directoryfinder, #dnslookup, #emailharvester, #informationgathering, #osint, #pastebinmonitoring, #recontools, #reconnaissance, #serverinfo, #sslanalitcs, #txtrecords, #virustotal, #webcrawler, #whoislookup
mendableai/firecrawl
🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.
Language:TypeScript
Total stars: 21906
Stars trend:
#typescript
#ai, #aiscraping, #crawler, #data, #htmltomarkdown, #llm, #markdown, #rag, #scraper, #scraping, #webcrawler, #webscraping
🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.
Language:TypeScript
Total stars: 21906
Stars trend:
20 Jan 2025
5pm ▏ +1
6pm ▎ +2
7pm ▉ +7
8pm ▊ +6
9pm ▉ +7
10pm █ +8
11pm ▋ +5
21 Jan 2025
12am ▉ +7
1am ▌ +4
2am ▍ +3
3am ██ +16
4am █▍ +11#typescript
#ai, #aiscraping, #crawler, #data, #htmltomarkdown, #llm, #markdown, #rag, #scraper, #scraping, #webcrawler, #webscraping