@benchristel/supercrawler
A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
- Latest release
- Jun 18, 2017
- Releases
- 1
- Known CVEs
- 0
- First release
- Jun 18, 2017
- License
- Apache-2.0
Repository
Source
- Stars
- —
- Forks
- —
- Open issues
- —
Security score
No OpenSSF Scorecard available for this repository.
Packages from this repo
No other tracked packages from this repository.
Insights
Activity
- Total releases
- 1
- Last 12 months
- 0
- Cadence
- —
- Dependencies
- 10
Releases per month
last 12 monthsRelease mix
1
releases
Dependencies
Depends on
0.16.0-
bluebird
^3.4.1
-
cheerio
^0.19.0
-
ioredis
^2.3.0
-
mime-types
^2.1.11
-
node-cache
^3.2.1
-
request
^2.73.0
-
robots-parser
github:brendonboshell/robots-parser#master
-
sequelize
^3.23.4
-
sqlite3
^3.1.4
-
underscore
^1.8.3
Used by
Nothing tracked depends on this yet.
Releases