Commit graph

9 commits

Author SHA1 Message Date
Tao Bror Bojlén 884810fa09
set status limit to 40 per page 2019-03-08 16:05:42 +00:00
Tao Bror Bojlén 2534b45487
remove trailing slash from endpoint crawled
this was causing issues on at least one pleroma instance, for some
reason
2019-03-08 15:59:07 +00:00
Tao Bror Bojlén 3a66e7464e
fix duplicates in instance peers 2019-03-01 14:42:05 +00:00
Tao Bror Bojlén e52f581067
bump personal instance threshold to 10 2019-02-28 18:06:16 +00:00
Tao Bror Bojlén dfc115f275
lowercase instance names (fix #48) 2019-02-28 18:05:54 +00:00
Tao Bror Bojlén 2e5696962b
only scrape 50 instances in one go 2019-02-28 18:03:08 +00:00
Tao Bror Bojlén b4ef0793e7
restore original status limit 2019-02-21 12:35:53 +00:00
Tao Bror Bojlén 1c1f193542
improve handling of cancelled scrape 2019-02-21 10:38:49 +00:00
Tao Bror Bojlén 1b11c70430
simplify folder structure 2019-02-21 10:26:55 +00:00
Renamed from backend/api/scraper/management/commands/scrape.py (Browse further)