inex
/
index.community
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

bump personal instance threshold to 10

This commit is contained in:
Tao Bror Bojlén 2019-02-28 18:06:16 +00:00
parent dfc115f275
commit e52f581067
No known key found for this signature in database
GPG Key ID: C6EC7AAB905F9E6F
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
backend/scraper/management/commands/scrape.py
View File

@ -24,7 +24,7 @@ from scraper.management.commands._util import require_lock, InvalidResponseExcep
SEED = 'mastodon.social'
TIMEOUT = 20 # seconds
NUM_THREADS = 16 # roughly 40MB each
PERSONAL_INSTANCE_THRESHOLD = 5 # instances with < this many users won't be crawled
PERSONAL_INSTANCE_THRESHOLD = 10 # instances with < this many users won't be crawled
MAX_STATUSES_PER_PAGE = 100
STATUS_SCRAPE_LIMIT = 5000
INSTANCE_SCRAPE_LIMIT = 50 # note: this does not include newly discovered instances! they will always be crawled.