inex/index.community
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

normalize instance names

Browse source
This commit is contained in:
Tao Bror Bojlén 2019-08-02 22:46:40 +03:00
parent 767bd4a947
commit 287c7b5624
No known key found for this signature in database
GPG key ID: C6EC7AAB905F9E6F
4 changed files with 6 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
CHANGELOG.md
View file

@ -17,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed ### Fixed
- Fixed some instances being duplicated (due to un-normalized data).
### Security ### Security
## [2.3.0 - 2019-08-02] ## [2.3.0 - 2019-08-02]

3
backend/config/config.exs
View file

@ -68,8 +68,7 @@ config :backend, :crawler,
user_agent: "fediverse.space crawler", user_agent: "fediverse.space crawler",
admin_phone: System.get_env("ADMIN_PHONE"), admin_phone: System.get_env("ADMIN_PHONE"),
twilio_phone: System.get_env("TWILIO_PHONE"), twilio_phone: System.get_env("TWILIO_PHONE"),
admin_email: System.get_env("ADMIN_EMAIL"), admin_email: System.get_env("ADMIN_EMAIL")
frontend_domain: "https://www.fediverse.space"
config :backend, Backend.Scheduler, config :backend, Backend.Scheduler,
jobs: [ jobs: [

3
backend/config/dev.exs
View file

@ -65,4 +65,5 @@ config :backend, :crawler,
blacklist: [ blacklist: [
"gab.best", "gab.best",
"4chan.icu" "4chan.icu"
] ],
frontend_domain: "localhost:3000"

1
backend/lib/backend/crawler/crawler.ex
View file

@ -148,6 +148,7 @@ defmodule Backend.Crawler do
|> Map.keys() |> Map.keys()
|> list_union(result.peers) |> list_union(result.peers)
|> Enum.filter(fn domain -> domain != nil and not is_blacklisted?(domain) end) |> Enum.filter(fn domain -> domain != nil and not is_blacklisted?(domain) end)
|> Enum.map(&clean_domain(&1))
peers = peers =
peers_domains peers_domains