You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
Inex Code f3cb4ce73c Revert to fediverse.space domain 1 month ago
.github/workflows Add yarn lint 3 months ago
.reuse fix DEP5 file to follow spec 2 years ago
.vscode add debug config 1 year ago
LICENSES move to REUSE copyright spec 2 years ago
backend Revert to fediverse.space domain 1 month ago
frontend Revert to fediverse.space domain 1 month ago
gephi only put instances with a type on the graph 2 years ago
.dokku-monorepo fix backend app name in .dokku-monorepo 2 years ago
.gitignore frontend/cytoscape feature parity 2 years ago
.gitlab-ci.yml use elixir 1.10 1 year ago
BILL-OF-MATERIALS.md prepare for 2.0.0 release 2 years ago
CHANGELOG.md fix #104 1 year ago
LICENSE Add LICENSE 2 years ago
README.md Update readme 3 months ago
example.env refactor/elixir backend 2 years ago
netlify.toml allow data: images in CSP 1 year ago
nlnet-logo.png add nlnet logo to about page 2 years ago
screenshot.png add deployment instructions to README 2 years ago

README.md

index.community 🌐

The map of the fediverse that you always wanted.

Read the latest updates on Mastodon: @indexCommunity

A screenshot of fediverse.space

Requirements

Note: examples here use podman. In most cases you should be able to replace podman with docker.

Though containerized, backend development is easiest if you have the following installed.

  • For the crawler + API:
    • Elixir
    • Postgres
  • For laying out the graph:
    • Java
  • For the frontend:
    • Node.js
    • Yarn

Running it

Backend

  • cp example.env .env and modify environment variables as required
  • podman build gephi && podman build phoenix
  • podman run --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:6.8.9
    • If you've run this container previously, use podman start elasticsearch
  • podman run --name postgres -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -p 5432:5432 postgres:12
  • podman-compose -f compose.backend-services.yml -f compose.phoenix.yml
  • Create the elasticsearch index:
    • iex -S mix app.start
    • Elasticsearch.Index.hot_swap(Backend.Elasticsearch.Cluster, :instances)

Frontend

  • cd frontend && yarn install
  • yarn start

Commands

Backend

./gradlew shadowJar compiles the graph layout program. java -Xmx1g -jar build/libs/graphBuilder.jar runs it.

Frontend

  • yarn build creates an optimized build for deployment

Privacy

This project doesn't crawl personal instances: the goal is to understand communities, not individuals. The threshold for what makes an instance "personal" is defined in the backend config and the graph builder SQL.

Deployment

You don't have to follow these instructions, but it's one way to set up a continuous deployment pipeline. The following are for the backend; the frontend is just a static HTML/JS site that can be deployed anywhere.

  1. Install Dokku on your web server.
  2. Install dokku-postgres, dokku-monorepo, dokku-elasticsearch, and dokku-letsencrypt.
  3. Create the apps
  • dokku apps:create phoenix
  • dokku apps:create gephi
  1. Create the backing database
  • dokku postgres:create fediversedb
  • dokku postgres:link fediversedb phoenix
  • dokku postgres:link fediversedb gephi
  1. Set up ElasticSearch
  • dokku elasticsearch:create fediverse
  • dokku elasticsearch:link fediverse phoenix
  1. Update the backend configuration. In particular, change the user_agent in config.exs to something descriptive.
  2. Push the apps, e.g. git push dokku@<DOMAIN>:phoenix (note that the first push cannot be from the CD pipeline).
  3. Set up SSL for the Phoenix app
  • dokku letsencrypt phoenix
  • dokku letsencrypt:cron-job --add
  1. Set up a cron job for the graph layout (use the dokku user). E.g.
SHELL=/bin/bash
0 2 * * * /usr/bin/dokku run gephi java -Xmx1g -jar build/libs/graphBuilder.jar

Before the app starts running, make sure that the Elasticsearch index exists -- otherwise it'll create one called instances, which should be the name of the alias. Then it won't be able to hot swap if you reindex in the future.

Acknowledgements

NLnet logo

Many thanks to NLnet for their support and guidance of this project.