Writing a minimal ActivityPub server from scratch

Lately, in the background

Twitter purchases by Elon Musk

people began to look for alternatives to it – and many found such an alternative in

Mastodon

.

Mastodon is a decentralized social network that works on a federation model, like email. The federation protocol is called ActivityPub and is a W3C standard, and Mastodon is far from the only implementation of it, but the most popular. Different protocol implementations are generally compatible with each other as much as their overlap in functionality allows. I also have my own ActivityPub server project – Smithereensuch a green decentralized VK, where I will someday return the wall.

In this article, we will look at the basics of the ActivityPub protocol and write a minimal possible server implementation that allows you to post to the network (“fediverse”), subscribe to other users and receive updates from them.


What is ActivityPub anyway?

In general, according to specifications,ActivityPub comes in two flavors: server-to-server and ,client-server. The client-server variety is strange and not particularly usable on the imperfect Internet, and in general no one really implements it, so we won’t consider it.

ActivityPub in relation to the federation consists of the following main parts:

  • Actors – objects (or subjects?) that can perform some actions. For example, users or groups. They are uniquely identified globally by the address (URL) where the actor’s JSON object is located.

  • Activities — objects representing these same actions, like “Vasya published a post.”

  • Inbox — endpoint on the server to which these activities are sent. His address is indicated in the field inbox at the actor.

All objects are just JSON with a specific schema. In fact, this is a slightly cursed JSON-LD with namespaces, but for our needs we can forget about them. ActivityPub objects have a mime type application/ld+json; profile="https://www.w3.org/ns/activitystreams" or application/activity+json.

It all works like this: actor sends activity to another actor in inbox, and he accepts it, checks it, and does something with it. For example, it puts a new post in its database, or creates a subscription and sends a “subscription accepted” activity in response. The activities themselves are sent via post requests signed with the actor’s key (each actor has a pair of RSA keys for authentication).

In addition to this, it is necessary to implement the protocol WebFinger to convert human-readable usernames like @vasya@example.social into real actor identifiers of the form https://example.social/users/vasya. Mastodon refuses to work with servers that do not implement this, even if you give it a direct link to the actor ¯\_(ツ)_/¯

What should a server be able to do to participate in the fediveverse?

In order for your server to be fully interoperable from Mastodon and other similar software, it must support the following:

  1. Give an actor object with a minimum set of fields: ID, inbox, public key, username, Person type.

  2. Respond to webfinger requests from endpoint /.well-known/webfinger. Mastodon will refuse to see your actor without this.

  3. Send your correctly signed activities to subscribers and anyone else to whom they may be relevant – for example, the users mentioned in the post.

  4. Accept POST requests in inbox and check their signatures. To begin with, it will be enough to support 4 types of activities: Follow, Undo{Follow}, Accept{Follow} and Create{Note}.

  5. Upon receiving the correct Follow activity, save new subscriber information somewhere on disk, send him Accept{Follow} and subsequently send him all the activities about, for example, the creation of new posts.

  6. And when receiving Undo{Follow} – remove it from this list (no need to send anything).

  7. It is highly desirable, but not strictly necessary, have URLs for already created postsfor which a Note object is returned so that they can be loaded on another server by inserting the address into the search field.

Practical part

Now let’s look at each point in detail and with code examples. My implementation will be in Java since it is my native programming language. You can either point to my example, or write it yourself in an image and likeness on your preferred stack. You will also need a domain and an HTTPS server/proxy – or some other

ngrok

or your own.

My example has two dependencies: Spark micro-framework to receive incoming requests and Gson for working with JSON. All code is available in its entirety on my github.

Before you begin, generate an RSA key pair and put them in your project folder:

openssl genrsa 2048 | openssl pkcs8 -topk8 -nocrypt -out private.pem
openssl rsa -in private.pem -outform PEM -pubout -out public.pem

Give the actor object

At any address convenient for you, send a JSON object of this type:

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/security/v1"
  ],
  "type": "Person",
  "id": "https://example.social/users/vasya",
  "preferredUsername": "vasya",
  "inbox": "https://example.social/inbox",
  "publicKey": {
    "id": "https://example.social/users/vasya#main-key",
    "owner": "https://example.social/users/vasya",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\n...\n----END PUBLIC KEY----"
  }
}

Must be submitted with a title

Content-Type: application/ld+json; profile="https://www.w3.org/ns/activitystreams"

. Purpose of fields:

  • @context – JSON-LD context. Don’t pay attention to it, just remember that it should be there. But if you are curious to know more about him, then Here And Here.

  • type – Object type. Person — a person’s personal profile. There are also Group, Organization, Application and Service.

  • id — the global identifier of the object, as well as the address at which it can be obtained (a link to itself, yeah).

  • preferredUsername — user’s username, which is displayed in the interface and is used for searches and mentions.

  • inbox — the address of the same inbox, endpoint that receives incoming activities.

  • publicKey — public RSA key with which the signature of the activity from this actor is verified:

    • id — key identifier. In theory there may be several of them, but in practice everyone has one. Just add #main-key after the actor ID.

    • owner — key owner ID, just your actor ID.

    • publicKeyPem — the key itself is in PEM format.

Additional optional fields you may want to add:

  • followers And following — addresses of collections of subscribers and subscriptions. You can return 403 or 404 from there, but some servers need these fields to simply be present in the object.

  • outbox – inbox in reverse, collection some activities sent by this user. Usually there are only Create{Note} and Announce{Note}.

  • url — link to the profile in the server web interface.

  • name — display name, for example, Vasya Pupkin.

  • icon And image — avatar and cover, respectively. Objects of type Image, with fields type, mediaType And url. Avatars are usually square.

  • summary — “about yourself” field. It usually contains HTML.

To look at an actor object (or post, or anything else) from another server, send a GET request with the header

Accept: application/ld+json; profile="https://www.w3.org/ns/activitystreams"

to the same address where you see the profile in the browser:

$ curl -H 'Accept: application/ld+json;  profile="https://www.w3.org/ns/activitystreams"' https://mastodon.social/@Gargron {"@context":["https://www.w3.org/ns/activitystreams", ...

Код для получения актора

private static final String AP_CONTENT_TYPE = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"";

/**
 * Получить объект актора с другого сервера
 * @param id идентификатор актора
 * @throws IOException в случае ошибки сети
 */
private static JsonObject fetchRemoteActor(URI id) throws IOException {
  try {
    HttpRequest req = HttpRequest.newBuilder()
        .GET()
        .uri(id)
        .header("Accept", AP_CONTENT_TYPE)
        .build();
    HttpResponse<String> resp = HTTP_CLIENT.send(req, HttpResponse.BodyHandlers.ofString());
    return JsonParser.parseString(resp.body()).getAsJsonObject();
  } catch(InterruptedException x) {
    throw new RuntimeException(x);
  }
}

Код для отдачи актора

private static final Gson GSON = new GsonBuilder().disableHtmlEscaping().create();

get("/actor", (req, res) -> {
  Map<String, Object> actorObj = Map.of(
      "@context", List.of("https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1"),
      "type", "Person",
      "id", ACTOR_ID,
      "preferredUsername", USERNAME,
      "inbox", "https://" + LOCAL_DOMAIN + "/inbox",
      "publicKey", Map.of(
          "id", ACTOR_ID + "#main-key",
          "owner", ACTOR_ID,
          "publicKeyPem", publicKey
      )
  );
  res.type(AP_CONTENT_TYPE);
  return GSON.toJson(actorObj);
});

Отвечаем на webfinger-запросы

Запрос и (урезанный до минимально необходимого) ответ выглядят вот так:

$ curl -v https://mastodon.social/.well-known/webfinger?resource=acct:gargron@mastodon.social
...
< HTTP/2 200 
< content-type: application/jrd+json; charset=utf-8
...

{
  "subject":"acct:Gargron@mastodon.social",
  "links":[
    {
      "rel":"self",
      "type":"application/activity+json",
      "href":"https://mastodon.social/users/Gargron"
    }
  ]
}

This is just a way of saying “The actor ID corresponding to the username gargron on the mastodon.social server is

https://mastodon.social/users/Gargron

“

These two endpoints are already enough to make your actor visible on other servers – try entering the actor’s object URL into the search field in Mastodon to see its profile.

Sending out activities

Activities are sent via POST requests to inboxes. HTTP signatures are used for authentication – this is a header signed by other headers using the actor key. It looks like this:

Signature: keyId="https://example.social/actor#main-key",headers="(request-target) host date digest",signature="..."

Where keyId — key identifier from the actor object, headers – the headings that we signed, and signature — the signature itself in base64. Signed headers must include Host, Date and “pseudo-title” (request-target) is a method and a way (for example, post /inbox). Time in Date should differ from the time of the receiving server by no more than 30 seconds – this is necessary to prevent replay attacks. Modern versions of Mastodon also require a header Digestthis is the SHA-256 to base64 from the request body:

Digest: SHA-256=n4hdMVUMmFN+frCs+jJiRUSB7nVuJ75uVZbr0O0THvc=

The line that needs to be signed is the names and values ​​of the headers in the same order as they are listed in the headers field. Names are written in lowercase letters and separated from values ​​by a colon and a space. After each header, except the last one, there is a line feed (\n):

(request-target): post /users/1/inbox
host: friends.grishka.me
date: Sun, 05 Nov 2023 01:23:45 GMT
digest: SHA-256=n4hdMVUMmFN+frCs+jJiRUSB7nVuJ75uVZbr0O0THvc=

Code for sending activity

/**
 * Отправить активити в чей-нибудь инбокс
 * @param activityJson JSON самой активити
 * @param inbox адрес инбокса
 * @param key приватный ключ для подписи
 * @throws IOException в случае ошибки сети
 */
private static void deliverOneActivity(String activityJson, URI inbox, PrivateKey key) throws IOException {
  try {
    byte[] body = activityJson.getBytes(StandardCharsets.UTF_8);
    String date = DateTimeFormatter.RFC_1123_DATE_TIME.format(Instant.now().atZone(ZoneId.of("GMT")));
    String digest = "SHA-256="+Base64.getEncoder().encodeToString(MessageDigest.getInstance("SHA-256").digest(body));
    String toSign = "(request-target): post " + inbox.getRawPath() + "\nhost: " + inbox.getHost() + "\ndate: " + date + "\ndigest: " + digest;

    Signature sig = Signature.getInstance("SHA256withRSA");
    sig.initSign(key);
    sig.update(toSign.getBytes(StandardCharsets.UTF_8));
    byte[] signature = sig.sign();

    HttpRequest req = HttpRequest.newBuilder()
        .POST(HttpRequest.BodyPublishers.ofByteArray(body))
        .uri(inbox)
        .header("Date", date)
        .header("Digest", digest)
        .header("Signature", "keyId=\""+ACTOR_ID+"#main-key\",headers=\"(request-target) host date digest\",signature=\""+Base64.getEncoder().encodeToString(signature)+"\",algorithm=\"rsa-sha256\"")
        .header("Content-Type", AP_CONTENT_TYPE)
        .build();
    HttpResponse<String> resp = HTTP_CLIENT.send(req, HttpResponse.BodyHandlers.ofString());
    System.out.println(resp);
  } catch(InterruptedException | NoSuchAlgorithmException | InvalidKeyException | SignatureException x) {
    throw new RuntimeException(x);
  }
}

You are all ready to send your first activity! Try leaving a comment below

my post about this article

– send this to my inbox,

https://friends.grishka.me/users/1/inbox

replacing example.social with the domain your server is running on:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://example.social/createHelloWorldPost",
  "type": "Create",
  "actor": "https://example.social/actor",
  "to": "https://www.w3.org/ns/activitystreams#Public",
  "object": {
    "id": "https://example.social/helloWorldPost",
    "type": "Note",
    "published": "2023-11-05T12:00:00Z",
    "attributedTo": "https://example.social/actor",
    "to": "https://www.w3.org/ns/activitystreams#Public",
    "inReplyTo": "https://friends.grishka.me/posts/884435",
    "content": "<p>Привет, федивёрс</p>"
  }
}

If you did everything correctly, your comment will appear under the post.

Here: Create – activity type, we created something. actor – who created object – what he created, to — to whom this activity is addressed (the whole world). And we created a “note” (that’s how posts are called in ActivityPub) with the text “Hello, Fediverse,” which is a response to my post. Post text supports a basic set of HTML formatting tags, but what is supported varies by server.

We accept the activity and check the signatures

We just sent Activities, now we need to learn how to accept them. There is no point in repeating ourselves, everything is the same. To verify the signature you need:

  • Parse the Date header. If the time differs from the current time by more than 30 seconds, reject the request (you can return code 400, for example).

  • Parse the request body. Get an actor object by URL in actor.

  • Check that keyId in the title Signature matches the actor key identifier.

  • Parse the public key, compose a signature line (see above) and verify the signature.

Code for obtaining activity with signature verification

post("/inbox", (req, res) -> {
  // Время в заголовке Date должно быть в пределах 30 секунд от текущего
  long timestamp = DateTimeFormatter.RFC_1123_DATE_TIME.parse(req.headers("Date"), Instant::from).getEpochSecond();
  if (Math.abs(timestamp - Instant.now().getEpochSecond()) > 30) {
    res.status(400);
    return "";
  }

  // Вытаскиваем актора
  JsonObject activity = JsonParser.parseString(req.body()).getAsJsonObject();
  URI actorID = new URI(activity.get("actor").getAsString());
  JsonObject actor = fetchRemoteActor(actorID);

  // Парсим заголовок и проверяем подпись
  Map<String, String> signatureHeader = Arrays.stream(req.headers("Signature").split(","))
      .map(part->part.split("=", 2))
      .collect(Collectors.toMap(keyValue->keyValue[0], keyValue->keyValue[1].replaceAll("\"", "")));
  if (!Objects.equals(actor.getAsJsonObject("publicKey").get("id").getAsString(), signatureHeader.get("keyId"))) {
    // ID ключа, которым подписан запрос, не совпадает с ключом актора
    res.status(400);
    return "";
  }
  List<String> signedHeaders = List.of(signatureHeader.get("headers").split(" "));
  if (!new HashSet<>(signedHeaders).containsAll(Set.of("(request-target)", "host", "date"))) {
    // Один или несколько обязательных для подписи заголовков не содержатся в подписи
    res.status(400);
    return "";
  }
  String toSign = signedHeaders.stream()
      .map(header -> {
        String value;
        if ("(request-target)".equals(header)) {
          value="post /inbox";
        } else {
          value=req.headers(header);
        }
        return header+": "+value;
      })
      .collect(Collectors.joining("\n"));
  PublicKey actorKey = Utils.decodePublicKey(actor.getAsJsonObject("publicKey").get("publicKeyPem").getAsString());
  Signature sig = Signature.getInstance("SHA256withRSA");
  sig.initVerify(actorKey);
  sig.update(toSign.getBytes(StandardCharsets.UTF_8));
  if (!sig.verify(Base64.getDecoder().decode(signatureHeader.get("signature")))) {
    // Подпись не проверилась
    res.status(400);
    return "";
  }

  // Всё получилось - запоминаем активити, чтобы потом показать её пользователю
  receivedActivities.addFirst(activity);

  return ""; // Достаточно просто ответа с кодом 200
});

Subscribe to people

You now have all the components needed to follow another user. Try subscribing to your Mastodon account, or, for example,

mastodon.social/@Mastodon

. Send this activity to the required actor (don’t forget to replace

example.social

to your domain and

object

to the id of the desired actor):

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "id": "https://example.social/oh-wow-i-followed-someone",
  "type": "Follow",
  "actor": "https://example.social/actor",
  "object": "https://mastodon.social/users/Mastodon"
}

You should receive an activation message shortly after this. Acceptwith your Follow inside as object (I write types of such nested activities in the format Accept{Follow}). This means that another server has accepted your subscription and will continue to send you, for example, Create, Announce And Delete about the posts that this actor will create, repost and delete. To unsubscribe, send Undo{Follow}.

What’s next?

If you have read this far and done everything according to the instructions, congratulations – you have a working ActivityPub server! You can try the obvious improvements:

  • Add the ability to subscribe on your actor

  • Don’t just put activities into an array, but process them depending on their type

  • And in the interface you can make not a large field for JSON, but normal buttons for specific actions

  • Connect a database to store users, posts and subscriptions

  • …and support more than one user on the server

  • Create a full-fledged web interface and/or API for client applications

  • Add some kind of authentication in the end

  • Cache objects from other servers locally to avoid making too many identical requests

useful links

More ActivityPub servers, good and different:

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *