Problem

Users, including ourselves, need a way to write private documents, and to collaborate in a close circle. Currently there's no way in Seed to share a document with a limited group of people.

User Stories

Publish Content

User Tasks and User Stories

Read Content

Activity: Reading Content User Tasks and User Stories:

Solution

To limit the scope, the initial implementation of the feature described in this document is very basic. Most limitations described below are temporary, just to simplify the implementation, and start learning in practice ASAP.

No new permissions

For now, only WRITER members of the site and its owner will be able to access private documents. No dedicated read-only permissions yet, and no segmentation on a per-document level — all writers will have access to all private documents.

This simplifies the business logic for enforcing permissions, and doesn't require creating any new roles and capabilities.

Paths and directories

Private documents will have random path names, and no nesting (i.e. no directories).

This gets rid of a whole class of problems like metadata leakage, cascading inheritance of permissions, privacy setting conflicts, and so on.

Hierarchies of private documents could be built with explicit links, and we'll see how it goes. Another goal with this limitation is to see whether we need an explicit concept of a directory, because to this day I'm still questioning whether our approach for implementing directories implicitly based on paths is a good idea. One day I'll write a separate document about it.

Coincidentally, and depending on the outcomes of a rabbit hole described later in this document, there probably will be no ability to change document privacy after creation. You could still create different documents pointing to the same changes, but they'll be treated as separate documents — same as it works today.

Networking

To sync private documents the site owner needs to designate a server. Eventually this could be any PeerID, not only a web server.

This lets us not worry about many things incidental to P2P syncing, like leaking metadata about private documents to peers that have no business knowing about them, and exposing relationships between Account IDs and Peer IDs to random peers.

At least for now, in order to create private documents the site has to be configured with a web domain, using the existing webUrl attribute. In order to sync private documents, and publish changes to them, peers would have to talk to that server. This calls for a change in the way we currently push data to servers, which is a long due problem for us, as our current push flow is a bunch of workarounds.

Another change we need to make is in BitSwap. We need to make sure we don't accidentally send private data via BitSwap just because someone had asked the CID of that data. So we need to somehow extend BitSwap in such a way that it knows whether the requested data is private, and whether the requester has access to it, and if not — pretend it doesn't know anything about it.

Peer <-> Account relationships

Currently we don't publicly expose information about the relationships between peers and accounts.

This was mostly an accident, but it let us implement multi-account in the app much easier. Then we realized that it could be a privacy-preserving benefit, and now we don't want to let it go.

Nevertheless, in order to enforce permissions for private documents, we need to authenticate peers. But we'll do it in a very limited fashion — we'd only expose this relationship to the servers, and only to those that are relevant for a given private document.

Permanent data changes

We decided to store the public/private status of a document at the level of the Ref blobs.

This makes it possible for the same underlying changes to appear in both private and public documents. That, in turn, enables useful workflows — for example, keeping a private “work-in-progress” document alongside a public “published” version.

The simplest implementation would be a private=true flag in the Ref. However, there are a few subtle implications to consider, and they lead into a bit of a rabbit hole (or a few).

Rabbit Holes

Signaling and metadata

When resolving a URL for a document, comment, or any other resource, it would be useful to know in advance whether the resource is private.

By default, the discovery process asks all connected peers if they know anything about the URL. However, if the resource is private, most of those peers shouldn’t even be aware that it exists, or that I’m interested in it. Ideally, we'd query only the relevant peers, which for now means the server.

But how can we determine, before resolving, whether a URL points to a private resource, so we could only ask the relevant peers? This problem is often called "signaling".

A similar question appears elsewhere — how do we know whether an ipfs://<cid> link points to an image or a video? In our case, we currently signal this information in the parent context. If a block type is image — we know the linked resource must be an image, if it’s video — it must be a video. But if you look at this link without the parent context, you can't reliably tell what it is. Granted, in case of media files at least, most file formats carry their own signaling inside the file itself, usually in the form of magic numbers.

So, signaling can live in the parent (container) context, or inside the content itself. In the context of our discussion, "content" is the URL of a private resource, and the question is whether they need to carry any signaling about their privacy situation or not.

Signaling options

One way to signal that a document is private could be with a special path prefix, such as /private.

Initially, I had very mixed feelings about this approach. However, in my conversation with ChatGPT about the overall problem it brought up the idea about this prefix being configurable by the site owner, and that made me like this idea more, even if we don't implement the configuration anytime soon.

Another way could be pattern matching.

If we say that private document paths are random, have fixed length, and don't have any slashes in them, then we could simply try to match the pattern of the URL to see if it could be private or not.

This will probably work fine, but it doesn't seem very elegant.

Coincidentally, two other rabbit hole arise: readability of private document URLs, and comment URLs.

Human-friendly private URLs

If we say private document URLs are random — we lose the human-friendly URLs. We already lost them with comments, and to be honest the experience hasn't been very pleasant (especially because we don't have meaningful OG images for comments). So, if users are going to exchange links to their private documents anywhere outside Seed, it may become hard for them to distinguish different documents.

This may or may not be a big deal. After all, many-many websites people use these days have this problem. We don't have human-readable URLs in Google Docs, YouTube, X, Instagram, Telegram, Bluesky, and so on. But we do have a human readable component in Notion, Linear, and other more modern tools.

So, I've been thinking that maybe private documents could have Notion-like URL paths, such as /this-is-the-slug-of-the-document-$<special-random-id>. The $ or any other sign could be used to extract the special random ID part, and to distinguish this type of URLs from user-defined paths that we currently use (we'd prohibit this sign in user-defined paths).

Comment URLs

Even if we decide that private document URLs should signal about their privacy, this wouldn't apply to comments. And it's pretty sad. Obviously, private documents also need comments, and those comments also need to stay private.

Currently comment URLs look like this: hm://<commenter-account-id>/<comment-tsid>. It seems like making comment URLs not having any relationship to their parent documents was a bit of a mistake.

How can we tell by looking at this URLs whether this comment is private or not? How can we even know, that we need to go to a server of the document to fetch that comment? The URL is not even tied to the namespace of the document, but rather to the namespace of the commenter ¯\\_(ツ)_/¯.

Sometimes I'm guessing whether we need different kinds of comments — the ones that are part of the document itself — for collaboration and shaping the document, and the external ones — for third-party feedback about the resulting document.

Or maybe we just need to support more than one way to address comments?

Or maybe we could add "signaling" about the parent document to the comment itself, e.g. with a query parameter?

I'm honestly not sure what to do about it at this point. Help wanted.

To Do

Despite being a simplistic version, there's still a lot of work to do, and and both need to be involved.

Here's the currently known list of things to do:

[x] Update libp2p and IPFS related dependencies. We're pretty outdated on this front, and there's been some big improvements that might be relevant for this project.
Implement a proper push algorithm with optional authentication (gateway would allow pushes from anyone probably). List of things to push here
[x] Extend BitSwap to make it aware of private blobs and authenticated peers.
- Turns out BitSwap has exactly what we need: https://pkg.go.dev/github.com/ipfs/boxo/bitswap#WithPeerBlockRequestFilter.
Address the rabbit holes described above, and make the necessary permanent data changes.
TBD...
Designs

$\"\"$

Private Documents Designs

Latest Designs - 1st Iteration simplified Document Settings Membership Permissions Joining Experience For Users

","enableWebSigning","https://dailyplanet.io","origin","homeMetadata",{"_644":4497,"_1141":4498,"_4207":4498,"_1142":4499,"_1144":4493,"_4444":50,"_4446":4498,"_4500":4501,"_1697":8,"_1147":1148,"_4448":4502},"Daily Planet","ipfs://bafkreibwi4h3nzuc7invuqbq55pdsa4n6gyh2stz6a6a5vi2qylp62w2na","ipfs://bafybeifzrxuorbf6gmsw65r3ro3t5jko3vvhbuxeiljflvvysmg447di6m","seedExperimentalHomeOrder","CreatedFirst",{"_4450":4486},"originHomeId",{"_662":4505,"_39":4506,"_636":4507,"_627":-5,"_664":-5,"_665":-5,"_666":-5,"_667":-5},"z6Mkp8UGrBu3RMJZkoNKTgBiRp7LTA2bZ2EUvqFokKDof8FW","hm://z6Mkp8UGrBu3RMJZkoNKTgBiRp7LTA2bZ2EUvqFokKDof8FW",[],"meta",{"_4510":4511,"_4805":4806},"values",{"_4512":4513,"_4515":4516,"_4517":4518,"_4519":4520,"_4522":4523,"_4524":4525,"_4526":4527,"_4529":4530,"_4531":4532,"_4533":4534,"_4535":4536,"_4537":4538,"_4539":4540,"_4541":4542,"_4543":4544,"_4545":4546,"_4547":4548,"_4549":4550,"_4551":4552,"_4553":4554,"_4555":4556,"_4557":4558,"_4559":4560,"_4561":4562,"_4563":4564,"_4565":4566,"_4567":4568,"_4569":4570,"_4571":4572,"_4573":4574,"_4575":4576,"_4577":4578,"_4579":4580,"_4581":4582,"_4583":4584,"_4585":4586,"_4587":4588,"_4589":4590,"_4591":4592,"_4593":4594,"_4595":4596,"_4597":4598,"_4599":4600,"_4601":4602,"_4603":4604,"_4605":4606,"_4607":4608,"_4609":4610,"_4611":4612,"_4613":4614,"_4615":4616,"_4617":4618,"_4619":4620,"_4621":4622,"_4623":4624,"_4625":4626,"_4627":4628,"_4629":4630,"_4631":4632,"_4633":4634,"_4635":4636,"_4637":4638,"_4639":4640,"_4641":4642,"_4643":4644,"_4645":4646,"_4647":4648,"_4649":4650,"_4651":4652,"_4653":4654,"_4655":4656,"_4657":4658,"_4659":4660,"_4661":4662,"_4663":4664,"_4665":4666,"_4667":4668,"_4669":4670,"_4671":4672,"_4673":4674,"_4675":4676,"_4677":4678,"_4679":4680,"_4681":4682,"_4683":4684,"_4685":4686,"_4687":4688,"_4689":4690,"_4691":4692,"_4693":4694,"_4695":4696,"_4697":4698,"_4699":4700,"_4701":4702,"_4703":4704,"_4705":4706,"_4707":4708,"_4709":4710,"_4711":4712,"_4713":4714,"_4715":4716,"_4717":4718,"_4719":4720,"_4721":4722,"_4723":4724,"_4725":4726,"_4727":4728,"_4729":4730,"_4731":4732,"_4733":4734,"_4735":4736,"_4737":4738,"_4739":4740,"_4741":4742,"_4743":4744,"_4745":4746,"_4747":4748,"_4749":4750,"_4751":4752,"_4753":4754,"_4755":4756,"_4757":4758,"_4759":4760,"_4761":4762,"_4763":4764,"_4765":4766,"_4767":4768,"_4769":4770,"_4771":4772,"_4773":4774,"_4775":4776,"_4777":4778,"_4779":4780,"_4781":4782,"_4783":4784,"_4785":4786,"_4787":4788,"_4789":4790,"_4791":4792,"_4793":4794,"_4795":4796,"_4797":4798,"_4799":4800,"_4801":4802,"_4803":4804},"document.generationInfo.generation",[4514],"bigint","dehydratedState.queries.0.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.1.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.1.queryKey.2",[4521],"undefined","dehydratedState.queries.2.state.data.0.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.0.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.0.sortTime",[4528],"Date","dehydratedState.queries.2.state.data.0.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.0.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.0.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.0.redirectInfo",[4521],"dehydratedState.queries.2.state.data.1.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.1.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.1.sortTime",[4528],"dehydratedState.queries.2.state.data.1.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.1.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.1.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.1.redirectInfo",[4521],"dehydratedState.queries.2.state.data.2.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.2.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.2.sortTime",[4528],"dehydratedState.queries.2.state.data.2.activitySummary.latestCommentTime",[4521],"dehydratedState.queries.2.state.data.2.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.2.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.2.redirectInfo",[4521],"dehydratedState.queries.2.state.data.3.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.3.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.3.sortTime",[4528],"dehydratedState.queries.2.state.data.3.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.3.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.3.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.3.redirectInfo",[4521],"dehydratedState.queries.2.state.data.4.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.4.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.4.sortTime",[4528],"dehydratedState.queries.2.state.data.4.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.4.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.4.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.4.redirectInfo",[4521],"dehydratedState.queries.2.state.data.5.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.5.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.5.sortTime",[4528],"dehydratedState.queries.2.state.data.5.activitySummary.latestCommentTime",[4521],"dehydratedState.queries.2.state.data.5.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.5.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.5.redirectInfo",[4521],"dehydratedState.queries.2.state.data.6.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.6.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.6.sortTime",[4528],"dehydratedState.queries.2.state.data.6.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.6.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.6.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.6.redirectInfo",[4521],"dehydratedState.queries.2.state.data.7.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.7.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.7.sortTime",[4528],"dehydratedState.queries.2.state.data.7.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.7.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.7.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.7.redirectInfo",[4521],"dehydratedState.queries.2.state.data.8.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.8.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.8.sortTime",[4528],"dehydratedState.queries.2.state.data.8.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.8.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.8.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.8.redirectInfo",[4521],"dehydratedState.queries.2.state.data.9.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.9.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.9.sortTime",[4528],"dehydratedState.queries.2.state.data.9.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.9.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.9.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.9.redirectInfo",[4521],"dehydratedState.queries.2.state.data.10.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.10.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.10.sortTime",[4528],"dehydratedState.queries.2.state.data.10.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.10.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.10.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.10.redirectInfo",[4521],"dehydratedState.queries.2.state.data.11.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.11.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.11.sortTime",[4528],"dehydratedState.queries.2.state.data.11.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.11.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.11.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.11.redirectInfo",[4521],"dehydratedState.queries.2.state.data.12.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.12.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.12.sortTime",[4528],"dehydratedState.queries.2.state.data.12.activitySummary.latestCommentTime",[4521],"dehydratedState.queries.2.state.data.12.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.12.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.12.redirectInfo",[4521],"dehydratedState.queries.2.state.data.13.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.13.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.13.sortTime",[4528],"dehydratedState.queries.2.state.data.13.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.13.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.13.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.13.redirectInfo",[4521],"dehydratedState.queries.2.state.data.14.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.14.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.14.sortTime",[4528],"dehydratedState.queries.2.state.data.14.activitySummary.latestCommentTime",[4521],"dehydratedState.queries.2.state.data.14.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.14.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.14.redirectInfo",[4521],"dehydratedState.queries.2.state.data.15.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.15.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.15.sortTime",[4528],"dehydratedState.queries.2.state.data.15.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.15.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.15.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.15.redirectInfo",[4521],"dehydratedState.queries.2.state.data.16.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.16.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.16.sortTime",[4528],"dehydratedState.queries.2.state.data.16.activitySummary.latestCommentTime",[4521],"dehydratedState.queries.2.state.data.16.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.16.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.16.redirectInfo",[4521],"dehydratedState.queries.2.state.data.17.createTime.seconds",[4514],"dehydratedState.queries.2.state.data.17.updateTime.seconds",[4514],"dehydratedState.queries.2.state.data.17.sortTime",[4528],"dehydratedState.queries.2.state.data.17.activitySummary.latestCommentTime.seconds",[4514],"dehydratedState.queries.2.state.data.17.activitySummary.latestChangeTime.seconds",[4514],"dehydratedState.queries.2.state.data.17.generationInfo.generation",[4514],"dehydratedState.queries.2.state.data.17.redirectInfo",[4521],"dehydratedState.queries.5.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.5.queryKey.2",[4521],"dehydratedState.queries.6.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.6.queryKey.2",[4521],"dehydratedState.queries.7.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.8.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.9.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.9.queryKey.2",[4521],"dehydratedState.queries.10.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.10.queryKey.2",[4521],"dehydratedState.queries.11.state.data.document.generationInfo.generation",[4514],"dehydratedState.queries.12.state.data.metadata.summary",[4521],"dehydratedState.queries.13.state.data.metadata.summary",[4521],"dehydratedState.queries.14.state.data.metadata.summary",[4521],"dehydratedState.queries.15.state.data.metadata.summary",[4521],"referentialEqualities",{"_4512":4807,"_4517":4808,"_4524":4809,"_4539":4810,"_4553":4811,"_4561":4812,"_4567":4813,"_4581":4814,"_4595":4815,"_4609":4816,"_4623":4817,"_4637":4818,"_4651":4819,"_4665":4820,"_4679":4821,"_4691":4822,"_4707":4823,"_4721":4824,"_4735":4825,"_4749":4826,"_4763":4827},[4515],[4533,4687,4701,4715,4787,4791,4795],[4531],[4545],[4559],[4775],[4573],[4587],[4601],[4615],[4629],[4643],[4657],[4671],[4685],[4693,4699],[4713],[4727],[4741],[4755],[4769],"actionData","errors"]