LoFiRe

Local-First Repositories for Collaborative Decentralized Applications

by P2Pcollab

Table of Contents

About

LoFiRe is a decentralized, local-first application platform built on local-first data repositories and a local-first network architecture.

LoFiRe consists of the following components:

Local-First Repositories
Data repositories offer end-to-end encrypted data storage with public key authentication, access control, and change validation based on Conflict-free Replicated Data Types (CRDTs) and immutable Directed Acyclic Graphs (DAG) with cryptographic authentication of commits.
Local-First Network
The two-tier, asynchronous peer-to-peer (P2P) network consists of a core and edge networks, and offers end-to-end encrypted object storage, as well as asynchronous, end-to-end encrypted publish-subscribe (pub/sub) change notifications with location privacy for end-user devices.
Local-First Applications
Applications use local-first repositories and local-first networking to enable private & secure collaboration for groups and individual users, with multi-device synchronization of application data.

LoFiRe has the following properties:

Data ownership and portability
Users own their data and have a local copy.
Self-Sovereign Identities
Users control their identities, and have the choice to use their already existing public key identities or create new identities for each repository.
End-to-end encryption
Data in the repository is stored end-to-end encrypted.
Privacy
Minimize the amount of user data and metadata exposed to intermediaries.
Permissions & Access control
Fine-grained permissions for write access to the repository.
Tamperproof
Once a transaction is acknowledged by a quorum, it cannot be removed.
Asynchronicity
Allow collaboration between users, even if they are not online at the same time or work offline.
Controlled data locality
Each repository is replicated within a private network composed only of community member’s devices and their authorized replicas.
Multiple devices per user
Data is available and synchronized on multiple user devices.

Applications

LoFiRe offers end-to-end encrypted storage and asynchronous change notifications local-first decentralized applications that use Conflict-free Replicated Data Types (CRDTs) as their data model. It is designed to support collaboration within communities and organizations, and supports asynchronous workflows with synchronization over the internet, as well as various local network transports that do not require internet connectivity.

Communities and organizations (including Decentralized Autonomous Organizations, DAOs) can use it to support secure and authenticated interaction of their members through wikis, knowledge bases, structured discussions and decision making tools.

Here’s a selection of applications LoFiRe can be used for that we intend to develop once we’re ready with the underlying infrastructure:

Introduction

LoFiRe is a decentralized, collaborative data repository with authentication, access control, and change validation. It is built on local-first data storage, synchronization, and change notification protocols that aim to protect privacy by minimizing metadata exposed to intermediaries. It enables local-first, asynchronous collaboration and data storage within communities while respecting privacy and maintaining data ownership, and provides foundations for developing local-first decentralized applications and community overlay network protocols.

Community members use local-first software to collaborate around a partition-tolerant, permissioned, tamper-proof data repository that contains Directed Acyclic Graphs (DAG) of causally related transactions with operations on Conflict-free Replicated Data Types (CRDTs) that are replicated with Byzantine Fault Tolerance & Strong Eventual Consistency guarantees. In other words, it is a permissioned, DAG-structured distributed ledger, or blockchain, with partially ordered CRDT transactions. CRDTs require only a partial order on transactions, there’s no need to determine a total order using a consensus protocol, which makes the protocol efficient and light-weight.

The DAG encodes a partial order of transactions through causality relations, and together with a reliable, causal publish-subscribe (pub/sub) protocol for change notifications and a DAG synchronization protocol, it provides strong eventual consistency of replicas, with persistence of transactions through a lightweight, quorum-based acknowledgement mechanism.

Each repository is synchronized within a private community overlay network that offers immutable block storage, data synchronization, and asynchronous publish-subscribe change notification services.

The two-tier network architecture consists of a stable core network with an overlay network for each repository with a low-latency P2P pub/sub protocol, and ephemeral edge networks that use Delay/Disruption Tolerant Networking protocols for synchronization. On edge networks, edge nodes can synchronize locally and directly between each other, and can also connect to designated core nodes that store and forwards encrypted objects and change notifications for them, with such a core node acting as a pub/sub broker and object store for edge nodes.

The system is composed of the following components:

Repository
Data structures, encryption, permissions, authentication and access control.
Network
Data synchronization, publish-subscribe change notification.
Applications
CRDT state machine & change validation.

Design overview

Conflict-free replicated data types enable asynchronous, conflict-free collaboration on shared data repositories, and make eventual consistency possible among a set of replicas.

Authenticated CRDT operations disseminated over pub/sub to subscribers form a tamper-proof log, which is stored in mergeable data repositories and replicated to subscribers with access control on the allowed operations.

This enables decentralized collaboration on data repositories without relying on a centralized server for coordination, and allows resource-constrained mobile and IoT devices on edge networks to participate in the network.

Authorization and access control

Authorization is based on public-key cryptography, where the repository owner can grant access rights to members based on their public key. Each operation is signed and encrypted by its author and disseminated to all replicas subscribed to the repository. Before a replica can merge an operation, it needs to verify that its causal dependencies are merged already and that the author is allowed to perform the operation according to the CRDT access control rules defined by the repository owners.

Immutable objects

Next to mutable objects, data repositories also store immutable objects using a content-addressed object store that stores encrypted chunked objects in the repository. These objects are referenced from the mutable store.

Protocol design & specifications

Repositories

Users

Projects using LoFiRe:

Contact

Funding

This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial support from the European Commission’s Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 825322.

See also