Interview, Fireside Chat
Towards a Decentralized Storage Network for the Blockchain Computer
- Problem Statement: Decentralized storage aims to distribute massive amounts of unused "latent" storage (e.g., phones, laptops) currently held by centralized providers like Amazon or Google into a peer-to-peer network.
- Core Challenge: Verifying that random, permissionless nodes are actually storing files correctly and retrievably without relying on institutional reputation or trust in specific human actors.
- Value Proposition: The shift to decentralized storage is driven by three primary arguments:
- Economic Efficiency: Unlocking underutilized storage resources, with an estimated average of 50% hard drive capacity remaining unused on personal devices.
- Censorship Resistance: Creating a fault-tolerant, amorphous network where no single government or entity can easily remove files or prevent access compared to centralized servers.
- Foundation for Decentralized Apps: Providing a necessary, non-centralized storage layer for the state of applications like Bitcoin and Ethereum, preventing the blockchain's accounting or computation layers from having a centralized weak link.
- Trust Mechanism Shift: The network replaces "trust in institutions" (e.g., Amazon's reputation) with "trust in security systems" (cryptographic protocols and incentive structures).
- Bruce Schneier's framework identifies this as moving from moral/institutional trust to security-based trust, allowing for scalable, global participation where users do not need to know the identity of storage providers.
- Cryptographic Building Blocks:
- Proof of Retrievalability (PoR): A communication-efficient proof allowing a client to verify a node holds a specific file by checking a small tag rather than downloading the whole file.
- Proof of Replication (PoR): A tighter cryptographic construct that proves a node is dedicating specific disk space to storing data (useful files) rather than just "spam" data like the digits of pi.
- The "Fake Identity" Problem: PoR prevents a malicious actor from claiming to store multiple independent copies of a file on a single drive by using one copy.
- Tightness Constraint: The protocol is designed so that it is economically and computationally impossible to use less than ~99% of the claimed storage to store the file; any attempt to cheat requires re-deriving encoded data that would take too long to complete within challenge time limits.
- Mechanism Design & Incentives:
- Work Transformation: The system adapts the Bitcoin model (Proof of Work) by replacing "useless" computational puzzles with "useful" storage work, incentivizing nodes to store valuable data rather than just empty cycles.
- Asymmetric Encoding: Files are encoded to require significant computational effort and time to reconstruct if any piece is deleted (acting as a deterrent to cheating), while maintaining the ability to decode and retrieve the original file quickly.
- Security & Commoditization Debate:
- Risk: Using commoditized storage (hard drives) for security could allow large entities (like Amazon) to overwhelm the network with existing infrastructure.
- Mitigation via Inconvenience: Proof of Replication creates a barrier to entry for massive centralized providers by requiring a specific, inconvenient encoding format that disrupts standard business operations.
- Small-scale participants (e.g., individuals with spare phone storage) can absorb the overhead, whereas large entities face disproportionate costs regarding latency and storage efficiency.
- ASIC Resistance: While not strictly ASIC-proof, the high "tightness" of the proof makes specialized hardware less advantageous for the massive scale of centralized providers compared to the negligible gains for individual participants.
- Future Directions:
- Accumulators: Ben Fish's recent work on accumulators aims to reduce the storage burden on consensus nodes.
- Consensus/State Separation: The goal is to decouple the "consensus layer" (verifying transactions) from the "storage layer" (holding data).
- This allows lightweight devices (e.g., phones) to participate in governance/consensus while heavy servers handle the actual data storage, optimizing for both security and capacity.