Back to projects
• 4 min read

SecStore

PythonFastAPIReed-SolomonAES-256-GCMDockerTerraform

What it does

You can lose nodes to failure or corruption and the system still reconstructs your data. Files are encrypted in the browser before they leave the client — the coordinator only ever sees ciphertext. Deduplication means identical files uploaded under different names share storage. Self-healing runs in the background on every download.

Features

  • Reed-Solomon erasure coding over GF(2^8) — custom implementation with Vandermonde matrices, not a library wrapper
  • End-to-end encryption — AES-256-GCM in the browser; coordinator never sees plaintext when encryption is enabled
  • Content-hash deduplication — same file uploaded twice only stores fragments once
  • Geo-aware placement — fragments spread across us-east-1a/b/c so a zone outage doesn’t take down everything
  • Background self-healing — corruption detected on download triggers automatic repair to healthy nodes
  • Node audit on return — when a node comes back online, the system checks for missing chunks and restores them
  • Web dashboard at http://localhost:8000/ui with live cluster event stream over WebSocket
  • Python CLI for scripted usage

Quick start

# 1. Generate docker-compose.yml for N nodes (minimum 3)
make cluster nodes=6

# 2. Build and start the cluster
make build

# 3. Access the system — pick one:
#    a) Open the web dashboard at http://localhost:8000/ui
#    b) Use the CLI directly (see commands below)

# Run integration tests
make test

# Show available commands
make help

# Demo workflows:
make demo-upload          # Upload a file, no encryption
make demo-corrupt         # Corrupt a fragment on Node 3
make demo-download        # Download and verify

# With E2E encryption:
make demo-upload-enc      # Upload with AES-256-GCM (key auto-generated)
make demo-download-enc    # Download and decrypt

# Shut down and clean up
make clean

Keys for encrypted uploads are stored in .secstore_keys.json in the directory where you run the command, keyed by remote path. The browser dashboard and CLI use separate key storage.

Architecture

coordinator/          # Control plane — encoding, deduplication, healing, API
  main.py             # FastAPI app, SQLite manifest
  erasure_coding.py   # GF(2^8) Reed-Solomon encoder/decoder
  static/index.html   # Web dashboard (HTML/JS, no frameworks)

node/                 # Binary fragment storage
  main.py             # Lightweight FastAPI service per node

client.py             # Python CLI with encryption support
deploy_cluster.py     # Generates docker-compose.yml with zone-aware placement
test_suite.py         # Integration tests (4 scenarios)
Makefile              # One-command cluster operations

How encoding works

With k=2, n=3: a file is split into 2 data chunks and 1 parity chunk (D0, D1, P0 = D0 XOR D1). Any 2 of the 3 fragments reconstruct the original. For larger k/n, the encoder builds a systematic Vandermonde matrix over GF(2^8) using primitive polynomial 0x11D and precomputed log/exp tables.

The decoder skips matrix inversion when all data chunks [0..k-1] are present — it just concatenates them directly.

Storage overhead: 1.5x at k=2, n=3 (survive 1 failure). At k=4, n=7 (survive 3 failures), it’s 1.75x. Compare this to 3x for triple replication.

Limits

  • k: 1–10, n: 1–15, and n must be ≤ k*3
  • Maximum file size: 20 MB (enforced at upload)
  • No authentication — all endpoints are open (appropriate for local/dev deployments)
  • Encryption key is browser-local; cross-browser access requires manual copy/paste

Tech stack

  • Python 3.10 / FastAPI / Uvicorn
  • SQLite for manifest storage
  • httpx for async HTTP between coordinator and nodes
  • Docker Compose for container orchestration
  • Web Crypto API (browser) and cryptography library (Python CLI) for AES-256-GCM
  • Terraform for AWS infrastructure (VPC, EC2, ECR)

Tests

Four integration test scenarios run via make test:

  1. Healthy upload + download
  2. Single node corruption — system recovers using surviving fragments
  3. Double node corruption — system correctly returns HTTP 500
  4. End-to-end encryption — encrypted upload survives encode-decode-decode pipeline and decryption produces original file

License

Apache 2.0

Back to top