Join 10350+ others. No spamming.
I promise!

Follow us at github.



attic-labs/noms

6103

attic-labs / noms

Go

The versioned, forkable, syncable database


READ ME

Use Cases  |  Setup  |  Status  |  Documentation  |  Contact

Build Status codecov GoDoc Slack

Welcome

Noms is a decentralized database philosophically descendant from the Git version control system.

Like Git, Noms is:

  • Versioned: By default, all previous versions of the database are retained. You can trivially track how the database evolved to its current state, easily and efficiently compare any two versions, or even rewind and branch from any previous version.
  • Synchronizable: Instances of a single Noms database can be disconnected from each other for any amount of time, then later reconcile their changes efficiently and correctly.

Unlike Git, Noms is a database, so it also:

  • Primarily stores structured data, not files and directories (see: the Noms type system)
  • Scales well to large amounts of data and concurrent clients
  • Supports atomic transactions (a single instance of Noms is CP, but Noms is typically run in production backed by S3, in which case it is "effectively CA")
  • Supports efficient indexes (see: Noms prolly-trees)
  • Features a flexible query model (see: GraphQL)

Finally, because Noms is content-addressed, it yields a very pleasant programming model.

Working with Noms is declarative. You don't INSERT new data, UPDATE existing data, or DELETE old data. You simply declare what the data ought to be right now. If you commit the same data twice, it will be deduplicated because of content-addressing. If you commit almost the same data, only the part that is different will be written.


Use Cases

DecentDB - A pretty good database for the decentralized web

Because Noms is very good at sync, it makes a great basis for rich, collaborative, fully-decentralized applications.

BucketDB - Horizontally scalable OLAP database in an S3 bucket

The immutable and content-addressable design of Noms makes it possible to build a full, horizontally scalable OLAP database atop cheap block storage. This separates storage costs from compute costs, so that you only pay for the compute that you use.

ClientDB (doc coming soon) - offline-first mobile applications

Embed Noms into mobile applications, making it easier to build offline-first, fully synchronizing mobile applications.


Setup

# You probably want to add this to your environment
export NOMS_VERSION_NEXT=1

go get github.com/attic-labs/noms/cmd/noms
go install github.com/attic-labs/noms/cmd/noms

Run

Import some data:

go install github.com/attic-labs/noms/samples/go/csv/import
curl 'https://data.cityofnewyork.us/api/views/kku6-nxdu/rows.csv?accessType=DOWNLOAD' > /tmp/data.csv
csv-import /tmp/data.csv /tmp/noms::nycdemo

Explore:

noms show /tmp/noms::nycdemo

Should show:

struct Commit {
  meta: struct Meta {
    date: "2017-09-19T19:33:01Z",
    inputFile: "/tmp/data.csv",
  },
  parents: set {},
  value: [  // 236 items
    struct Row {
      countAmericanIndian: "0",
      countAsianNonHispanic: "3",
      countBlackNonHispanic: "21",
      countCitizenStatusTotal: "44",
      countCitizenStatusUnknown: "0",
      countEthnicityTotal: "44",
...

Status

Data Format

We are fairly confident in the core data format, and plan to support Noms database version 7 and forward. If you create a database with Noms today, future versions will have migration tools to pull your databases forward.

Roadmap

We plan to implement the following for Noms version 8:


Learn More About Noms

For the decentralized web: The Decentralized Database

Learn the basics: Technical Overview

Tour the CLI: Command-Line Interface Tour

Tour the Go API: Go SDK Tour


Contact Us

Interested in using Noms? Awesome! We would be happy to work with you to help understand whether Noms is a fit for your problem, or even to prioritize work that you need.

Reach out at noms@attic.io or via: