Data Storage

SQLite Studio

Single binary, single command SQLite database explorer.

sqlite-studio <sqlite_db>

Features

• Overview page with common metadata.


• Tables page with each table's metadata, including the disk size being used by each table.


• Infinite scroll rows view.


• A custom query page that gives you more access to your db.

More features available on the r... See more

Getting Started

Keyv is a simple key-value storage system that supports multiple backends. It's designed to be a simple and consistent way to work with key-value stores.

To learn how to use Keyv, check out the keyv README. To learn how to use a specific storage adapter, check out the README for that adapter under Storage Adapters.

PachaDB

Pacha is an embeddable, immutable, graph, edge database.

Pacha is an edge database . This means you can run it on edge providers like Cloduflare, and it'll scale all over the world without you doing much.

Pacha is a graph database . This means that the data in Pacha is stored as tiny little facts that form a huge graph of knowledge. Like... See more

memary: Open-Source Longterm Memory for Autonomous Agents

memary demo

Why use memary?

Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve... See more

At the current pace of media content creation, Reddit expects their media metadata to be roughly 50 terabytes. This means they need to implement sharding and partition their tables across multiple Postgres instances.

Reddit shards their tables based on post_id where they use range-based partitioning. All posts with a post_id in a certain range will... See more

This README is under construction as we work to build a new community driven high performance key-value store.

This project was forked from the open source Redis project right before the transition to their new source available licenses.

This README is just a fast quick start document. We are currently working on a more permanent documentation page.

W... See more

1. Scalability is crucial - systems need to be designed with the assumption that query volume, document corpus size, indexing complexity etc. could increase by 10x. What works at one scale may completely break at a higher scale.


2. Sharding the index, either by document or by word, is important to distribute the indexing and querying load across machines.

... See more

pgmock

Demo — Discord

pgmock is an in-memory PostgreSQL mock server for unit and E2E tests. It requires no external dependencies and runs entirely within WebAssembly on both Node.js and the browser.

Installation

npm install pgmock

If you'd like to run pgmock in a browser, see the Browser support section for detailed instructions.

Denormalization

Another way Reddit minimizes joins is by using denormalization.

They took all the metadata fields required for displaying an image post and put them together into a single JSONB field. Instead of fetching different fields and combining them, they can just fetch that single JSONB field.

This made it much more efficient to fetch all the... See more