Data Storage
SQLite Studio
Single binary, single command SQLite database explorer.
sqlite-studio <sqlite_db>
Features
More features available on the r... See more
Single binary, single command SQLite database explorer.
sqlite-studio <sqlite_db>
Features
- Overview page with common metadata.
- Tables page with each table's metadata, including the disk size being used by each table.
- Infinite scroll rows view.
- A custom query page that gives you more access to your db.
More features available on the r... See more
GitHub - frectonz/sqlite-studio: SQLite database explorer
Getting Started
Keyv is a simple key-value storage system that supports multiple backends. It's designed to be a simple and consistent way to work with key-value stores.
To learn how to use Keyv, check out the keyv README. To learn how to use a specific storage adapter, check out the README for that adapter under Storage Adapters.
Keyv is a simple key-value storage system that supports multiple backends. It's designed to be a simple and consistent way to work with key-value stores.
To learn how to use Keyv, check out the keyv README. To learn how to use a specific storage adapter, check out the README for that adapter under Storage Adapters.
jaredwray • GitHub - jaredwray/keyv: Simple key-value storage with support for multiple backends
PachaDB
Pacha is an embeddable, immutable, graph, edge database.
Pacha is an edge database . This means you can run it on edge providers like Cloduflare, and it'll scale all over the world without you doing much.
Pacha is a graph database . This means that the data in Pacha is stored as tiny little facts that form a huge graph of knowledge. Like... See more
Pacha is an embeddable, immutable, graph, edge database.
Pacha is an edge database . This means you can run it on edge providers like Cloduflare, and it'll scale all over the world without you doing much.
Pacha is a graph database . This means that the data in Pacha is stored as tiny little facts that form a huge graph of knowledge. Like... See more
GitHub - leostera/pachadb: an edge database
memary: Open-Source Longterm Memory for Autonomous Agents
memary demo
Why use memary?
Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve... See more
memary demo
Why use memary?
Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve... See more
GitHub - kingjulio8238/memary: Longterm Memory for Autonomous Agents.
Data
At the current pace of media content creation, Reddit expects their media metadata to be roughly 50 terabytes. This means they need to implement sharding and partition their tables across multiple Postgres instances.
Reddit shards their tables based on post_id where they use range-based partitioning. All posts with a post_id in a certain range will... See more
Reddit shards their tables based on post_id where they use range-based partitioning. All posts with a post_id in a certain range will... See more
Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]
This README is under construction as we work to build a new community driven high performance key-value store.
This project was forked from the open source Redis project right before the transition to their new source available licenses.
This README is just a fast quick start document. We are currently working on a more permanent documentation page.
W... See more
This project was forked from the open source Redis project right before the transition to their new source available licenses.
This README is just a fast quick start document. We are currently working on a more permanent documentation page.
W... See more
GitHub - valkey-io/valkey: A new project to resume development on the formerly open-source Redis project. We're calling it Valkey, like a Valkyrie.
- Scalability is crucial - systems need to be designed with the assumption that query volume, document corpus size, indexing complexity etc. could increase by 10x. What works at one scale may completely break at a higher scale.
- Sharding the index, either by document or by word, is important to distribute the indexing and querying load across machines.
Claude
pgmock
Demo — Discord
pgmock is an in-memory PostgreSQL mock server for unit and E2E tests. It requires no external dependencies and runs entirely within WebAssembly on both Node.js and the browser.
Installation
npm install pgmock
If you'd like to run pgmock in a browser, see the Browser support section for detailed instructions.
Demo — Discord
pgmock is an in-memory PostgreSQL mock server for unit and E2E tests. It requires no external dependencies and runs entirely within WebAssembly on both Node.js and the browser.
Installation
npm install pgmock
If you'd like to run pgmock in a browser, see the Browser support section for detailed instructions.
stackframe-projects • GitHub - stackframe-projects/pgmock: In-memory Postgres for unit/E2E tests
Denormalization
Another way Reddit minimizes joins is by using denormalization.
They took all the metadata fields required for displaying an image post and put them together into a single JSONB field. Instead of fetching different fields and combining them, they can just fetch that single JSONB field.
This made it much more efficient to fetch all the... See more
Another way Reddit minimizes joins is by using denormalization.
They took all the metadata fields required for displaying an image post and put them together into a single JSONB field. Instead of fetching different fields and combining them, they can just fetch that single JSONB field.
This made it much more efficient to fetch all the... See more