Data Storage
WebDataset
WebDataset is a library for writing I/O pipelines for large datasets. Its sequential I/O and sharding features make it especially useful for streaming large-scale datasets to a DataLoader.
The WebDataset format
A WebDataset file is a TAR archive containing a series of data files. All successive data files with the same prefix are consider... See more
WebDataset is a library for writing I/O pipelines for large datasets. Its sequential I/O and sharding features make it especially useful for streaming large-scale datasets to a DataLoader.
The WebDataset format
A WebDataset file is a TAR archive containing a series of data files. All successive data files with the same prefix are consider... See more
WebDataset
SQLite Studio
Single binary, single command SQLite database explorer.
sqlite-studio <sqlite_db>
Features
More features available ... See more
Single binary, single command SQLite database explorer.
sqlite-studio <sqlite_db>
Features
- Overview page with common metadata.
- Tables page with each table's metadata, including the disk size being used by each table.
- Infinite scroll rows view.
- A custom query page that gives you more access to your db.
More features available ... See more
GitHub - frectonz/sqlite-studio: SQLite database explorer
This README is under construction as we work to build a new community driven high performance key-value store.
This project was forked from the open source Redis project right before the transition to their new source available licenses.
This README is just a fast quick start document. We are currently working on a more permanent documentation page.
W... See more
This project was forked from the open source Redis project right before the transition to their new source available licenses.
This README is just a fast quick start document. We are currently working on a more permanent documentation page.
W... See more
GitHub - valkey-io/valkey: A new project to resume development on the formerly open-source Redis project. We're calling it Valkey, like a Valkyrie.
Local database for development
Each table in the database had an accompanying script that would generate a subset of the data for use in local development, since the final database was too large to run on a developer's machine.
This let each developer work with a live, local, copy of the database and enabled efficient development of changes.
I highly... See more
Each table in the database had an accompanying script that would generate a subset of the data for use in local development, since the final database was too large to run on a developer's machine.
This let each developer work with a live, local, copy of the database and enabled efficient development of changes.
I highly... See more
Bill Mill • notes.billmill.org
SQL Studio
Single binary, single command SQL database explorer. SQL studio supports SQLite , libSQL , PostgreSQL , MySQL and DuckDB .
Local SQLite DB File
sql-studio sqlite [sqlite_db]
Remote libSQL Server
sql-studio libsql [url] [auth_token]
PostgreSQL Server
sql-studio postgres [url]
MySQL/MariaDB Server
sql-studio mysql [url]
Local DuckDB File
sq... See more
Single binary, single command SQL database explorer. SQL studio supports SQLite , libSQL , PostgreSQL , MySQL and DuckDB .
Local SQLite DB File
sql-studio sqlite [sqlite_db]
Remote libSQL Server
sql-studio libsql [url] [auth_token]
PostgreSQL Server
sql-studio postgres [url]
MySQL/MariaDB Server
sql-studio mysql [url]
Local DuckDB File
sq... See more
frectonz • GitHub - frectonz/sql-studio: SQL Database Explorer [SQLite, libSQL, PostgreSQL, MySQL/MariaDB, DuckDB, ClickHouse]
Search
more
/
Sub-second search & analytics
engine on cloud storage
with
less
more
/
Sub-second search & analytics
engine on cloud storage
with
less
Search more with less | Quickwit
A serverless vector database
built from first principles on object storage: 10-100x cheaper, usage-based pricing, massive scalability
built from first principles on object storage: 10-100x cheaper, usage-based pricing, massive scalability
turbopuffer
Getting Started
Keyv is a simple key-value storage system that supports multiple backends. It's designed to be a simple and consistent way to work with key-value stores.
To learn how to use Keyv, check out the keyv README. To learn how to use a specific storage adapter, check out the README for that adapter under Storage Adapters.
Keyv is a simple key-value storage system that supports multiple backends. It's designed to be a simple and consistent way to work with key-value stores.
To learn how to use Keyv, check out the keyv README. To learn how to use a specific storage adapter, check out the README for that adapter under Storage Adapters.
jaredwray • GitHub - jaredwray/keyv: Simple key-value storage with support for multiple backends
Data bases have gotten so good at this, that the term is almost misleading now. “Base” suggests something rigid, without which the data would slip away. But the data is always there, just bits on a nameless hard disk. The structure and the accessibility that a modern database provides exist completely independently from that hard disk. That’s right... See more