Data Storage
by Nicolay Gerold · updated 8d ago
Data Storage
- PachaDB
Pacha is an embeddable, immutable, graph, edge database.
Pacha is an edge database . This means you can run it on edge providers like Cloduflare, and it'll scale all over the world without you doing much.
Pacha is a graph database . This means that the data in Pacha is stored as tiny little facts that form a huge graph of knowledge. Like this... See morefrom GitHub - leostera/pachadb: an edge database
Nicolay Gerold added 8d ago
- Search
more
/
Sub-second search & analytics
engine on cloud storage
with
lessfrom Search more with less | Quickwit
Nicolay Gerold added 17d ago
- Scalability is crucial - systems need to be designed with the assumption that query volume, document corpus size, indexing complexity etc. could increase by 10x. What works at one scale may completely break at a higher scale.
- Sharding the index, either by document or by word, is important to distribute the indexing and querying load across machines.
from Claude
Nicolay Gerold added 17d ago
- SQLite Studio
Single binary, single command SQLite database explorer.
sqlite-studio <sqlite_db>
Features- Overview page with common metadata.
- Tables page with each table's metadata, including the disk size being used by each table.
- Infinite scroll rows view.
- A custom query page that gives you more access to your db.
More features available ... See morefrom GitHub - frectonz/sqlite-studio: SQLite database explorer
Nicolay Gerold added 20d ago
- Unlike some other popular algorithms, DiskANN is designed to keep memory usage to a minimum. This makes it a great match for use cases where Turso already excels at.
#Multitenancy
Turso allows for an easy implementation of a database-per-tenant pattern, where databases can be cheaply created on-demand. Keeping memory consumption at bay is critical f... See morefrom Turso brings Native Vector Search to SQLite
Nicolay Gerold added 1mo ago
- Classwords are suffixes added to database column names to indicate the type of data they contain. This improves readability and makes it easier to understand the database schema. Base classwords include text, calendar, numeric and domain-specific types. It is best to avoid redundancy in column names, as this can lead to unnecessary verbosity. Using... See more
from Gemini - chat to supercharge your ideas
Nicolay Gerold added 1mo ago
Text Classwords
identifier (or id)
code[_<standard>]
name
description (or desc)
indicator (or ind)
number
text
Calendar Classwords
date
datetime[<timezone>] (or dt[<timezone>])
timestamp[<timezone>] (or ts[<timezone>])
Numeric Classwords
count
amount[_<currency>]
<quantity_property>[_<unit_of_measure>]
ratio
factor
percent (or pct)
Domain-Specific Classwords
uri
address
email
sku
json
geojson
- For low throughput data, Grab uses Parquet with Copy on Write (CoW) .
Here's the main operations for Copy on Write:- Write Operations - Whenever there's a write, you create a new version of the file that includes the latest change. You can also keep the previous version for consistency and rollback purposes. This helps prevent data corruption, incon
from The Architecture of Grab's Data Lake
Nicolay Gerold added 1mo ago
- For High Throughput data, Grab uses Apache Avro with a strategy called Merge on Read (MOR) .
Here's the main operations with Merge on Read:- Write Operations - When data is written, it's appended to the end of a log file. This is much more efficient than merging it in the current data and reduces the latency of writes.
- Read Operations - When you need
from The Architecture of Grab's Data Lake
Nicolay Gerold added 1mo ago
- 7 must-know strategies to scale your database
Indexing:
Check the query patterns of your application and create the right indexes.
Materialized Views:
Pre-compute complex query results and store them for faster access.
Denormalization:
Reduce complex joins to improve query performance.
Vertical Scaling
Boost your database server by adding more CPU, RAM, or... See morefrom Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]
Nicolay Gerold added 1mo ago