Data Storage

It turns out there's a handy feature in PostgreSQL called row constructor comparisons that allows me to compare tuples of columns. That's exactly what we need. Instead of doing CreateAt > ?1 OR (CreateAt = ?1 AND Id > ?2) , we can do ( CreateAt, Id) > (?1, ?2) . And the row constructor comparisons are lexicographical, meaning that it's... See more

For low throughput data, Grab uses Parquet with Copy on Write (CoW) .

Here's the main operations for Copy on Write:

• Write Operations - Whenever there's a write, you create a new version of the file that includes the latest change. You can also keep the previous version for consistency and rollback purposes. This helps prevent data corruption,

... See more

Overview

pg_lakehouse is an extension that transforms Postgres into an analytical query engine over object stores like S3 and table formats like Delta Lake. Queries are pushed down to Apache DataFusion, which delivers excellent analytical performance. Combinations of the following object stores, table formats, and file formats are supported.

Object... See more

PGlite - Postgres in WASM

PGlite is a WASM Postgres build packaged into a TypeScript client library that enables you to run Postgres in the browser, Node.js and Bun, with no need to install any other dependencies. It is only 3.7mb gzipped.

import { PGlite } from "@electric-sql/pglite"

const db = new PGlite()

await db.query("select 'Hello world' as... See more

memary: Open-Source Longterm Memory for Autonomous Agents

memary demo

Why use memary?

Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve... See more

SQL Studio

Single binary, single command SQL database explorer. SQL studio supports SQLite , libSQL , PostgreSQL , MySQL and DuckDB .

Local SQLite DB File

sql-studio sqlite [sqlite_db]

Remote libSQL Server

sql-studio libsql [url] [auth_token]

PostgreSQL Server

sql-studio postgres [url]

MySQL/MariaDB Server

sql-studio mysql [url]

Local DuckDB File

sq... See more

Expose Delta Tables via REST APIs

Git repo to test 3 architectures to expose delta tables via REST APIs. See also my blogpost here. Architectures can be described as follows:

• Architecture A: Direct, Web App with DuckDB. In this architecture, APIs are directly connecting to the delta table and there is no layer in between. This implies that all data

... See more

SQL has limitations as it is built on relational concepts and relies on binary joins.

The future of databases is shifting towards relational knowledge graphs, allowing the flexibility to work with various data structures beyond tables.

Businesses are moving towards explicitly modeling business semantics and logic, which are often stored in... See more

Who will this data model serve? These are the stakeholders and users of the data model.

Why does this data model need to be built? What is the purpose and objective of the data model?

What are the data model’s core entities, attributes, and relationships? This is where you learn to see the business through the lens of data.

When is the timeframe for... See more