Data Storage
by Nicolay Gerold · updated 2mo ago
Data Storage
- pg_vectorize: a VectorDB for Postgres
A Postgres extension that automates the transformation and orchestration of text to embeddings and provides hooks into the most popular LLMs. This allows you to do vector search and build LLM applications on existing data with as little as two function calls.
This project relies heavily on the work by pgvector f... See morefrom GitHub - tembo-io/pg_vectorize: The simplest way to orchestrate vector search on Postgres
Nicolay Gerold added 7mo ago
- Search
more
/
Sub-second search & analytics
engine on cloud storage
with
lessfrom Search more with less | Quickwit
Nicolay Gerold added 4mo ago
- memary: Open-Source Longterm Memory for Autonomous Agents
memary demo
Why use memary?
Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve relevan... See morefrom GitHub - kingjulio8238/memary: Longterm Memory for Autonomous Agents.
Nicolay Gerold added 5mo ago
Data
- It turns out there's a handy feature in PostgreSQL called row constructor comparisons that allows me to compare tuples of columns. That's exactly what we need. Instead of doing CreateAt > ?1 OR (CreateAt = ?1 AND Id > ?2) , we can do ( CreateAt, Id) > (?1, ?2) . And the row constructor comparisons are lexicographical, meaning t... See more
from Making a Postgres query 1,000 times faster
Nicolay Gerold added 4mo ago
- Our Goals
We made it lightweight and kept the efficiency in mind:- Self-contained
We ship a single dependency-free binary that runs on all Linux distributions - Fast to deploy, safe to operate
We are sysadmins, we know the value of operator-friendly software - Deploy everywhere on every machine
We do not have a dedicated backbone, and neither do you,
so
from Garage - An open-source distributed object storage service
Nicolay Gerold added 2mo ago
- Self-contained
- SQLite Studio
Single binary, single command SQLite database explorer.
sqlite-studio <sqlite_db>
Features- Overview page with common metadata.
- Tables page with each table's metadata, including the disk size being used by each table.
- Infinite scroll rows view.
- A custom query page that gives you more access to your db.
More features available ... See morefrom GitHub - frectonz/sqlite-studio: SQLite database explorer
Nicolay Gerold added 4mo ago
- Datasette is a tool for exploring and publishing data. It helps people take data of any shape, analyze and explore it, and publish it as an interactive website and accompanying API.
Datasette is aimed at data journalists, museum curators, archivists, local governments, scientists, researchers and anyone else who has data that they wish to share with... See morefrom Datasette
Nicolay Gerold added 5mo ago
- At the current pace of media content creation, Reddit expects their media metadata to be roughly 50 terabytes. This means they need to implement sharding and partition their tables across multiple Postgres instances.
Reddit shards their tables based on post_id where they use range-based partitioning. All posts with a post_id in a certain range will ... See morefrom Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]
Nicolay Gerold added 5mo ago
- PGlite - Postgres in WASM
PGlite is a WASM Postgres build packaged into a TypeScript client library that enables you to run Postgres in the browser, Node.js and Bun, with no need to install any other dependencies. It is only 3.7mb gzipped.
import { PGlite } from "@electric-sql/pglite"
const db = new PGlite()
await db.query("select 'Hello world' as mes... See morefrom GitHub - electric-sql/pglite: Lightweight Postgres packaged as WASM into a TypeScript library for the browser, Node.js, Bun and Deno by electric-sql
Nicolay Gerold added 7mo ago