Data Storage

VectorDB-recipes

Dive into building GenAI applications! This repository contains examples, applications, starter code, & tutorials to help you kickstart your GenAI projects.

These are built using LanceDB, a free, open-source, serverless vectorDB that requires no setup .

It integrates into python data ecosystem so you can simply start using these in

lancedb • GitHub - lancedb/vectordb-recipes: High quality resources & applications for LLMs, multi-modal models and VectorDBs

SQL has limitations as it is built on relational concepts and relies on binary joins.

The future of databases is shifting towards relational knowledge graphs, allowing the flexibility to work with various data structures beyond tables.

Businesses are moving towards explicitly modeling business semantics and logic, which are often stored in... See more

Nicolay Gerold • Tweet

Classwords are suffixes added to database column names to indicate the type of data they contain. This improves readability and makes it easier to understand the database schema. Base classwords include text, calendar, numeric and domain-specific types. It is best to avoid redundancy in column names, as this can lead to unnecessary verbosity. Using... See more

‎Gemini - chat to supercharge your ideas

Text Classwords

identifier (or id)
code[_<standard>]
name
description (or desc)
indicator (or ind)
number
text

Calendar Classwords

date
datetime[<timezone>] (or dt[<timezone>])
timestamp[<timezone>] (or ts[<timezone>])

Numeric Classwords

count
amount[_<currency>]
<quantity_property>[_<unit_of_measure>]
ratio
factor
percent (or pct)

Domain-Specific Classwords

uri
address
email
sku
json
geojson

Local database for development

Each table in the database had an accompanying script that would generate a subset of the data for use in local development, since the final database was too large to run on a developer's machine.

This let each developer work with a live, local, copy of the database and enabled efficient development of changes.

I highly... See more

Bill Mill • notes.billmill.org

Search

more

/

Sub-second search & analytics

engine on cloud storage

with

less

Search more with less | Quickwit

At the current pace of media content creation, Reddit expects their media metadata to be roughly 50 terabytes. This means they need to implement sharding and partition their tables across multiple Postgres instances.

Reddit shards their tables based on post_id where they use range-based partitioning. All posts with a post_id in a certain range will... See more

Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]

Spice.ai OSS

What is Spice?

Spice is a small, portable runtime that provides developers with a unified SQL query interface to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake.

Spice makes it easy to build data-driven and data-intensive applications by streamlining the use of data and... See more

spiceai • GitHub - spiceai/spiceai: A unified SQL query interface and portable runtime to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake.

SQL Studio

Single binary, single command SQL database explorer. SQL studio supports SQLite , libSQL , PostgreSQL , MySQL and DuckDB .

Local SQLite DB File

sql-studio sqlite [sqlite_db]

Remote libSQL Server

sql-studio libsql [url] [auth_token]

PostgreSQL Server

sql-studio postgres [url]

MySQL/MariaDB Server

sql-studio mysql [url]

Local DuckDB File

sq... See more

frectonz • GitHub - frectonz/sql-studio: SQL Database Explorer [SQLite, libSQL, PostgreSQL, MySQL/MariaDB, DuckDB, ClickHouse]

We can't share the exact formula for our search ranking, but here are the few parameters we consider:

Exact match (rank #1)

Frequency of matching lexemes using ts_rank

Similarity score using similarity

Type of record

Popularity of the search result

Similarity between the result’s alias and query

Inverse of the result’s string length