fastlite

Rottnest : Data Lake Indices

You don't need ElasticSearch or some vector database to do full text search or vector search. Parquet + Rottnest is all you need. Rottnest is like Postgres indices for Parquet. Read more on what it can do for e.g. logs here.

Installation

Local installation: pip install rottnest .

Rottnest supports many different index typ... See more

Data bases have gotten so good at this, that the term is almost misleading now. “Base” suggests something rigid, without which the data would slip away. But the data is always there, just bits on a nameless hard disk. The structure and the accessibility that a modern database provides exist completely independently from that hard disk. That’s right... See more

• ClickHouse: It's a high performance columnar database that's great for real time queries. It enables querying and storing large amounts of data on commodity hardware. Some of my customers have millions of page views and I don't have an unlimited budget, so it's been very handy.


• PostgreSQL: My favorite database. Sane defaults, battle-tested, and well

... See more

2-5x faster 50% less memory local LLM finetuning

• Manual autograd engine - hand derived backprop steps.


• 2x to 5x faster than QLoRA. 50% less memory usage.


• All kernels written in OpenAI's Triton language.


• 0% loss in accuracy - no approximation methods - all exact.


• No change of hardware necessary. Supports NVIDIA GPUs since 2018+. Minimum CUDA Compute Cap

... See more

Spice.ai OSS

What is Spice?

Spice is a small, portable runtime that provides developers with a unified SQL query interface to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake.

Spice makes it easy to build data-driven and data-intensive applications by streamlining the use of data and mach... See more