Nicolay Gerold

For High Throughput data, Grab uses Apache Avro with a strategy called Merge on Read (MOR) .

Here's the main operations with Merge on Read:

• Write Operations - When data is written, it's appended to the end of a log file. This is much more efficient than merging it in the current data and reduces the latency of writes.


• Read Operations - When you need

... See more

In 2024, Supabase has emerged as a very robust authentication solution and offers easy integration with React apps, both on the server, say, in a Next.js project, and on the client. In past years, Firebase was chosen for similar reasons, but it’s harder to integrate on the server side of things.

If you’re using Next.js, you have a ton of options... See more

• Some datasets become standards for data exchange . S&P’s CUSIP for publicly traded securities, D&B’s DUNS for private companies, Datavant’s universal patient key for medical records, LiveRamp’s RampID for advertiser audiences are all examples here, albeit for varying degrees of “table-stakes-ness / industry adoption”.


• Some datasets become evaluation

... See more

At the current pace of media content creation, Reddit expects their media metadata to be roughly 50 terabytes. This means they need to implement sharding and partition their tables across multiple Postgres instances.

Reddit shards their tables based on post_id where they use range-based partitioning. All posts with a post_id in a certain range will... See more

Glutamine also reduces sugar craving by increasing blood sugar levels. Emulsifiers strip away the mucosal lining of the blood and cause the neurons to retract deeper into the tract. This stops the triggering of CCK signals. They also increase the neurons measuring sugar and trigger dopamine in the brain. This can lead to an overcompensation effect... See more

Rottnest : Data Lake Indices

You don't need ElasticSearch or some vector database to do full text search or vector search. Parquet + Rottnest is all you need. Rottnest is like Postgres indices for Parquet. Read more on what it can do for e.g. logs here.

Installation

Local installation: pip install rottnest .

Rottnest supports many different index... See more

I’m often asked what problem I’d solve if I were to start another company. I probably won’t do a startup any time soon (because startups are hard), but here are some of the problems I find interesting. If you’re solving any of them, I’d love to chat.

1. Data synthesis: AI has become really good both at generating and annotating data. The challenge... See more

1. Introduction of DeepSeek Coder

DeepSeek Coder is composed of a series of code language models, each trained from scratch on 2T tokens, with a composition of 87% code and 13% natural language in both English and Chinese. We provide various sizes of the code model, ranging from 1B to 33B versions. Each model is pre-trained on project-level code... See more