Polars — Processing hundreds of GBs of textual data on a daily basis at MDPI

The language is always only as good as its community. Let’s look at some of the existing open-source tools and frameworks built in and around Rust:

• DataFusion based on Apache Arrow: Apache Arrow DataFusion SQL Query Engine similar to Spark


• Polars: It’s a faster Pandas. Probably going to compete with DuckDB (?)


• Delta Lake Rust: A native Rust library fo

... See more

Overview¶

Ballista is a distributed compute platform primarily implemented in Rust, and powered by Apache Arrow.

Ballista has a scheduler and an executor process that are standard Rust executables and can be executed directly, but Dockerfiles are provided to build images for use in containerized environments, such as Docker, Docker Compose, and Kube... See more