r/dataengineering - Reddit
reddit.com
r/dataengineering - Reddit
Many of our interviewees questioned whether a data scientist could possess all the needed skills and were taking a team-based approach to assembling them.
Thomas H. Davenport • Big Data at Work: Dispelling the Myths, Uncovering the Opportunities
(1) The separation between storage and compute , as encouraged by data lake architectures (e.g. the implementation of P would look different in a traditional database like PostgreSQL, or a cloud warehouse like Snowflake). This architecture is the focus of the current system, and it is prevalent in most mid-to-large enterprises (its benefits that be... See more
Jacopo Tagliabue • Reproducible data science over data lakes: replayable data pipelines with Bauplan and Nessie.
The usage of dbt+Snowflake in spilling to disk in running dbt models at Instacart is bananas. 🍌🍌
*This is very expensive, and unnecessary
**Instacart is throwing cash at the problem, which they can do because the wheelbarrow of VC money, but this will get you fired in a… Show more