Software Engineering
Why is Continuous Delivery for ML/AI hard(er)?
Since the challenge is not new and many valid solutions exist targeting traditional software projects, is there a reason to treat ML/AI systems any differently? Consider these three core challenges that are endemic in ML, AI, and data projects:
Since the challenge is not new and many valid solutions exist targeting traditional software projects, is there a reason to treat ML/AI systems any differently? Consider these three core challenges that are endemic in ML, AI, and data projects:
- Development and debugging cycles are more tedious due to c
How To Organize Continuous Delivery of ML/AI Systems: a 10-Stage Maturity Model | Outerbounds
Problem : Tight coupling leads to fragility - a change in one microservice breaks connected services.
Low cohesion results in unclear responsibilities.
Low cohesion results in unclear responsibilities.
- Solution : Apply loose coupling and high cohesion. In a loosely coupled system, each component is independent and changes in one will not affect the others. High cohesion means that each component i
Muaath Bin Ali • Microservices Design Principles
But now we are only logging that error message. It would be better to define a custom Exception that we can then handle in our API in order to return a specific error code to the user:
import pandas as pd
import logging
class DataLoadError(Exception):
"""Exception raised when the data cannot be loaded."""
def __init__(self, message="Data could not be lo... See more
import pandas as pd
import logging
class DataLoadError(Exception):
"""Exception raised when the data cannot be loaded."""
def __init__(self, message="Data could not be lo... See more
How to Write Clean Code in Python
And then, in the primary function of your API:
try:
df = load_data('path/to/data.csv')
# Further processing and model prediction
except DataLoadError as e:
# Return a response to the user with the error message
# For example: return Response({"error": str(e)}, status=400)Hopefully, at this point, you see where I’m going with this. What I’ve concluded, based on experience, is that positioning your project as an alternative implementation of something is a losing proposition. It doesn’t matter how smart you are. It doesn’t matter how hard you work. The problem is, when you build an alternative implementation, you’ve ... See more
The Alternative Implementation Problem
The complete Protobuf platform
Accelerate gRPC adoption with the Buf Schema Registry — built by the world's Protobuf experts.
Accelerate gRPC adoption with the Buf Schema Registry — built by the world's Protobuf experts.
Buf
Loose Coupling and High Cohesion: How To Avoid Tight Coupling and Low Cohesion?
Cohesion is a measure of the number of relationships that parts of a component have with each other.
High cohesion means that : All of the parts that are needed to deliver the component's functionality are included in the component.
Coupling is a measure of the number of ... See more
Cohesion is a measure of the number of relationships that parts of a component have with each other.
High cohesion means that : All of the parts that are needed to deliver the component's functionality are included in the component.
Coupling is a measure of the number of ... See more
Muaath Bin Ali • Microservices Design Principles
- Terraform Providers: Terraform is primarily used for defining the infrastructure resources. Its strength lies in its vast collection of providers that offer standardized ways to interact with various cloud services (AWS, Azure, GCP, etc.) and on-premises infrastructure. These providers are essentially plugins that understand the specific APIs and
Gemini - chat to supercharge your ideas
Your job involves turning the unknown into the known and translating vague ideas into actionable plans. This often involves a blend of:
- Detective work — ask the right questions, collect evidence, build a case theory, validate it
- Isolating uncertainty to the smallest components and proving / disproving theories
- Divide and conquer — breaking down the
3 Critical Skills You Need to Grow Beyond Senior Levels in Engineering
Pydantic Logfire — Uncomplicated Observability
From the team behind Pydantic, Logfire is an observability platform built on the same belief as our open source library — that the most powerful tools can be easy to use.
What sets Logfire apart:
From the team behind Pydantic, Logfire is an observability platform built on the same belief as our open source library — that the most powerful tools can be easy to use.
What sets Logfire apart:
- Simple and Powerful: Logfire's dashboard is simple relative to the power it provides, ensuring your entire e