Software Engineering
What is Hatchet?
Hatchet replaces difficult to manage legacy queues or pub/sub systems so you can design durable workloads that recover from failure and solve for problems like concurrency , fairness , and rate limiting . Instead of managing your own task queue or pub/sub system, you can use Hatchet to distribute your functions between a set of... See more
Hatchet replaces difficult to manage legacy queues or pub/sub systems so you can design durable workloads that recover from failure and solve for problems like concurrency , fairness , and rate limiting . Instead of managing your own task queue or pub/sub system, you can use Hatchet to distribute your functions between a set of... See more
hatchet-dev • GitHub - hatchet-dev/hatchet: A distributed, fault-tolerant task queue
But now we are only logging that error message. It would be better to define a custom Exception that we can then handle in our API in order to return a specific error code to the user:
import pandas as pd
import logging
class DataLoadError(Exception):
"""Exception raised when the data cannot be loaded."""
def __init__(self, message="Data could not be... See more
import pandas as pd
import logging
class DataLoadError(Exception):
"""Exception raised when the data cannot be loaded."""
def __init__(self, message="Data could not be... See more
How to Write Clean Code in Python
And then, in the primary function of your API:
try:
df = load_data('path/to/data.csv')
# Further processing and model prediction
except DataLoadError as e:
# Return a response to the user with the error message
# For example: return Response({"error": str(e)}, status=400)Build & Deployments
Our build process starts by pushing changes to a repository on GitHub. When code is pushed to a repository through a pull request, it triggers a job to build the changes made to the branch and deploy them in isolation. This worklow happens interactively within the pull request, where the author has visibility of all steps.
Some... See more
Our build process starts by pushing changes to a repository on GitHub. When code is pushed to a repository through a pull request, it triggers a job to build the changes made to the branch and deploy them in isolation. This worklow happens interactively within the pull request, where the author has visibility of all steps.
Some... See more
How we approach CI/CD · Resend
Why is Continuous Delivery for ML/AI hard(er)?
Since the challenge is not new and many valid solutions exist targeting traditional software projects, is there a reason to treat ML/AI systems any differently? Consider these three core challenges that are endemic in ML, AI, and data projects:
Since the challenge is not new and many valid solutions exist targeting traditional software projects, is there a reason to treat ML/AI systems any differently? Consider these three core challenges that are endemic in ML, AI, and data projects:
- Development and debugging cycles are more tedious due to
How To Organize Continuous Delivery of ML/AI Systems: a 10-Stage Maturity Model | Outerbounds
" Knowledge is powerful, be careful how you use it! "
A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools, and more.
Created by trimstray and contributors
📔 What is it?
This repository is a collection of various materials and tools that I use every day in my work. It contains a lot of useful information... See more
A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools, and more.
Created by trimstray and contributors
📔 What is it?
This repository is a collection of various materials and tools that I use every day in my work. It contains a lot of useful information... See more
GitHub - trimstray/the-book-of-secret-knowledge: A collection of inspiring lists, manuals, cheatsheets, blogs, hacks, one-liners, cli/web tools and more.
More than reading popular books on Design Patterns, two things that helped me write and structure a large codebase better were
reading a lot of good open source codebases (with similar stack)coding and collaborating a lot on the same codebase
I understand the importance of reading general-purpose design patterns, but they might not suit your... See more
reading a lot of good open source codebases (with similar stack)coding and collaborating a lot on the same codebase
I understand the importance of reading general-purpose design patterns, but they might not suit your... See more
How to get better at writing and structuring a large codebase?
Hopefully, at this point, you see where I’m going with this. What I’ve concluded, based on experience, is that positioning your project as an alternative implementation of something is a losing proposition. It doesn’t matter how smart you are. It doesn’t matter how hard you work. The problem is, when you build an alternative implementation, you’ve... See more
The Alternative Implementation Problem
- Tornado is a Python web framework and asynchronous networking library designed for handling long-lived network connections. It is built to be non-blocking and uses an event loop to manage all operations. While Tornado itself does not rely on a thread pool for its core operations, it provides interfaces for integrating with thread pools (e.g.,
Saverio Mazza • FastAPI: Thread Pool and Event Loop
Some important formatting rules available in this style guide:
But remember: The formatting rules should make the code more readable. Sometimes,... See more
- Use four spaces for code indentation
- Limit all lines to a maximum of 79 characters
- Avoid extraneous whitespace in certain situations (i.e., inside brackets, between trailing comma and close parenthesis, ...)
But remember: The formatting rules should make the code more readable. Sometimes,... See more