Software Engineering
1. Streamlining data exploration and analysis:
- Context-Aware Suggestions: Receive AI-powered suggestions for relevant tools, functions, and libraries based on your specific dataset and analysis goals.
- Contextual search for functions and libraries: Quickly find relevant functions and libraries from various programming languages, such as Python and R,
Martha J. Lindeman • How Data Science AI Tools are Simplifying Workflows
Before you start working, figure out:
After you’ve finished your work, check:
4. Did I miss anything?
- What does the business want?
- Which criteria should I fulfill to call my work “good”?
- What do I need to do to complete this particular task?
After you’ve finished your work, check:
4. Did I miss anything?
It’s time to upgrade from “hard-working” to “highly efficient”
Here's a list of those log levels from lowest precedence to highest:
notset (0) - Indicates that ancestor loggers should be consulted for the log level or that all events are logged (default setting)
debug (10) - Detailed information that would be of interest to the developer for diagnostic purposes
info (20) - Information that confirms that your... See more
notset (0) - Indicates that ancestor loggers should be consulted for the log level or that all events are logged (default setting)
debug (10) - Detailed information that would be of interest to the developer for diagnostic purposes
info (20) - Information that confirms that your... See more
Shortwave — The smartest email app on planet Earth
To make life for developers easier, be explicit in what exactly is being returned. In the Stripe API, we have an object field in the response that makes it abundantly clear what we’re working with. For example, the API route
/v1/customers/:customer/payment_methods/:payment_method
Enter fullscreen mode
Exit fullscreen mode
returns a PaymentMethod... See more
/v1/customers/:customer/payment_methods/:payment_method
Enter fullscreen mode
Exit fullscreen mode
returns a PaymentMethod... See more
Common Design Patterns at Stripe
Your job involves turning the unknown into the known and translating vague ideas into actionable plans. This often involves a blend of:
- Detective work — ask the right questions, collect evidence, build a case theory, validate it
- Isolating uncertainty to the smallest components and proving / disproving theories
- Divide and conquer — breaking down the
3 Critical Skills You Need to Grow Beyond Senior Levels in Engineering
A thread pool is a collection of pre-initialized threads that can be used to execute tasks. Instead of starting a new thread for each task (which can be resource-intensive), a thread from this pool is reused to perform the task. This approach is beneficial for handling multiple operations in parallel, especially when these operations are blocking... See more
Saverio Mazza • FastAPI: Thread Pool and Event Loop
But now we are only logging that error message. It would be better to define a custom Exception that we can then handle in our API in order to return a specific error code to the user:
import pandas as pd
import logging
class DataLoadError(Exception):
"""Exception raised when the data cannot be loaded."""
def __init__(self, message="Data could not be... See more
import pandas as pd
import logging
class DataLoadError(Exception):
"""Exception raised when the data cannot be loaded."""
def __init__(self, message="Data could not be... See more
How to Write Clean Code in Python
And then, in the primary function of your API:
try:
df = load_data('path/to/data.csv')
# Further processing and model prediction
except DataLoadError as e:
# Return a response to the user with the error message
# For example: return Response({"error": str(e)}, status=400)Why is Continuous Delivery for ML/AI hard(er)?
Since the challenge is not new and many valid solutions exist targeting traditional software projects, is there a reason to treat ML/AI systems any differently? Consider these three core challenges that are endemic in ML, AI, and data projects:
Since the challenge is not new and many valid solutions exist targeting traditional software projects, is there a reason to treat ML/AI systems any differently? Consider these three core challenges that are endemic in ML, AI, and data projects:
- Development and debugging cycles are more tedious due to
How To Organize Continuous Delivery of ML/AI Systems: a 10-Stage Maturity Model | Outerbounds
Build & Deployments
Our build process starts by pushing changes to a repository on GitHub. When code is pushed to a repository through a pull request, it triggers a job to build the changes made to the branch and deploy them in isolation. This worklow happens interactively within the pull request, where the author has visibility of all steps.
Some... See more
Our build process starts by pushing changes to a repository on GitHub. When code is pushed to a repository through a pull request, it triggers a job to build the changes made to the branch and deploy them in isolation. This worklow happens interactively within the pull request, where the author has visibility of all steps.
Some... See more