added by sari · updated 9mo ago
ML Infrastructure Tools for Model Building
- We found the ML engineering workflow to revolve around the following stages (Figure 1): (1) Data Preparation , which includes scheduled data acquisition, cleaning, labeling, and trans-formation, (2) Experimentation , which includes both data-driven and model-driven changes to increase overall ML performance, and is typically measured by metrics suc... See more
from "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning. by Shreya Shankar
Nicolay Gerold added
- (1) Tasks (a) Data collection, cleaning & labeling: human annotators , exploratory data analysis (b) Embeddings & feature engineering: normalization , bucketing / binning , word2vec (c) Data modeling & experimentation: accuracy , F1-score , precision , recall (d) Testing: scenario testing , AB testing , adaptive test-data (2) Biz/Org Ma... See more
from "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning. by Shreya Shankar
Nicolay Gerold added
- ata Collection Experimentation Evaluation and Deployment Monitoring and Response Metadata Data catalogs, Amundsen, AWS Glue, Hive metas-tores Weights & Biases, MLFlow, train/test set parameter configs, A/B test tracking tools Dashboards, SQL, metric functions and window sizes Unit Data cleaning tools Tensorflow, ML-lib, PyTorch, Scikit-learn, X... See more
from "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning. by Shreya Shankar
Nicolay Gerold added