LLMs
- You have access to a proprietary asset (like data) that others don’t have easy access to. In our “write job postings” example, perhaps you have a corpus of thousands of job postings including some outcome scores (as to how well they did). You could use this data to create better job postings. Others don’t have ready access to this data. Note: The
Dharmesh Shah • How To Build a Defensible A.I. Startup
Protecting LLM products:
(1) Is hard to bootstrap. This already hints to existing customers or you need to get a bunch of your customers to co-develop (insurance model → companies pooling their data to solve a problem they all have). This runs into a bunch of issues: competitive drive of the companies, data privacy and security.
(2) Reserved for existing companies. This is the co-pilot model.
(3) This might be the most sustainable one, but it is also the hardest one. I have not seen anything in that direction yet besides OpenAI.
📦 Service Deployment - Ray Serve (https://lnkd.in/eAV-Y6RN)
🧰 Data Transformation - Ray Data (https://lnkd.in/e7wYmenc)
🔌 LLM Integration - AIConfig (https://lnkd.in/esvH5NQa)
🗄 Vector Database - Weaviate (https://weaviate.io/)
📚 Supervised LLM Fine-Tuning - HuggingFace TLR (https://lnkd.in/e8_QYF-P)
📈 LLM Observability - Weights & Biases Traces (https... See more
🧰 Data Transformation - Ray Data (https://lnkd.in/e7wYmenc)
🔌 LLM Integration - AIConfig (https://lnkd.in/esvH5NQa)
🗄 Vector Database - Weaviate (https://weaviate.io/)
📚 Supervised LLM Fine-Tuning - HuggingFace TLR (https://lnkd.in/e8_QYF-P)
📈 LLM Observability - Weights & Biases Traces (https... See more
Paul Venuto • feed updates
Top considerations when choosing foundation models
Accuracy
Cost
Latency
Privacy
Top challenges when deploying production AI
Serving cost
Evaluation
Infra reliability
Model quality
Accuracy
Cost
Latency
Privacy
Top challenges when deploying production AI
Serving cost
Evaluation
Infra reliability
Model quality
Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.
.png?table=block&id=e2eaaa6a-a9a8-4f09-a88e-888ba717d58d&spaceId=996f2b3b-deaa-4214-aedb-cbc914a1833e&width=1200&userId=&cache=v2)
MLServer aims to provide an easy way to start serving your machine learning models through a REST and gRPC interface, fully compliant with KFServing's V2 Dataplane spec. Watch a quick video introducing the project here.
- Multi-model serving, letting users run multiple models within the same process.
- Ability to run inference in parallel for vertical
GitHub - SeldonIO/MLServer: An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
The context size of the input is too small for when you want to analyse CSV's with 1000's of rows and embedding doesn't really work because it loses context.
r/LLMDevs - Reddit
ANY
LLM of your choice, statistical methods, or NLP models that runs
locally on your machine
:
- G-Eval
- Summarization
- Answer Relevancy
- Faithfulness
- Contextual Recall
- Contextual Precision
- RAGAS
- Hallucination
- Toxicity
- Bias
- etc.
