LLMs

Traditional AI - The most secure, understandable, and performant. However, Good implementations of traditional AI require that we define the rules behind the system, which makes it unfeasible for many of the use cases that the other 2 techniques thrive on.

Supervised Machine Learning- Middle of the road b/w AI and Deep Learning. Good when we have

Devansh • How to Pick between Traditional AI, Supervised Machine Learning, and Deep Learning [Thoughts]

Where would I add generative AI? Generative AI has the ease of accessibility of traditional AI, where people think it is understandable, but it does not have that feature in itself. It also has the opaque and costly nature of DL. Many companies are at the moment rushing into developing things with generative AI without having any prior foundation in AI and any processes set up to manage it: data ops, devops, …

Traditional AI forces you to think about how something works, understand the system, and then define the rules for it. ML lets you use features and feature importance to shortcut some. Deep Learning allows you to brute force it. Generative AI allows you to brute force without any background in DL.

First of all, I'd say you have a bigger problem where your company is trying to find nails with a hammer. That is where your sentiment comes from, and could be an obstacle for both you and the company. It's the same deal when I see people keep on talking about RAG, and nowadays "modular RAG", when really, you could treat everything as a software... See more

r/MachineLearning - Reddit

The quality of dataset is 95% of everything. The rest 5% is not to ruin it with bad parameters.

After 500+ LoRAs made, here is the secret

Document search and synthesis

Scores of organizations want to harness generative AI so employees can easily find the most relevant documents through improved search results and summaries. For example, your organization can reduce the time it takes employees to find answers to common HR- and process-related questions. Internal manuals and sites are... See more

Donna Schut • The Prompt: Takeaways from hundreds of conversations about generative AI - part 1 | Google Cloud Blog

Setting up the necessary machine learning infrastructure to run these big models is another challenge. We need a dedicated model server for running model inference (using frameworks like Triton oder vLLM), powerful GPUs to run everything robustly, and configurability in our servers to make sure they're high throughput and low latency. Tuning the... See more

Developing Rapidly with Generative AI

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Table of Contents

Introduction

Key LLM Serving Techniques

Dynamic SplitFuse: A Novel Prompt and Generation Composition Strategy

Performance Evaluation

DeepSpeed-FastGen: Implementation and Usage

Try out DeepSpeed-FastGen

Acknowledgements

1. Introduction

Large... See more

microsoft • DeepSpeed-FastGen

memary: Open-Source Longterm Memory for Autonomous Agents

memary demo

Why use memary?

Agents use LLMs that are currently constrained to finite context windows. memary overcomes this limitation by allowing your agents to store a large corpus of information in knowledge graphs, infer user knowledge through our memory modules, and only retrieve... See more

GitHub - kingjulio8238/memary: Longterm Memory for Autonomous Agents.

Data

What is Substrate?

Substrate is an AI inference platform. In particular, it excels at enabling complex multi-model workloads . At its core, Substrate is 1) a collection of cutting-edge AI models – tuned for optimum performance, and 2) a set of composable APIs for relating these models to each other. We believe having both of these components in one... See more

Nextra: the next docs builder

Deploying a Generative AI model requires more than a VM with a GPU. It normally includes:

Container Service : Most often Kubernetes to run LLM Serving solutions like Hugging Face Text Generation Inference or vLLM.

Compute Resources : GPUs for running models, CPUs for management services

Networking and DNS : Routing traffic to the appropriate