LLMs

How enterprises are using open source LLMs: 16 examples.

Many use Llama-2: Brave, Wells Fargo, IBM, The Grammy Awards, Perplexity, Shopify, LyRise, Niantic....

Quote: “A lot of customer are asking themselves: Wait a second, why am I paying for super large model that knows very little about my business? Couldn’t I just use one of these open-source... See more

Setting up the necessary machine learning infrastructure to run these big models is another challenge. We need a dedicated model server for running model inference (using frameworks like Triton oder vLLM), powerful GPUs to run everything robustly, and configurability in our servers to make sure they're high throughput and low latency. Tuning the... See more

Amplify Partners was running a survey among 800+ AI engineers to bring transparency to the AI Engineering space. The report is concise, yet it provides a wealth of insights into the technologies and methods employed by companies for the implementation of AI products.

Highlights

👉 Top AI use cases are code intelligence, data extraction and workflow... See more

In general, I see LLMs to be used in two broad categories: data processing, which is more of a worker use-cases, where the latency isn't the biggest issue but rather quality, and in user-interactions, where latency is a big factor. I think for the faster case a faster fallback is necessary. Or you escalate upwards, you first rely on a smaller more... See more

In addition to using our built-in capabilities, you can also define custom actions by making one or more APIs available to the GPT. Like plugins, actions allow GPTs to integrate external data or interact with the real-world. Connect GPTs to databases, plug them into emails, or make them your shopping assistant. For example, you could integrate a... See more

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Table of Contents

1. Introduction


2. Key LLM Serving Techniques


3. Dynamic SplitFuse: A Novel Prompt and Generation Composition Strategy


4. Performance Evaluation


5. DeepSpeed-FastGen: Implementation and Usage


6. Try out DeepSpeed-FastGen


7. Acknowledgements

1. Introduction

Large... See more

Document search and synthesis

Scores of organizations want to harness generative AI so employees can easily find the most relevant documents through improved search results and summaries. For example, your organization can reduce the time it takes employees to find answers to common HR- and process-related questions. Internal manuals and sites are... See more

🤖 Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

• Why CrewAI


• Getting Started


• Key Features


• Examples


• Local Open Source Models


• CrewAI x AutoGen x ChatDev


• Contribution


• 💬 CrewAI Discord Community


• Hire Consulting


• License

... See more