LLMs

We identified 30 types of tasks that UX professionals used generative AI tools for in their work. We grouped these tasks under four roles: content editor, research assistant, ideation partner, or design assistant.

• Content editor : Generating and editing text, from microcopy to social media posts, based on specifications or copy given by UX

... See more

The xAI PromptIDE is an integrated development environment for prompt engineering and interpretability research. It accelerates prompt engineering through an SDK that allows implementing complex prompting techniques and rich analytics that visualize the network's outputs. We use it heavily in our continuous development of Grok.

So right now, LLMs (Large Language Models) are all the rage. But in the future, it’s possible that the way we get things done is composing things with a combination of LLMs, SMMs (Small, Mighty Models), agents and tools.

It’s what I call Cognitive Composition (because it sounds cool and I have a longtime love affair with alliteration).

This is how we... See more

Generative AI can automate simple tasks

By automating simpler, tedious tasks (generating boilerplate code, fixing linter errors, generating unit tests, etc.), generative AI can help engineers focus on more complex tasks.

Generative AI can improve quality & reliability

Since generative AI models are trained on large codebases, they have the potential... See more

📦 Service Deployment - Ray Serve (https://lnkd.in/eAV-Y6RN)

🧰 Data Transformation - Ray Data (https://lnkd.in/e7wYmenc)

🔌 LLM Integration - AIConfig (https://lnkd.in/esvH5NQa)

🗄 Vector Database - Weaviate (https://weaviate.io/)

📚 Supervised LLM Fine-Tuning - HuggingFace TLR (https://lnkd.in/e8_QYF-P)

📈 LLM Observability - Weights & Biases Traces (https... See more

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Table of Contents

1. Introduction


2. Key LLM Serving Techniques


3. Dynamic SplitFuse: A Novel Prompt and Generation Composition Strategy


4. Performance Evaluation


5. DeepSpeed-FastGen: Implementation and Usage


6. Try out DeepSpeed-FastGen


7. Acknowledgements

1. Introduction

Large... See more

• Multiple indices. Splitting the document corpus up into multiple indices and then routing queries based on some criteria. This means that the search is over a much smaller set of documents rather than the entire dataset. Again, it is not always useful, but it can be helpful for certain datasets. The same approach works with the LLMs themselves.

... See more

Overview

MaxText is a high performance , highly scalable , open-source LLM written in pure Python/Jax and targeting Google Cloud TPUs and GPUs for training and inference . MaxText achieves high MFUs and scales from single host to very large clusters while staying simple and "optimization-free" thanks to the power of Jax and the XLA compiler.

MaxText... See more