LLMs

In some applications, such as inline code suggestions, the best AI models are too expensive, so tools like Github Copilot use carefully tuned smaller models and various search heuristics to provide results. In other applications, even the largest models, like GPT-4, are too cheap!

Matei Zaharia, Omar Khattab, Lingjiao Chen, et al. • The Shift From Models to Compound AI Systems

Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.

I’ve been giving talks and speaking with engineers and non-technical audiences about interpretability since 2022, and I still struggle to explain exactly what a “feature” is. I often use words like “concept” or “style”, or establish metaphors to debugging programs or making fMRI scans of brains. Both metaphors help people outside of the subfield... See more

Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]

OpenAI is treating its new marketplace seriously now: The brand new GPT store will come with REVENUE SHARING.... (missing in the Plugins launch)

and launching a Stateful Assistants API:

- Persistent Threads (/api/openai/threads)

- Built in Retrieval (chunking etc done for you)

- Code Interpreter (RIP Adv Data Analysis?)

- Speech to Text and Text to... See more

swyx • Tweet

The need for better AI or LLM-specific infrastructure, along with the host of problems that come with non-deterministic of LLMs, means that there’s more software work ahead of us, not less. Abstraction layers like LLMs create more possibilities and thus, more work.

Is this a good thing or a bad thing? I’m not sure.

A great example of this is frontend... See more

Shortwave — rajhesh.panchanadhan@gmail.com [Gmail alternative]

LLM-PowerHouse: A Curated Guide for Large Language Models with Custom Training and Inferencing

Welcome to LLM-PowerHouse, your ultimate resource for unleashing the full potential of Large Language Models (LLMs) with custom training and inferencing. This GitHub repository is a comprehensive and curated guide designed to empower developers,... See more

ghimiresunil • GitHub - ghimiresunil/LLM-PowerHouse-A-Curated-Guide-for-Large-Language-Models-with-Custom-Training-and-Inferencing: LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.

Overview

Loki is our open-source solution designed to automate the process of verifying factuality. It provides a comprehensive pipeline for dissecting long texts into individual claims, assessing their worthiness for verification, generating queries for evidence search, crawling for evidence, and ultimately verifying the claims. This tool is... See more

Libr-AI • GitHub - Libr-AI/OpenFactVerification: Open-source solution designed to automate the process of verifying factuality

We consider these aspects of our problem:

Latency : How fast does the system need to respond to user input?

Task Complexity : What level of understanding is required from the LLM? Is the input context and prompt super domain-specific?

Prompt Length : How much context needs to be provided for the LLM to do its task?

Quality : What is the acceptable

Developing Rapidly with Generative AI

We generally lean towards picking more advanced commercial LLMs to quickly validate our ideas and obtain early feedback from users. Although they may be expensive, the general idea is that if problems can't be adequately solved with state-of-the-art foundational models like GPT-4, then more often than not, those problems may not be addressable... See more