LLMs

The way that most RLHF is done to date has the entire response from a language model get an associated score. To anyone with an RL background, this is disappointing, because it limits the ability for RL methods to make connections about the value of each sub-component of text. Futures have been pointed to where this multi-step optimization comes at... See more

MLServer aims to provide an easy way to start serving your machine learning models through a REST and gRPC interface, fully compliant with KFServing's V2 Dataplane spec. Watch a quick video introducing the project here.

• Multi-model serving, letting users run multiple models within the same process.


• Ability to run inference in parallel for vertical

... See more

Langfuse is an open source observability & analytics solution for LLM-based applications. It is mostly geared towards production usage but some users also use it for local development of their LLM applications.

Langfuse is focused on applications built on top of LLMs. Many new abstractions and common best practices evolved recently, e.g. agents,... See more

A core research interest of mine is imagining new kinds of interfaces to text documents that are made possible by modern AI and software. I think an interesting place to look for such ideas may be interface designs for reading and writing legal documents .

Legal document-wrangling tools have a handful of properties that make it fertile ground for... See more

In general, I see LLMs to be used in two broad categories: data processing, which is more of a worker use-cases, where the latency isn't the biggest issue but rather quality, and in user-interactions, where latency is a big factor. I think for the faster case a faster fallback is necessary. Or you escalate upwards, you first rely on a smaller more... See more

We went to OpenAI's office in San Francisco yesterday to ask them all the questions we had on Quivr (YC W24), here is what we learned:

1. Their office is super nice & you can eat damn good croissant in SF!

2. We can expect GPT 3.5 & 4 price to keep going down

3. A lot of people are using the Assistants API to build their use cases

4. It costs 2M$ to... See more

• Query the RAG anyway and let the LLM itself chose whether to use the the RAG context or its built in knowledge


• Query the RAG but only provide the result to the LLM if it meets some level of relevancy (ie embedding distance) to the question


• Run the LLM both on it's own and with the RAG response, use a heuristic (or another LLM) to pick the best answer

... See more

Since we launched ChatGPT Enterprise a few months ago, early customers have expressed the desire for even more customization that aligns with their business. GPTs answer this call by allowing you to create versions of ChatGPT for specific use cases, departments, or proprietary datasets. Early customers like Amgen, Bain, and Square are already... See more