LLMs

How do models represent style, and how can we more precisely extract and steer it?

A commonly requested feature in almost any LLM-based writing application is “I want the AI to respond in my style of writing,” or “I want the AI to adhere to this style guide.” Aside from costly and complicated multi-stage finetuning processes like Anthropic’s RL with... See more

Principles for growable tools

There are three critical pieces to building a tool that can grow around its users over time.

1. Design around play . Sometimes I call this design around experimentation . Using the tool for day-to-day work should involve playing and experimenting with what’s possible with the tool. Whether that’s writing small programs to

... See more

Announcing Together Inference Engine – the fastest inference available

November 13, 2023・By Together

The Together Inference Engine is multiple times faster than any other inference service, with 117 tokens per second on Llama-2-70B-Chat and 171 tokens per second on Llama-2-13B-Chat

‍

Today we are announcing Together Inference Engine, the world’s... See more

My $0.02 is that a lot of the future research/work there will be figuring out how to identify effective sub-graphs to provide additional context, to avoid having to pass in the entire graph. As well as trying to identify ontology-less structures in real-time, which includes NER and RE, as well as named entity/relationship... See more

Zerox OCR

A dead simple way of OCR-ing a document for AI ingestion. Documents are meant to be a visual representation after all. With weird layouts, tables, charts, etc. The vision models just make sense!

The general logic:

• Pass in a PDF (URL or file buffer)


• Turn the PDF into a series of images


• Pass each image to GPT and ask nicely for Markdown


• Aggregat

... See more

To do this, we employ a technique known as AI-assisted evaluation, alongside traditional metrics for measuring performance. This helps us pick the prompts that lead to better quality outputs, making the end product more appealing to users. AI-assisted evaluation uses best-in-class LLMs (like GPT-4) to automatically critique how well the AI's... See more

When it comes to identifying where generative AI can make an impact, we dig into challenges that commonly:

• Involve analysis, interpretation, or review of unstructured content (e.g. text) at scale


• Require massive scaling that may be otherwise prohibitive due to limited resources


• Would be challenging for rules-based or traditional ML approaches

However, a key risk with several of these startups is the potential lack of a long-term moat. It is difficult to read too much into it given the stage of these startups and the limited public information available but it’s not difficult to poke holes at their long term defensibility. For example:

• If a startup is built on the premise of taking base

... See more

The exact metrics we use depend on the application — our main goal is to understand how users use the feature and quickly make improvements to better meet their needs. For internal applications, this might mean measuring efficiency and sentiment. For consumer-facing applications, we similarly focus on measures of user satisfaction - direct user... See more