LLMs

Here's my read on the situation:

* The TAM is massive, still so many businesses trying to figure out AI

* If you do deployments you’ll need to spend a of time hand holding clients through scoping projects (not unlike other dev works) since the material is so new

* Lot’s of opportunity in education

* The hard part isn’t the expertise, it’s distribution... See more

GPT-4 Turbo performs better than our previous models on tasks that require the careful following of instructions, such as generating specific formats (e.g., “always respond in XML”). It also supports our new JSON mode, which ensures the model will respond with valid JSON. The new API parameter response_format enables the model to constrain its... See more

We consider these aspects of our problem:

• Latency : How fast does the system need to respond to user input?


• Task Complexity : What level of understanding is required from the LLM? Is the input context and prompt super domain-specific?


• Prompt Length : How much context needs to be provided for the LLM to do its task?


• Quality : What is the acceptable

... See more

A new v0.4.0 release of lm-evaluation-harness is available !

New updates and features include:

• Internal refactoring


• Config-based task creation and configuration


• Easier import and sharing of externally-defined task config YAMLs


• Support for Jinja2 prompt design, easy modification of prompts + prompt imports from Promptsource


• More advanced configuration

... See more

MLServer aims to provide an easy way to start serving your machine learning models through a REST and gRPC interface, fully compliant with KFServing's V2 Dataplane spec. Watch a quick video introducing the project here.

• Multi-model serving, letting users run multiple models within the same process.


• Ability to run inference in parallel for vertical

... See more

Top considerations when choosing foundation models

Accuracy

Cost

Latency

Privacy

Top challenges when deploying production AI

Serving cost

Evaluation

Infra reliability

Model quality

How do models represent style, and how can we more precisely extract and steer it?

A commonly requested feature in almost any LLM-based writing application is “I want the AI to respond in my style of writing,” or “I want the AI to adhere to this style guide.” Aside from costly and complicated multi-stage finetuning processes like Anthropic’s RL with... See more

Today, we’re releasing the Assistants API, our first step towards helping developers build agent-like experiences within their own applications. An assistant is a purpose-built AI that has specific instructions, leverages extra knowledge, and can call models and tools to perform tasks. The new Assistants API provides new capabilities such as Code... See more