Controllable Generation

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with LLMs faster and more controllable by co-designing the frontend language and the runtime system.

The core features of SGLang include:

• A Flexible Front-End Language : This allows for easy programming of LLM applications with multiple ch

... See more

Super JSON Mode is a Python framework that enables the efficient creation of structured output from an LLM by breaking up a target schema into atomic components and then performing generations in parallel.

It supports both state of the art LLMs via OpenAI 's legacy completions API and open source LLMs such as via Hugging Face Transformers and vLLM .... See more

LMQL is a programming language for LLMs.Robust and modular LLM prompting using types, templates, constraints and an optimizing runtime.

HeimdaLLM

Pronounced [ˈhaɪm.dɔl.əm] or HEIM-dall-EM

HeimdaLLM is a robust static analysis framework for validating that LLM-generated structured output is safe. It currently supports SQL.

In simple terms, it helps makes sure that AI won't wreck your systems.

Some techniques I have seen in the wild

Create a semi-strict programming language like promptlang - https://github.com/ruvnet/promptlang

this style was popular back in the early days of gpt-3.5 when it landed, for some reason, it fell out of fashion

Write a super long persona mapping prompt with a well defined action space

Best example of this is the V... See more