LLMs

The context size of the input is too small for when you want to analyse CSV's with 1000's of rows and embedding doesn't really work because it loses context.

a couple of the top of my head:

• LLM in the loop with preference optimization


• synthetic data generation


• cross modality "distillation" / dictionary remapping


• constrained decoding

My $0.02 is that a lot of the future research/work there will be figuring out how to identify effective sub-graphs to provide additional context, to avoid having to pass in the entire graph. As well as trying to identify ontology-less structures in real-time, which includes NER and RE, as well as named entity/relationship... See more

First time here? Go to our setup guide

Features

• 🤖 Multiple model integrations: OpenAI, transformers, llama.cpp, exllama2, mamba


• 🖍️ Simple and powerful prompting primitives based on the Jinja templating engine


• 🚄 Multiple choices, type constraints and dynamic stopping


• ⚡ Fast regex-structured generation


• 🔥 Fast JSON generation following a JSON schema

... See more

However development time, and maintenance can offset these savings. Hiring skilled data scientists, machine learning engineers, and DevOps professionals can be expensive and time consuming. Using available resources for “reimplementing” solutions hinder innovation and lead to a lack of focus. Since You not longer work on improving your model or... See more

However, a key risk with several of these startups is the potential lack of a long-term moat. It is difficult to read too much into it given the stage of these startups and the limited public information available but it’s not difficult to poke holes at their long term defensibility. For example:

• If a startup is built on the premise of taking base

... See more

• Multiple indices. Splitting the document corpus up into multiple indices and then routing queries based on some criteria. This means that the search is over a much smaller set of documents rather than the entire dataset. Again, it is not always useful, but it can be helpful for certain datasets. The same approach works with the LLMs themselves.

... See more

Langfuse is an open source observability & analytics solution for LLM-based applications. It is mostly geared towards production usage but some users also use it for local development of their LLM applications.

Langfuse is focused on applications built on top of LLMs. Many new abstractions and common best practices evolved recently, e.g. agents,... See more

we’re in a capability overhang - the AI tech that already exists has huge potential impact, whether you engage or not, so get ahead by exploring

the appropriate approach is pathfinding which uses experiments to learn and, critically, artefacts to tell the organisation what to do next.