LLMs
ANY
LLM of your choice, statistical methods, or NLP models that runs
locally on your machine
:
- G-Eval
- Summarization
- Answer Relevancy
- Faithfulness
- Contextual Recall
- Contextual Precision
- RAGAS
- Hallucination
- Toxicity
- Bias
- etc.
GitHub - confident-ai/deepeval: The LLM Evaluation Framework
- Right now, GPTs are the easiest way of sharing structured prompts, which are programs, written in plain English (or another language), that can get the AI to do useful things. I discussed creating structured prompts last week, and all the same techniques apply, but the GPT system makes structured prompts more powerful and much easier to create,
Ethan Mollick โข Almost an Agent: What GPTs can do
OpenAI is treating its new marketplace seriously now: The brand new GPT store will come with REVENUE SHARING.... (missing in the Plugins launch)
and launching a Stateful Assistants API:
- Persistent Threads (/api/openai/threads)
- Built in Retrieval (chunking etc done for you)
- Code Interpreter (RIP Adv Data Analysis?)
- Speech to Text and Text to... See more
and launching a Stateful Assistants API:
- Persistent Threads (/api/openai/threads)
- Built in Retrieval (chunking etc done for you)
- Code Interpreter (RIP Adv Data Analysis?)
- Speech to Text and Text to... See more
swyx โข Tweet
๐ฑ๐ฆ๐ณ๐ง๐ฐ๐ณ๐ฎ๐ข๐ฏ๐ค๐ฆ: it will improve your LLM performance on given use cases (e.g., coding, extracting text, etc.). Mainly, the LLM will specialize in a given task (a specialist will always beat a generalist in its domain)
๐ค๐ฐ๐ฏ๐ต๐ณ๐ฐ๐ญ: you can refine how your model should behave on specific inputs and outputs, resulting in a more robust product
๐ฎ๐ฐ๐ฅ๐ถ๐ญ๐ข๐ณ๐ช๐ป๐ข๐ต๐ช๐ฐ๐ฏ:... See more
๐ค๐ฐ๐ฏ๐ต๐ณ๐ฐ๐ญ: you can refine how your model should behave on specific inputs and outputs, resulting in a more robust product
๐ฎ๐ฐ๐ฅ๐ถ๐ญ๐ข๐ณ๐ช๐ป๐ข๐ต๐ช๐ฐ๐ฏ:... See more
Shortwave โ rajhesh.panchanadhan@gmail.com [Gmail alternative]
Motivation for finetuning
The exact metrics we use depend on the application โ our main goal is to understand how users use the feature and quickly make improvements to better meet their needs. For internal applications, this might mean measuring efficiency and sentiment. For consumer-facing applications, we similarly focus on measures of user satisfaction - direct user... See more
Developing Rapidly with Generative AI
The new seed parameter enables reproducible outputs by making the model return consistent completions most of the time. This beta feature is useful for use cases such as replaying requests for debugging, writing more comprehensive unit tests, and generally having a higher degree of control over the model behavior. We at OpenAI have been using this... See more
New models and developer products announced at DevDay
๐บ๐ฒ๐๐ต๐ผ๐ฑ๐ ๐ผ๐ณ ๐ณ๐ถ๐ป๐ฒ-๐๐๐ป๐ถ๐ป๐ด ๐ฎ๐ป ๐ผ๐ฝ๐ฒ๐ป-๐๐ผ๐๐ฟ๐ฐ๐ฒ ๐๐๐ ๐ฒ๐
๐ถ๐t โ
- ๐๐ฐ๐ฏ๐ต๐ช๐ฏ๐ถ๐ฆ๐ฅ ๐ฑ๐ณ๐ฆ-๐ต๐ณ๐ข๐ช๐ฏ๐ช๐ฏ๐จ: utilize domain-specific data to apply the same pre-training process (next token prediction) on the pre-trained (base) model
- ๐๐ฏ๐ด๐ต๐ณ๐ถ๐ค๐ต๐ช๐ฐ๐ฏ ๐ง๐ช๐ฏ๐ฆ-๐ต๐ถ๐ฏ๐ช๐ฏ๐จ: the pre-trained (base) model is fine-tuned on a Q&A dataset to learn to answer questions
- ๐๐ช๐ฏ๐จ๐ญ๐ฆ-๐ต๐ข๐ด๐ฌ ๐ง๐ช๐ฏ๐ฆ-๐ต๐ถ๐ฏ๐ช๐ฏ๐จ: the... See more
- ๐๐ฐ๐ฏ๐ต๐ช๐ฏ๐ถ๐ฆ๐ฅ ๐ฑ๐ณ๐ฆ-๐ต๐ณ๐ข๐ช๐ฏ๐ช๐ฏ๐จ: utilize domain-specific data to apply the same pre-training process (next token prediction) on the pre-trained (base) model
- ๐๐ฏ๐ด๐ต๐ณ๐ถ๐ค๐ต๐ช๐ฐ๐ฏ ๐ง๐ช๐ฏ๐ฆ-๐ต๐ถ๐ฏ๐ช๐ฏ๐จ: the pre-trained (base) model is fine-tuned on a Q&A dataset to learn to answer questions
- ๐๐ช๐ฏ๐จ๐ญ๐ฆ-๐ต๐ข๐ด๐ฌ ๐ง๐ช๐ฏ๐ฆ-๐ต๐ถ๐ฏ๐ช๐ฏ๐จ: the... See more
Shortwave โ rajhesh.panchanadhan@gmail.com [Gmail alternative]
Today, weโre releasing the Assistants API, our first step towards helping developers build agent-like experiences within their own applications. An assistant is a purpose-built AI that has specific instructions, leverages extra knowledge, and can call models and tools to perform tasks. The new Assistants API provides new capabilities such as Code... See more
New models and developer products announced at DevDay
In addition to using our built-in capabilities, you can also define custom actions by making one or more APIs available to the GPT. Like plugins, actions allow GPTs to integrate external data or interact with the real-world. Connect GPTs to databases, plug them into emails, or make them your shopping assistant. For example, you could integrate a... See more