LLMs

We identified 30 types of tasks that UX professionals used generative AI tools for in their work. We grouped these tasks under four roles: content editor, research assistant, ideation partner, or design assistant.

• Content editor : Generating and editing text, from microcopy to social media posts, based on specifications or copy given by UX

... See more

Pipeline RobustQA Avg. score Avg. response time (secs) Azure Cognitive Search Retriever + GPT4 + Ada 72.36 >1.0s Canopy (Pinecone) 59.61 >1.0s Langchain + Pinecone + OpenAI 61.42 <0.8s Langchain + Pinecone + Cohere 69.02 <0.6s LlamaIndex + Weaviate Vector Store - Hybrid Search 75.89 <1.0s RAG Google Cloud VertexAI-Search + Bison... See more

Top considerations when choosing foundation models

Accuracy

Cost

Latency

Privacy

Top challenges when deploying production AI

Serving cost

Evaluation

Infra reliability

Model quality

Setting up the necessary machine learning infrastructure to run these big models is another challenge. We need a dedicated model server for running model inference (using frameworks like Triton oder vLLM), powerful GPUs to run everything robustly, and configurability in our servers to make sure they're high throughput and low latency. Tuning the... See more

The quality of dataset is 95% of everything. The rest 5% is not to ruin it with bad parameters.

The way that most RLHF is done to date has the entire response from a language model get an associated score. To anyone with an RL background, this is disappointing, because it limits the ability for RL methods to make connections about the value of each sub-component of text. Futures have been pointed to where this multi-step optimization comes at... See more

GPT-4 Turbo performs better than our previous models on tasks that require the careful following of instructions, such as generating specific formats (e.g., “always respond in XML”). It also supports our new JSON mode, which ensures the model will respond with valid JSON. The new API parameter response_format enables the model to constrain its... See more