LLMs

For the deployment side of things, we found that the performance of our training process was quite slow, especially when it gets into these large language models and when you train from scratch. MosaicML offers what's called programmatic optimization, which is not so much on the hardware side of things, but rather on the algorithmic side. Can you... See more

Easily chunk complex documents the same way a human would.

Chunking documents is a challenging task that underpins any RAG system. High quality results are critical to a sucessful AI application, yet most open-source libraries are limited in their ability to handle complex documents.

Open Parse is designed to fill this gap by providing a flexible,... See more

Generative AI can automate simple tasks

By automating simpler, tedious tasks (generating boilerplate code, fixing linter errors, generating unit tests, etc.), generative AI can help engineers focus on more complex tasks.

Generative AI can improve quality & reliability

Since generative AI models are trained on large codebases, they have the potential... See more

Developers can now generate human-quality speech from text via the text-to-speech API. Our new TTS model offers six preset voices to choose from and two model variants, tts-1 and tts-1-hd . tts is optimized for real-time use cases and tts-1-hd is optimized for quality. Pricing starts at $0.015 per input 1,000 characters. Check out our TTS guide to... See more

API wrappers, general-purpose AI tools and third-party AI tools for big platforms.

API wrappers have a weak moat.

General AI tools try to be the jack-of-all-trades.

Big platforms will eat up small apps by adding similar AI features themselves.

Deploying a Generative AI model requires more than a VM with a GPU. It normally includes:

• Container Service : Most often Kubernetes to run LLM Serving solutions like Hugging Face Text Generation Inference or vLLM.


• Compute Resources : GPUs for running models, CPUs for management services


• Networking and DNS : Routing traffic to the appropriate

... See more

To do this, we employ a technique known as AI-assisted evaluation, alongside traditional metrics for measuring performance. This helps us pick the prompts that lead to better quality outputs, making the end product more appealing to users. AI-assisted evaluation uses best-in-class LLMs (like GPT-4) to automatically critique how well the AI's... See more

no reason to build any kind of software product these days that doesn't have a significant UX/domain knowledge component

• Multiple indices. Splitting the document corpus up into multiple indices and then routing queries based on some criteria. This means that the search is over a much smaller set of documents rather than the entire dataset. Again, it is not always useful, but it can be helpful for certain datasets. The same approach works with the LLMs themselves.

... See more