FOD#27: "Now And Then"

Phi-1.5

Phi-1.5 is a "small" 1.3 billion parameter LLM with an impressive performance for its size.

Annotated figures from the Textbooks Is All You Need II paper

How does this small model accomplish such a good performance? The secret ingredient seems to be the high-quality data.

The pretraining is based on the Textbooks Is All You Need approach that... See more

LLaVA v1.5, a new open-source multimodal model stepping onto the scene as a contender against GPT-4 with multimodal capabilities. It uses a simple projection matrix to connect the pre-trained CLIP ViT-L/14 vision encoder with Vicuna LLM, resulting in a robust model that can handle images and text. The model is trained in two stages: first, updated ... See more