Above and below the LLMs