Prompting
by Nicolay Gerold · updated 5mo ago
Prompting
by Nicolay Gerold · updated 5mo ago
Nicolay Gerold added 8mo ago
Nicolay Gerold added 1y ago
Nicolay Gerold added 1y ago
Nicolay Gerold added 1y ago
Nicolay Gerold added 5mo ago
Nicolay Gerold added 8mo ago
They have a fast jsond ecoding feature with a finite state machine.
Intuition : Prompt tokens add very little latency to completion calls. Time to generate completion tokens is much longer, as tokens are generated one at a time. Longer generation lengths will accumulate latency due to generation required for each token.
Nicolay Gerold added 1y ago
Nicolay Gerold added 8mo ago
Nicolay Gerold added 8mo ago