Nicolay Gerold

TL;DR

We are thrilled to announce the release of the FASTEST Voice LLM to date! Experience real-time speech streaming from text in 300ms or less. Dive in and test it using our Playground, available SDKs, or these Replit demos for both Nodejs and Python and a chatGPT integration.

Introduction

At PlayHT, our vision revolves around redefining human... See more

nanosearch

Nanosearch is an in-memory search engine designed for small (< 10,000 URL) websites.

With Nanosearch, you can build a search engine in a few lines of code.

Nanosearch supports the BM25 and TF/IDF algorithms.

Nanosearch also computes a link graph and uses the number of inlinks to a page as a ranking factor. This is useful for ranking... See more

Adrenaline protects you from infections and activated neuroplasticity in the brain. When you sense a stressor (wound, threat) the sympathetic chain ganglia and adrenals (body) and the locus cerulius (brain) release adrenaline all over the body.

• ClickHouse: It's a high performance columnar database that's great for real time queries. It enables querying and storing large amounts of data on commodity hardware. Some of my customers have millions of page views and I don't have an unlimited budget, so it's been very handy.


• PostgreSQL: My favorite database. Sane defaults, battle-tested, and well

... See more

If you are ever having trouble coming up with a name, do the following:

1. Write a comment above the function/variable where you describe what it is, in human language , as if you were describing it to your colleague. It might be one sentence or multiple sentences. This is the essence of what your function/variable does, what it is.


2. Now, you take the

... See more

Supported Models

Suggest Edits

Where possible, we try to match the Hugging Face implementation. We are open to adjusting the API, so please reach out with feedback regarding these details.

Model

Context Length

Model Type

codellama-34b-instruct

16384

Chat Completion

llama-2-70b-chat

4096

Chat Completion

mistral-7b-instruct

4096 [1]

Chat Completion

pplx-7b-c... See more

Setting up the necessary machine learning infrastructure to run these big models is another challenge. We need a dedicated model server for running model inference (using frameworks like Triton oder vLLM), powerful GPUs to run everything robustly, and configurability in our servers to make sure they're high throughput and low latency. Tuning the... See more

We consider these aspects of our problem:

• Latency : How fast does the system need to respond to user input?


• Task Complexity : What level of understanding is required from the LLM? Is the input context and prompt super domain-specific?


• Prompt Length : How much context needs to be provided for the LLM to do its task?


• Quality : What is the acceptable

... See more

Hono - [炎] means flame🔥 in Japanese - is a small, simple, and ultrafast web framework for the Edges. It works on any JavaScript runtime: Cloudflare Workers, Fastly Compute@Edge, Deno, Bun, Vercel, Lagon, AWS Lambda, Lambda@Edge, and Node.js.

Fast, but not only fast.

import { Hono } from 'hono'

const app = new Hono()

app.get('/', (c) =>... See more