GitHub - Portkey-AI/gateway: A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API.
Higher performance and lower cost than any single LLM
We invented the first LLM router. By dynamically routing between multiple models, Martian can beat GPT-4 on performance, reduce costs by 20%-97%, and simplify the process of using AI
We invented the first LLM router. By dynamically routing between multiple models, Martian can beat GPT-4 on performance, reduce costs by 20%-97%, and simplify the process of using AI
Martian's Model Router: Optimize AI Performance and Reduce Costs
Nicolay Gerold added
GitHub - AI4Finance-Foundation/FinRobot: FinRobot: An Open-Source AI Agent Platform for Financial Applications using LLMs 🚀 🚀 🚀
GitHub - AI4Finance-Foundation/FinRobot: FinRobot: An Open-Source AI Agent Platform for Financial Applications using LLMs 🚀 🚀 🚀
Steve Werber added
📦 Service Deployment - Ray Serve (https://lnkd.in/eAV-Y6RN)
🧰 Data Transformation - Ray Data (https://lnkd.in/e7wYmenc)
🔌 LLM Integration - AIConfig (https://lnkd.in/esvH5NQa)
🗄 Vector Database - Weaviate (https://weaviate.io/)
📚 Supervised LLM Fine-Tuning - HuggingFace TLR (https://lnkd.in/e8_QYF-P)
📈 LLM Observability - Weights & Biases Tra... See more
🧰 Data Transformation - Ray Data (https://lnkd.in/e7wYmenc)
🔌 LLM Integration - AIConfig (https://lnkd.in/esvH5NQa)
🗄 Vector Database - Weaviate (https://weaviate.io/)
📚 Supervised LLM Fine-Tuning - HuggingFace TLR (https://lnkd.in/e8_QYF-P)
📈 LLM Observability - Weights & Biases Tra... See more
Feed | LinkedIn
LLM-PowerHouse: A Curated Guide for Large Language Models with Custom Training and Inferencing
Welcome to LLM-PowerHouse, your ultimate resource for unleashing the full potential of Large Language Models (LLMs) with custom training and inferencing. This GitHub repository is a comprehensive and curated guide designed to empower developers, researche... See more
Welcome to LLM-PowerHouse, your ultimate resource for unleashing the full potential of Large Language Models (LLMs) with custom training and inferencing. This GitHub repository is a comprehensive and curated guide designed to empower developers, researche... See more
ghimiresunil • GitHub - ghimiresunil/LLM-PowerHouse-A-Curated-Guide-for-Large-Language-Models-with-Custom-Training-and-Inferencing: LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.
Nicolay Gerold added
Full stack framework for building cross-platform mobile AI apps supporting LLM real-time / streaming text and chat UIs, image services and natural language to images with multiple models, and image processing.
Features
Features
- LLM support for OpenAI ChatGPT, Anthropic Claude, Cohere and Cohere Web
- An array of image models provided by Fal.ai
- Real-time / stream
dabit3 • GitHub - dabit3/react-native-ai: Full stack framework for building cross-platform mobile AI apps
Nicolay Gerold added
Lightning-fast OSS ModelsTry our APIExperience the world's fastest LLM inference platform. Use a state-of-the-art, open-source model or fine-tune and deploy your own at no additional cost, with Fireworks.ai.Query Large Language Models with Fireworks.aiIt is super easy to get started with Fireworks.ai LLM API. You can use the Python client (install ... See more
Fireworks Console
Nicolay Gerold added
The human-centric platform for production ML & AI
Access data easily, scale compute cost-efficiently, and ship to production confidently with fully managed infrastructure, running securely in your cloud.
Access data easily, scale compute cost-efficiently, and ship to production confidently with fully managed infrastructure, running securely in your cloud.
Infrastructure for ML, AI, and Data Science | Outerbounds
Nicolay Gerold added
How to Host Powerful AI Models in the Cloud using Groq Cloud & OpenWebUI
youtube.com# Key Information Summary
## Query on Hosting Large Language Models
- Context: Many users are interested in hosting large language models locally without having access to a powerful GPU or with limited computing resources.
## Overview of Grock
- Grock GQ: A relevant option called Grock is highlighted for hosting large language models remotely.
- Functionality: Grock provides access to large models through an API, allowing users to leverage cloud resources.
- User Action: Users must sign up at grockcloud.com, create an API key, and use it within their applications like Open Web UI.
## Model Examples
- Llama 3 Model: Example given is Llama 3 with 70 billion parameters, which is too large for personal hosting capabilities (e.g., only able to host Llama with 34 billion parameters on an Nvidia 4090 GPU).
- Performance:
- Inference time for the Llama 3 model is 888 milliseconds.
- Tokens processed per second: 311.
## Setup Instructions
1. Sign Up: Sign in to Grock Cloud.
2. Create API Key: After account setup, generate an API key.
3. Integration:
- Paste the API key in the Open Web UI interface under the admin panel settings.
- Verify the connection to ensure successful integration.
## Cost and Accessibility
- Free vs Paid:
- A free version of Grock is available, but usage is limited.
- Additional usage requires a paid subscription, which is described as reasonably priced.
## Engagement Call
- User Interaction: Viewers are encouraged to leave comments or suggestions for future content.
## Additional Resources
- Information about pricing plans will be shared through links in the description.
- References to an Open Web UI playlist for further guidance are also provided.