On-Premise or Cloud - Where Should You Host Your AI Applications?
Recently, data collection has shifted to JavaScript-based solutions and therefore data collection, storage, and processing moved into the cloud (hosted by your application service provider rather than in-house). This has eliminated the need to maintain IT teams for web analytics, except perhaps to implement tags and other required code on web pages
... See moreAvinash Kaushik • Web Analytics 2.0: The Art of Online Accountability and Science of Customer Centricity
Unlike consumers, enterprises want control over how their data is used and shared with companies, including the providers of AI software. Enterprises have spent a lot effort in consolidating data from different sources and bringing them in-house (this article Partner integrations + System of Intelligence: Today’s deepest Moat by fellow Medium autho... See more
AI Startup Trends: Insights from Y Combinator’s Latest Batch
AI systems must, therefore, have the capability to process and prioritize data locally, sending back only the most relevant information.
Samuel Thorpe • The Essential Beginner’s Guide to AI
A New Cohort of Agentic Applications
With Generative AI’s budding reasoning capabilities, a new class of agentic applications is starting to emerge.
What shape do these application layer companies take? Interestingly, these companies look different than their cloud predecessors:
With Generative AI’s budding reasoning capabilities, a new class of agentic applications is starting to emerge.
What shape do these application layer companies take? Interestingly, these companies look different than their cloud predecessors:
- Cloud companies targeted the software profit pool. AI companies target
Sonya Huang • Generative AI’s Act O1
For example they have lower gross margins. Anecdotally, we have seen gross margins of ML companies at around 50/60% vs 60–80% of comparable SaaS businesses. This is due to two factors. Firstly, the hidden cost of the cloud infrastructure. Training a single ML model costs thousands of dollars in computing resources. Secondly, many applications requi... See more
Matilde Giglio • What we look for in AI companies (and why they are different)
Top considerations when choosing foundation models
Accuracy
Cost
Latency
Privacy
Top challenges when deploying production AI
Serving cost
Evaluation
Infra reliability
Model quality
Accuracy
Cost
Latency
Privacy
Top challenges when deploying production AI
Serving cost
Evaluation
Infra reliability
Model quality
Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.
A solution is to self-host an open-sourced or custom fine-tuned LLM. Opting for a self-hosted model can reduce costs dramatically - but with additional development time, maintenance overhead, and possible performance implications. Considering self-hosted solutions requires weighing these different trade-offs carefully.
Developing Rapidly with Generative AI
Cloud-based hosting is different from traditional hosting models in that you pay for your hosting based on the resources you use, rather than paying for a fixed hardware resource and monthly allowance of space and bandwidth.