r/MachineLearning - Reddit

The Mac Studio is an absolute monster for inferencing; but there are a couple of caveats.

• Its slower, pound for pound, than a 4090 when dealing with models the 4090 can fit in its VRAM. So a 13b model on the 4090 is almost twice as fast as it running on the M2.


• The M1 Ultra Mac Studio with 128GB costs far less ($3700 or so) and the inference speed is

... See more

M3 max is objectively worse than the M2 for inference.

The M2 ultra has a higher max RAM size of 192 GB

The M1 ultra has 128 GB max ram.

When it comes to these ram numbers something like 2/3 of it is available for inference.

So I see no reason why not to make a general recommendation for the M1 ultra unless you have some reason you want to run q5_K_M 1... See more

转：关于 DeepSeek 的研究和思考 (Archerman Capital) 关于这几天很火的 DeepSeek, 我们 (Archerman Capital) 做了一些研究和思考, 和大家分享, enjoy! 灰色部分是技术细节, 不感兴趣的可略过。 几个事实 1) DeepSeek 不是套壳不是蒸馏美国的大模型。 虽然中国有些大模型是套壳和蒸馏的, 但 DeepSeek 不是。 2) 核心架构还是基于 Transformer, deepseek 在架构、工程设计上进行了创新和工艺提升, 实现效率优化。架构上, 采用了混合专家模型 (MoE)、多头潜注意力 (MLA)、多令牌预测 (MTP)、长链式推理 (CoT)、DualPipe 算法等设计, 并进行了依赖强化学习 (RL)... See more

However development time, and maintenance can offset these savings. Hiring skilled data scientists, machine learning engineers, and DevOps professionals can be expensive and time consuming. Using available resources for “reimplementing” solutions hinder innovation and lead to a lack of focus. Since You not longer work on improving your model or pro... See more