Sublime
An inspiration engine for ideas
Dense Discovery – Issue 305
densediscovery.com
转：关于 DeepSeek 的研究和思考 (Archerman Capital)
关于这几天很火的 DeepSeek, 我们 (Archerman Capital) 做了一些研究和思考, 和大家分享, enjoy! 灰色部分是技术细节, 不感兴趣的可略过。
几个事实
1) DeepSeek 不是套壳不是蒸馏美国的大模型。 虽然中国有些大模型是套壳和蒸馏的, 但 DeepSeek 不是。
2) 核心架构还是基于 Transformer, deepseek 在架构、工程设计上进行了创新和工艺提升, 实现效率优化。架构上, 采用了混合专家模型 (MoE)、多头潜注意力 (MLA)、多令牌预测 (MTP)、长链式推理 (CoT)、DualPipe 算法等设计, 并进行了依赖强化学习 (RL)... See more
SELF-DISCOVER
This implements Google's algorithm from https://arxiv.org/pdf/2402.03620.pdf
Setup
Install Conda: https://docs.conda.io/projects/miniconda/en/latest/
This implements Google's algorithm from https://arxiv.org/pdf/2402.03620.pdf
Setup
Install Conda: https://docs.conda.io/projects/miniconda/en/latest/
catid • GitHub - catid/self-discover: Implementation of Google's SELF-DISCOVER
FieldGuide_Digital_B&W
An engaging exploration of futures thinking, blending creativity, sustainability, and community. Reflect on multi-layered futures, mundane magic, and plant better futures with a mix of playful activities and thoughtful insights.
Link
