Public KG record
Hybrid Attention
Recorded overview
Alternates sliding window and full attention for local/global context.
An evidence-backed entity from the project knowledge graph; detailed tool metadata is still being added.
Recorded details
- Integrations: Delta Attention
- Integrations: Heavily Compressed Attention
- Performance: 匹配计算量下支持约4倍更长序列
- Performance: 在匹配计算量下支持比全注意力模型长4倍的序列
Evidence coverage
The KG currently records 8 known information dimensions and has 11 undocumented dimensions, linked to 6 public source articles. These counts describe record coverage, not model performance or verified fit.
Related public records
- Delta Attention — integrates
- Heavily Compressed Attention — integrates
- MiMo-V2.6-Distill-Qwen-9B — part of
- DeepSeek-V4 — uses
- Inkling-Small — uses