Introduction

Mamba的横空出世打破了Transformer在深度学习作为主要backbone的局面，给研究者提供了一种新的思路，他的线性复杂度和处理超长序列信息的能力正是它能够挑战Transformer的底气。在序列模型中，例如语音、文本、DNA序列等大多数信号都是因果信号，而Mamba又是基于RNN的，因此这种因果序列信息对于Mamba这个模型天生地更加友好。

然而，在深度学习任务中，还有一大部分的信号都是图像信号，而图像信号天生属于非因果信号，如果直接采用Mamba这种因果网络来处理图像信息，那么将无法避免的出现网络的图像信息强行加入因果特性，例如先输入网络的图像块会随着信息的输入而被渐渐遗忘，使得图像某个部分的信号被人为的认为更加重要（实际上它们应该是同等重要的），即使是LSTM、Mamba也只是减缓了这种遗忘速度。

目前大多数将Mamba应用于图像处理的网络大多数都是采用多向扫描的机制来减缓网络因果性对于图像信号的影响，这种做法可以应用于不太需要关注较长距离的以来的任务，例如分割、分类。然而，一方面这种多向扫描虽然能够缓解因果性但是依然存在一定的非因果性，例如从四周往中间的扫描方式更加关注中间的像素，另一方面对于图像压缩这种需要对整个图像全局关注的任务，因果网络这种特天生不匹配。

另一种是加窗口，在窗口内认为是时不变的，

能否添加一个参数来学习图像的重要性

Attention机制这种天生具有非因果特性的网络和图像中的要求十分匹配，然而它对于大图像具有二次复杂度，这又使得它无法处理超大的图像，这种矛盾能够通过借鉴两者的优势得到一个结合的网络，既具有非因果特性又具有线性的复杂度？

Some tutorial:

CUDA学习第18课：Blelloch扫描算法以及如何选择扫描算法_哔哩哔哩_bilibili

【BBuf的CUDA笔记】十五，OpenAI Triton入门笔记三 FusedAttention - GiantPandaCV

State Space Model (SSM)

(NeurIPS 2020 Spotlight) HiPPO: Recurrent Memory with Optimal Polynomial Projections Paper Code

(ICLR 2022) S4: Efficiently Modeling Long Sequences with Structured State Spaces Paper Code

(ICLR 2023) H3: Hungry Hungry Hippos: Toward Language Modeling with State Space Models Paper Code

(Arxiv 24.05.26) A Unified Implicit Attention Formulation for Gated-Linear Recurrent Sequence Models Paper Code

(Arxiv 24.05.27) The Expressive Capacity of State Space Models: A Formal Language Perspective Paper Code

(Arxiv 24.05.31, ICML24, Mamba-2) Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality Paper Code

(Arxiv 24.06.04) GrootVL: Tree Topology is All You Need in State Space Model Paper Code

(Arxiv 24.06.12) An Empirical Study of Mamba-based Language Models Paper

Mamba

(Arxiv 23.12.01) Mamba: Linear-Time Sequence Modeling with Selective State Spaces Paper Code

(Arxiv 24.01.08) MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts Paper

(Arxiv 24.01.24) MambaByte: Token-free Selective State Space Model Paper Code

(Arxiv 24.01.31) LOCOST: State-Space Models for Long Document Abstractive Summarization Paper Code

(Arxiv 24.02.01) BlackMamba: Mixture of Experts for State-Space Models Paper Code

(Arxiv 24.02.06) Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks Paper

(Arxiv 24.02.08) Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data Paper

(Arxiv 24.02.15) Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling Paper Code

(Arxiv 24.02.19) Pan-Mamba: Effective pan-sharpening with State Space Model Paper Code

(CVPR24) State Space Models for Event Cameras Paper Code

(Arxiv 24.02.26) DenseMamba: State Space Models with Dense Hidden Connection for Efficient Large Language Models Paper Code

(Arxiv 24.03.03) The Hidden Attention of Mamba Models Paper Code

(Arxiv 24.03.08) MamMIL: Multiple Instance Learning for Whole Slide Images with State Space Models Paper

(Arxiv 24.03.11) MambaMIL: Enhancing Long Sequence Modeling with Sequence Reordering in Computational Pathology Paper Code

(Arxiv 24.03.12) Motion Mamba: Efficient and Long Sequence Motion Generation with Hierarchical and Bidirectional Selective SSM Paper Code

(Arxiv 24.03.13) ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notes Paper

(Arxiv 24.03.26) State Space Models as Foundation Models: A Control Theoretic Overview Paper

(Arxiv 24.03.28) Jamba: A Hybrid Transformer-Mamba Language Model Paper Code

(Arxiv 24.03.29) HARMamba: Efficient Wearable Sensor Human Activity Recognition Based on Bidirectional Selective SSM Paper

(Arxiv 24.04.07) VMambaMorph: a Visual Mamba-based Framework with Cross-Scan Module for Deformable 3D Image Registration Paper Code

(Arxiv 24.04.12) SpectralMamba: Efficient Mamba for Hyperspectral Image Classification Paper Code

(Arxiv 24.05.13) MambaOut: Do We Really Need Mamba for Vision? Paper Code

(Arxiv 24.05.19) NetMamba: Efficient Network Traffic Classification via Pre-training Unidirectional Mamba Paper

(Arxiv 24.05.23) EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records Paper

(Arxiv 24.05.26) Mamba4KT:An Efficient and Effective Mamba-based Knowledge Tracing Model Paper

(Arxiv 24.05.26) Zamba: A Compact 7B SSM Hybrid Model Paper

(Arxiv 24.05.30) MSSC-BiMamba: Multimodal Sleep Stage Classification and Early Diagnosis of Sleep Disorders with Bidirectional Mamba Paper

Vision

(Arxiv 24.01.17) Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model Paper Code

(Arxiv 24.01.18) VMamba: Visual State Space Model Paper Code

(Arxiv 24.02.05) Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining Paper Code

(Arxiv 24.02.06) U-shaped Vision Mamba for Single Image Dehazing Paper Code

(Arxiv 24.02.23) MambaIR: A Simple Baseline for Image Restoration with State-Space Model Paper Code

(Arxiv 24.02.24) Res-VMamba: Fine-Grained Food Category Visual Classification Using Selective State Space Models with Deep Residual Learning Paper Code

(Arxiv 24.03.04) MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection Paper Code

(Arxiv 24.03.15) EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba Paper Code

(Arxiv 24.03.15) On the low-shot transferability of [V]-Mamba Paper

(Arxiv 24.03.26) PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition Paper Code

(Arxiv 24.03.26) Integrating Mamba Sequence Model and Hierarchical Upsampling Network for Accurate Semantic Segmentation of Multiple Sclerosis Legion Paper

(Arxiv 24.03.27) Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction Paper

(Arxiv 24.03.27) ReMamber: Referring Image Segmentation with Mamba Twister Paper

(Arxiv 24.03.28) RSMamba: Remote Sensing Image Classification with State Space Model Paper Code

(Arxiv 24.04.02) Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model Paper Code

(Arxiv 24.04.03) RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation Paper Code

(Arxiv 24.04.03) RS-Mamba for Large Remote Sensing Image Dense Prediction Paper Code

(Arxiv 24.04.04) ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model Paper Code

(Arxiv 24.04.05) Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation Paper Code

(Arxiv 24.04.09) MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection Paper Code

(Arxiv 24.04.15) FreqMamba: Viewing Mamba from a Frequency Perspective for Image Deraining Paper

(Arxiv 24.04.16) Exploring Learning-based Motion Models in Multi-Object Tracking Paper

(Arxiv 24.04.17) CU-Mamba: Selective State Space Models with Channel Learning for Image Restoration Paper

(Arxiv 24.04.17) Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion Paper

(Arxiv 24.04.20) Vim4Path: Self-Supervised Vision Mamba for Histopathology Images Paper Code

(Arxiv 24.04.28) Mamba-FETrack: Frame-Event Tracking via State Space Model Paper Code

(Arxiv 24.04.28) S2Mamba: A Spatial-spectral State Space Model for Hyperspectral Image Classification Paper Code

(Arxiv 24.04.29) Spectral-Spatial Mamba for Hyperspectral Image Classification Paper

(Arxiv 24.04.29) RSCaMa: Remote Sensing Image Change Captioning with State Space Model Paper Code

(Arxiv 24.05.02) SOAR: Advancements in Small Body Object Detection for Aerial Imagery Using State Space Models and Programmable Gradients Paper Code

(Arxiv 24.05.02) SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising Paper Code

(Arxiv 24.05.05) DVMSR: Distillated Vision Mamba for Efficient Super-Resolution Paper Code

(Arxiv 24.05.05) AC-MAMBASEG: An adaptive convolution and Mamba-based architecture for enhanced skin lesion segmentation Paper Code

(Arxiv 24.05.06) Retinexmamba: Retinex-based Mamba for Low-light Image Enhancement Paper Code

(Arxiv 24.05.07) VMambaCC: A Visual State Space Model for Crowd Counting Paper

(Arxiv 24.05.08) Frequency-Assisted Mamba for Remote Sensing Image Super-Resolution Paper

(Arxiv 24.05.09) Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba Paper

(Arxiv 24.05.13) GMSR:Gradient-Guided Mamba for Spectral Reconstruction from RGB Images Paper

(Arxiv 24.05.13) OverlapMamba: Novel Shift State Space Model for LiDAR-based Place Recognition Paper

(Arxiv 24.05.14) Rethinking Scanning Strategies with Vision Mamba in Semantic Segmentation of Remote Sensing Imagery: An Experimental Study Paper

(Arxiv 24.05.16) IRSRMamba: Infrared Image Super-Resolution via Mamba-based Wavelet Transform Feature Modulation Model Paper Code

(Arxiv 24.05.16) RSDehamba: Lightweight Vision Mamba for Remote Sensing Satellite Image Dehazing Paper

(Arxiv 24.05.17) CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation Paper Code

(Arxiv 24.05.16) IRSRMamba: Infrared Image Super-Resolution via Mamba-based Wavelet Transform Feature Modulation Model Paper Code

(Arxiv 24.05.20) Mamba-in-Mamba: Centralized Mamba-Cross-Scan in Tokenized Mamba Model for Hyperspectral Image Classification Paper Code

(Arxiv 24.05.21) 3DSS-Mamba: 3D-Spectral-Spatial Mamba for Hyperspectral Image Classification Paper

(Arxiv 24.05.23) Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model Paper Code

(Arxiv 24.05.23) Scalable Visual State Space Model with Fractal Scanning Paper

(Arxiv 24.05.23) Mamba-R: Vision Mamba ALSO Needs Registers Paper Code

(Arxiv 24.05.24) MUCM-Net: A Mamba Powered UCM-Net for Skin Lesion Segmentation Paper

(Arxiv 24.05.26) Demystify Mamba in Vision: A Linear Attention Perspective Paper Code

(Arxiv 24.05.29) Vim-F: Visual State Space Model Benefiting from Learning in the Frequency Domain Paper Code

(Arxiv 24.05.29) FourierMamba: Fourier Learning Integration with State Space Models for Image Deraining Paper

(Arxiv 24.06.06) MambaDepth: Enhancing Long-range Dependency for Self-Supervised Fine-Structured Monocular Depth Estimation Paper

(Arxiv 24.06.09) HDMba: Hyperspectral Remote Sensing Imagery Dehazing with State Space Model Paper

(Arxiv 24.06.09) Mamba YOLO: SSMs-Based YOLO For Object Detection Paper

(Arxiv 24.06.09) MHS-VM: Multi-Head Scanning in Parallel Subspaces for Vision Mamba Paper Code

(Arxiv 24.06.11) DualMamba: A Lightweight Spectral-Spatial Mamba-Convolution Network for Hyperspectral Image Classification Paper

(Arxiv 24.06.11) Autoregressive Pretraining with Mamba in Vision Paper Code