Background
The connector sits between the visual encoder and the language model. Its structure determines how visual tokens are transformed, preserved, or compressed before they reach the language model.
Motivation
Connector designs vary widely, but their effects are often discussed as secondary implementation details. This project studies connector selection as a central design decision in multimodal language models.
What We Study
- How connector structures control visual information flow.
- The tradeoff between preserving visual tokens and compressing them.
- Design choices that influence efficiency and multimodal performance.
Personal Notes
I will use this section to record how my understanding of connector design developed during this project.