大模型时代的多模态融合：方法、评测与前沿挑战综述

Multimodal Fusion in the Era of Large Models: Methods, Evaluation, and Frontier Challenges

摘要: 目的：系统梳理大模型时代多模态融合的方法演进与评测难点，说明如何在提高能力的同时，让实验结果可重复、评测过程可核查。方法：建立对齐、桥接、深度交互与统一建模四层分析框架，对照代表模型路线、训练数据的组织方式以及分阶段训练策略，并提出MMP-Next评测草案，包括模型与数据说明清单、评测运行配置表、推理过程是否稳定等指标，以及鲁棒性与安全方面的最小测试集合。结果：多模态大模型在通用理解、任务迁移和日常交互上进步明显，但长文本下的信息压缩损失、多模态幻觉、解码与提示词带来的分数波动，以及换场景或遇对抗样本时的安全与稳定性不足，在短期内仍难以单靠扩大模型规模根除。结论：后续研究宜在扩大参数之外，同步改进融合方式与评测规范，推动能对照证据的推理方式，以及有统一格式、可复核的评测流程。

Abstract: Objective: To review methodological trends and evaluation difficulties of multimodal fusion in the foundation-model era, and to clarify how stronger capability can coexist with reproducible experiments and auditable evaluation. Methods: We build a four-layer framework spanning alignment, bridging, deep interaction, and unified modeling; we relate representative model routes to how training data are organized and to staged training strategies, and we outline the MMP-Next evaluation draft, including model/data disclosure checklists, evaluation run sheets, indicators of whether inference is stable across settings, and a minimal test set for robustness and safety. Results: Multimodal large models improve markedly in general understanding, task transfer, and everyday interactive use, yet information loss under long-context compression, multimodal hallucination, score volatility from decoding and prompting, and gaps in safety and stability under domain shift or adversarial conditions are unlikely to be removed in the short term by parameter scaling alone. Conclusion: Beyond enlarging model size, research should jointly refine fusion mechanisms and evaluation norms, advancing evidence-grounded reasoning together with standardized, reviewable evaluation workflows.