时序证据双门控青花瓷风格迁移
Time-Evidence Dual Gating for Blue-and-White Porcelain Style Transfer
摘要: LoRA微调技术已广泛应用于稳定扩散(Stable Diffusion)文生图模型,但在青花瓷高精度图像生成中,难以同时保证纹饰风格学习与器型结构完整性,常出现形变。本文提出一种适用于Transformer架构、无需额外训练的LoRA风格迁移方法:将LoRA注入过程解耦为时间门控与提示词证据门控,并通过乘法融合动态调节风格强度——在结构敏感阶段抑制扰动,在细节生成阶段增强纹饰表现。实验证明,相比经开发集调参后的恒定强度对比,LPIPS相对下降52.1%,SSIM相对提升31.1%,MS-SSIM、PSNR与DISTS亦呈一致改善,有效实现了结构保持下的高质量风格迁移。
Abstract: LoRA (Low-Rank Adaptation) fine-tuning has been widely adopted in Stable Diffusion for text-to-image generation. However, when applied to high-fidelity image synthesis of blue-and-white porcelain, it often struggles to simultaneously preserve structural integrity of vessel forms and accurately capture intricate decorative styles, frequently resulting in geometric distortions. To address this challenge, we propose a novel LoRA-based style transfer method compatible with Transformer architectures that requires no additional training. Our approach decouples the LoRA injection process into two dynamic gating mechanisms: a time-aware gate and a prompt-evidence gate. These gates are multiplicatively fused to adaptively modulate style intensity—suppressing stylistic perturbations during structure-sensitive denoising steps while enhancing decorative detail rendering in later stages. Experimental results demonstrate significant improvements over a baseline using constant, development-set-tuned LoRA strength: LPIPS decreases by 52.1%, SSIM increases by 31.1%, and consistent gains are observed across MS-SSIM, PSNR, and DISTS metrics. This confirms the effectiveness of our method in achieving high-quality style transfer while faithfully preserving structural fidelity.
文章引用:徐子墨, 李嘉辉. 时序证据双门控青花瓷风格迁移[J]. 建模与仿真, 2026, 15(5): 207-216. https://doi.org/10.12677/mos.2026.155084

参考文献

[1] Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., et al. (2024) Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arXiv: 2403.03206.
https://arxiv.org/abs/2403.03206
[2] Peebles, W. and Xie, S. (2023) Scalable Diffusion Models with Transformers. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, 1-6 October 2023, 4172-4181. [Google Scholar] [CrossRef
[3] Liu, X., Gong, C. and Liu, Q. (2023) Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. International Conference on Learning Representations (ICLR) 2023, Kigali, 1-5 May 2023, 1-33.
[4] Lin, S., Liu, B., Li, J. and Yang, X. (2024) Common Diffusion Noise Schedules and Sample Steps Are Flawed. 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, 3-8 January 2024, 5392-5399. [Google Scholar] [CrossRef
[5] Karras, T., Aittala, M., Aila, T. and Laine, S. (2022) Elucidating the Design Space of Diffusion-Based Generative Models. arXiv: 2206.00364.
[6] Zhang, L., Rao, A. and Agrawala, M. (2023) Adding Conditional Control to Text-To-Image Diffusion Models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, 1-6 October 2023, 3813-3824. [Google Scholar] [CrossRef
[7] Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., et al. (2024) T2I-Adapter: Learning Adapters to Dig Out More Controllable Ability for Text-To-Image Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38, 4296-4304. [Google Scholar] [CrossRef
[8] Ye, H., Zhang, J., Liu, S., Han, X. and Yang, W. (2023) IP-Adapter: Text Compatible Image Prompt Adapter for Text-To-Image Diffusion Models. arXiv: 2308.06721.
[9] Wang, Z., Bovik, A.C., Sheikh, H.R. and Simoncelli, E.P. (2004) Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 13, 600-612. [Google Scholar] [CrossRef] [PubMed]
[10] Wang, Z., Simoncelli, E.P. and Bovik, A.C. (2003) Multiscale Structural Similarity for Image Quality Assessment. The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, Pacific Grove, 9-12 November 2003, 1398-1402. [Google Scholar] [CrossRef
[11] Zhou Wang, and Bovik, A.C. (2009) Mean Squared Error: Love It or Leave It? A New Look at Signal Fidelity Measures. IEEE Signal Processing Magazine, 26, 98-117. [Google Scholar] [CrossRef
[12] Zhang, R., Isola, P., Efros, A.A., Shechtman, E. and Wang, O. (2018) The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, 18-23 June 2018, 586-595. [Google Scholar] [CrossRef
[13] Ding, K., Ma, K., Wang, S. and Simoncelli, E.P. (2020) Image Quality Assessment: Unifying Structure and Texture Similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 2567-2581. [Google Scholar] [CrossRef] [PubMed]
[14] Radford, A., Kim, J. W., Hallacy, C., et al. (2021) Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the 38th International Conference on Machine Learning (ICML), 18-24 July 2021, 8748-8763.