基于扩散Transformer的复杂图像加速推理改进研究

Research on Improved Complex Image Acceleration Inference Based on Diffusion Transformer

摘要: 文生图模型的相关研究发展迅速，使用扩散模型的Transformer架构更是其中的主流。面对当今文生图模型在复杂图像处理和提高推理速度的瓶颈时，本文参考扩散Transformer提出了一种基于图像空间复杂度的自适应生成策略。本文设计了一种结合纹理和结构信息的图像空间复杂度计算方法，并将其作为生成模型的输入，动态分析生成图像的复杂结构。基于该复杂度指标，进一步提出了分块自适应生成策略，依据子图像空间复杂度的高低来调节推理模型深度的深浅，从而在保证图像质量的同时，显著提升生成效率。我们的模型在ImageNet 256 × 256和MSCOCO 2017数据集上进行了实验验证，其在FID、sFID和IS指标上均优于主流图像生成模型，同时推理速度明显加快。生成的结果显示，其不仅能复原各种潜在的细节信息，也能有效提高模型迭代速度，证明了该方法在复杂图像生成与加速推理上的有效性和可行性。

Abstract: Research on image-generating models has advanced rapidly, with the Transformer architecture using diffusion models becoming the mainstream. Addressing the bottlenecks faced by current image-generating models in processing complex images and improving inference speed, this paper proposes an adaptive generation strategy based on image spatial complexity, drawing inspiration from the diffusion Transformer. We devise a method to calculate image spatial complexity by incorporating texture and structural information, using this as input to a generative model to dynamically analyze the complex structure of the generated image. Based on this complexity metric, we further propose a block-wise adaptive generation strategy that adjusts the depth of the inference model based on the spatial complexity of the sub-images, significantly improving generation efficiency while maintaining image quality. Our model is experimentally validated on the ImageNet 256 × 256 and MSCOCO 2017 datasets, outperforming mainstream image generation models in terms of FID, sFID, and IS metrics, while significantly accelerating inference speed. The generated results demonstrate that it not only recovers a wide range of underlying details, but also effectively improves model iteration speed, demonstrating the effectiveness and feasibility of this approach for complex image generation and accelerated inference.