| 引用本文: | 张振兴,杨任农,俞利新,王瑛,李永林,郭安新.基于词级注意力的多样化文本到图像生成[J].哈尔滨工业大学学报,2026,58(9):47.DOI:10.11918/202412036 |
| Zhang Zhenxing,Yang Rennong,Yu Lixin,Wang Ying,Li Yonglin,Guo Anxin.Diverse text-to-image synthesis based on word-level attention[J].Journal of Harbin Institute of Technology,2026,58(9):47.DOI:10.11918/202412036 |
|
| 摘要: |
| 为解决单阶段文本到图像生成技术存在的生成结果单一、精确调控困难等局限性,提出一种基于词级注意力的多样化文本到图像生成对抗网络框架(diverse text-to-image generative adversarial network framework based on word-level attention,WordGAN)。该框架在条件生成对抗网络的基础上,引入双重注意力机制、双残差连接设计和线性层策略。其中,双重注意力机制由条件自适应实例层归一化、通道注意力模块和像素注意力模块组成,通过量化分析文本描述中各词汇的语义贡献度捕捉网络特征通道及空间像素的权重分布;双残差连接设计用于维持底层视觉特征完整性和支持更深层次的网络扩展;线性层策略利用线性层改变原有图像模式,促进框架探索更多次要模式。在CUB bird、MSCOCO和Oxford-102基准数据集上进行一系列实验,结果表明:WordGAN有效地实现了生成图像的多样性,而且不会损害质量和语义一致性,其性能水平优于所对比的文本到图像生成框架。本文提出的基于词级注意力的多样化框架为单阶段文本到图像生成任务研究提供了新的思路,具有实际应用价值。 |
| 关键词: 词级注意力机制 双残差网络 模式坍塌 单阶段框架 文本到图像生成 |
| DOI:10.11918/202412036 |
| 分类号:V247 |
| 文献标识码:A |
| 基金项目:西安市青年人才托举计划(0959202513098);国家自然科学基金(4,4); 陕西省自然科学基金(2021JQ370) |
|
| Diverse text-to-image synthesis based on word-level attention |
|
Zhang Zhenxing,Yang Rennong,Yu Lixin,Wang Ying,Li Yonglin,Guo Anxin
|
|
(School of Air Traffic Control and Navigation, Air Force Engineering University, Xian 710051, China)
|
| Abstract: |
| To address the limitations of single-stage text-to-image generation technology, such as limited diversity of generated results and difficulty in precise control, this paper proposes a diverse text-to-image generative adversarial network framework based on word-level attention (WordGAN). Building on conditional generative adversarial networks, the framework incorporates a dual attention mechanism, a dual residual connection design, and a linear layer strategy. The dual attention mechanism consists of conditionally adaptive instance-layer normalization, a channel-attention module, and a pixel-attention module. It captures the weight distribution of network feature channels and spatial pixels by quantitatively analyzing the semantic contribution of each word in the text description. The dual residual connection design maintains the integrity of low-level visual features and supports deeper network expansion. The linear layer strategy uses linear layers to alter original image patterns, encouraging the framework to explore more secondary patterns. A series of experiments was conducted on the CUB bird, MSCOCO, and Oxford-102 benchmark datasets. The results show that WordGAN effectively achieves diversity in generated images without compromising quality and semantic consistency, and it outperforms other baseline text-to-image generation frameworks. The proposed diverse framework based on word-level attention provides a new approach for research on single-stage text-to-image generation tasks and demonstrates practical application value. |
| Key words: word-level attention mechanism dual-residual network mode collapse single-stage architecture text-to-image generation |