基于词级注意力的多样化文本到图像生成
CSTR:
作者:
作者单位:

(空军工程大学 空管领航学院,西安 710051)

作者简介:

张振兴(1993—),男,博士,讲师;杨任农(1969—),男,教授,博士生导师

通讯作者:

俞利新,2207621676@qq.com

中图分类号:

V247

基金项目:

西安市青年人才托举计划(0959202513098);国家自然科学基金(4,4); 陕西省自然科学基金(2021JQ370)


Diverse text-to-image synthesis based on word-level attention
Author:
Affiliation:

(School of Air Traffic Control and Navigation, Air Force Engineering University, Xian 710051, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    为解决单阶段文本到图像生成技术存在的生成结果单一、精确调控困难等局限性,提出一种基于词级注意力的多样化文本到图像生成对抗网络框架(diverse text-to-image generative adversarial network framework based on word-level attention,WordGAN)。该框架在条件生成对抗网络的基础上,引入双重注意力机制、双残差连接设计和线性层策略。其中,双重注意力机制由条件自适应实例层归一化、通道注意力模块和像素注意力模块组成,通过量化分析文本描述中各词汇的语义贡献度捕捉网络特征通道及空间像素的权重分布;双残差连接设计用于维持底层视觉特征完整性和支持更深层次的网络扩展;线性层策略利用线性层改变原有图像模式,促进框架探索更多次要模式。在CUB bird、MSCOCO和Oxford-102基准数据集上进行一系列实验,结果表明:WordGAN有效地实现了生成图像的多样性,而且不会损害质量和语义一致性,其性能水平优于所对比的文本到图像生成框架。本文提出的基于词级注意力的多样化框架为单阶段文本到图像生成任务研究提供了新的思路,具有实际应用价值。

    Abstract:

    To address the limitations of single-stage text-to-image generation technology, such as limited diversity of generated results and difficulty in precise control, this paper proposes a diverse text-to-image generative adversarial network framework based on word-level attention (WordGAN). Building on conditional generative adversarial networks, the framework incorporates a dual attention mechanism, a dual residual connection design, and a linear layer strategy. The dual attention mechanism consists of conditionally adaptive instance-layer normalization, a channel-attention module, and a pixel-attention module. It captures the weight distribution of network feature channels and spatial pixels by quantitatively analyzing the semantic contribution of each word in the text description. The dual residual connection design maintains the integrity of low-level visual features and supports deeper network expansion. The linear layer strategy uses linear layers to alter original image patterns, encouraging the framework to explore more secondary patterns. A series of experiments was conducted on the CUB bird, MSCOCO, and Oxford-102 benchmark datasets. The results show that WordGAN effectively achieves diversity in generated images without compromising quality and semantic consistency, and it outperforms other baseline text-to-image generation frameworks. The proposed diverse framework based on word-level attention provides a new approach for research on single-stage text-to-image generation tasks and demonstrates practical application value.

    参考文献
    相似文献
    引证文献
引用本文

张振兴,杨任农,俞利新,王瑛,李永林,郭安新.基于词级注意力的多样化文本到图像生成[J].哈尔滨工业大学学报,2026,58(9):47. DOI:10.11918/202412036

复制
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-12-11
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2026-10-11
  • 出版日期:
文章二维码