计算机与现代化 ›› 2024, Vol. 0 ›› Issue (03): 92-96.doi: 10.3969/j.issn.1006-2475.2024.03.015

• 图像处理 • 上一篇    下一篇

基于生成对抗网络的乳腺癌免疫组化图像生成

  



  1. (1.武汉大学物理科学与技术学院,湖北 武汉 430072; 2.桂林航天工业学院电子信息与自动化学院,广西 桂林 541004)
  • 出版日期:2024-03-28 发布日期:2024-04-28
  • 基金资助:
    国家重点研发计划项目(2011CB707900); 广西高校中青年教师科研基础能力提升项目(2019KY0816)

Breast Cancer Immunohistochemical Image Generation Based on Generative Adversarial Network


  1. (1. School of Physics and Technology, Wuhan University, Wuhan 430072, China;
    2. School of Electronic Information and Automation, Guilin University of Aerospace Technology, Guilin 541004, China)
  • Online:2024-03-28 Published:2024-04-28
  • About author:卢梓菡(1999—),男,河南南阳人,硕士研究生,研究方向:图像处理,E-mail: 1044243182@qq.com; 张东(1963—), 男,教授,研究方向:数字图像处理,E-mail: dz_whu@163.com; 杨艳(1968—),女,副教授,研究方向:计算机视觉,E-mail: yyan_whu@163.com; 杨双(1979—),女,博士研究生,研究方向:计算机视觉,E-mail: yangshuang@guat.edu.cn。

摘要:
摘要:乳腺癌是一种凶险的恶性肿瘤,医学上需要根据人表皮生长因子受体2(HER2)水平来判断乳腺癌的侵袭性,从而制定治疗方案,这就需要对组织切片进行免疫组化(IHC)染色。为了解决IHC染色昂贵且费时的问题,首先,提出一种基于混合注意力残差模块的HER2预测网络,在残差模块中加入了CBAM模块,使得网络能够在空间、通道维度上更有侧重性地学习。预测网络能够由HE染色切片直接预测HER2水平,并且预测准确率达到97.5%以上,对比其他网络提升了2.5个百分点以上。随后提出一种多尺度生成对抗网络,使用引入混合注意力残差模块的ResNet-9blocks作为生成器,PatchGan作为判别器,并自定义多尺度损失函数。生成对抗网络可以由HE染色切片直接生成模拟IHC染色切片,在低HER2水平下生成的图像与真实图像的SSIM为0.498,PSNR为24.49 dB。

关键词: 关键词:生成对抗网络, 图像处理, 混合注意力机制, 类别预测

Abstract:
Abstract: Breast cancer is a dangerous malignant tumor. In medicine, human epidermal growth factor receptor 2(HER2)levels are needed to determine the aggressiveness of breast cancer in order to develop a treatment plan, this requires immunohistochemical(IHC)staining of the tissue sections. In order to solve the problem that IHC staining is expensive and time-consuming, firstly, a HER2 prediction network based on mixed attention residual module is proposed, and a CBAM module is added to the residual module, so that the network can focus on learning at the spatial and channel levels. The prediction network could directly predict HER2 level from HE stained sections, and the prediction accuracy reached more than 97.5%, which increased by more than 2.5 percentage points compared with other networks. Subsequently, a multi-scale generative adversarial network is proposed, which uses ResNet-9blocks with mixed attention residuals module as generator and PatchGan as discriminator and self-defines multi-scale loss function. This network can directly generate simulated IHC slices from HE stained slices. At low HER2 level, SSIM and PSNR between the generated image and the real image are 0.498 and 24.49 dB.

Key words: Key words: generative adversarial network, image processing, mixed attention mechanism, category prediction

中图分类号: