SkrGAN: Sketching-rendering unconditional generative adversarial networks for medical image synthesis

Zhang，Tianyang1,2; Fu，Huazhu3; Zhao，Yitian2; Cheng，Jun2,4; Guo，Mengjie5; Gu，Zaiwang6; Yang，Bing2; Xiao，Yuting5; Gao，Shenghua5; Liu，Jiang6

10.1007/978-3-030-32251-9_85

2019

0302-9743

1611-3349

LECT NOTES ARTIF INT

11767 LNCS

777-785

GEWERBESTRASSE 11, CHAM, CH-6330, SWITZERLAND

SPRINGER INTERNATIONAL PUBLISHING AG

Generative Adversarial Networks (GANs) have the capability of synthesizing images, which have been successfully applied to medical image synthesis tasks. However, most of existing methods merely consider the global contextual information and ignore the fine foreground structures, e.g., vessel, skeleton, which may contain diagnostic indicators for medical image analysis. Inspired by human painting procedure, which is composed of stroking and color rendering steps, we propose a Sketching-rendering Unconditional Generative Adversarial Network (SkrGAN) to introduce a sketch prior constraint to guide the medical image generation. In our SkrGAN, a sketch guidance module is utilized to generate a high quality structural sketch from random noise, then a color render mapping is used to embed the sketch-based representations and resemble the background appearances. Experimental results show that the proposed SkrGAN achieves the state-of-the-art results in synthesizing images for various image modalities, including retinal color fundus, X-Ray, Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). In addition, we also show that the performances of medical image segmentation method has been improved by using our synthesized images as data augmentation.

Deep learning Generative Adversarial Networks Medical image synthesis

其他

英语

CPCI-S ; EI

Computer Science ; Engineering ; Neurosciences & Neurology ; Imaging Science & Photographic Technology ; Radiology, Nuclear Medicine & Medical Imaging

Computer Science, Artificial Intelligence ; Computer Science, Software Engineering ; Engineering, Biomedical ; Neuroimaging ; Imaging Science & Photographic Technology ; Radiology, Nuclear Medicine & Medical Imaging

WOS:000548735900085

20194807769947

Color ; Deep learning ; Magnetic resonance imaging ; Medical imaging ; Diagnosis ; Rendering (computer graphics) ; Computerized tomography ; Image segmentation ; Image enhancement

Biomedical Engineering:461.1 ; Ergonomics and Human Factors Engineering:461.4 ; Medicine and Pharmacology:461.6 ; Magnetism: Basic Concepts and Phenomena:701.2 ; Data Processing and Image Processing:723.2 ; Artificial Intelligence:723.4 ; Computer Applications:723.5 ; Light/Optics:741.1 ; Imaging Techniques:746

2-s2.0-85075665995

Scopus

1.University of Chinese Academy of Sciences,Beijing,China
2.Cixi Institute of Biomedical Engineering,Chinese Academy of Sciences,Ningbo,China
3.Inception Institute of Artificial Intelligence,Abu Dhabi,United Arab Emirates
4.UBTech Research,Shenzhen,China
5.ShanghaiTech University,Shanghai,China
6.Southern University of Science and Technology,Shenzhen,China

Zhang，Tianyang,Fu，Huazhu,Zhao，Yitian,et al. SkrGAN: Sketching-rendering unconditional generative adversarial networks for medical image synthesis[C]. GEWERBESTRASSE 11, CHAM, CH-6330, SWITZERLAND:SPRINGER INTERNATIONAL PUBLISHING AG,2019:777-785.