Deep triplet hashing network for case-based medical image retrieval

Fang，Jiansheng1,2,4; Fu，Huazhu3; Liu，Jiang2

2021-04-01

10.1016/j.media.2021.101981

MEDICAL IMAGE ANALYSIS   影响因子和分区

1361-8415

1361-8423

Deep hashing methods have been shown to be the most efficient approximate nearest neighbor search techniques for large-scale image retrieval. However, existing deep hashing methods have a poor small-sample ranking performance for case-based medical image retrieval. The top-ranked images in the returned query results may be as a different class than the query image. This ranking problem is caused by classification, regions of interest (ROI), and small-sample information loss in the hashing space. To address the ranking problem, we propose an end-to-end framework, called Attention-based Triplet Hashing (ATH) network, to learn low-dimensional hash codes that preserve the classification, ROI, and small-sample information. We embed a spatial-attention module into the network structure of our ATH to focus on ROI information. The spatial-attention module aggregates the spatial information of feature maps by utilizing max-pooling, element-wise maximum, and element-wise mean operations jointly along the channel axis. To highlight the essential role of classification in direntiating case-based medical images, we propose a novel triplet cross-entropy loss to achieve maximal class-separability and maximal hash code-discriminability simultaneously during model training. The triplet cross-entropy loss can help to map the classification information of images and similarity between images into the hash codes. Moreover, by adopting triplet labels during model training, we can utilize the small-sample information fully to alleviate the imbalanced-sample problem. Extensive experiments on two case-based medical datasets demonstrate that our proposed ATH can further improve the retrieval performance compared to the state-of-the-art deep hashing methods and boost the ranking performance for small samples. Compared to the other loss methods, the triplet cross-entropy loss can enhance the classification performance and hash code-discriminability.

Deep hashing methods Medical image retrieval Region of interest Spatial attention Triplet labels

SCI ; EI

英语

通讯

WOS:000639621600008

20210709912403

Content based retrieval ; Entropy ; Hash functions ; Medical imaging ; Nearest neighbor search

Thermodynamics:641.1 ; Information Theory and Signal Processing:716.1 ; Imaging Techniques:746 ; Optimization Techniques:921.5

COMPUTER SCIENCE

2-s2.0-85100684632

Scopus

1.School of Computer Science and Technology,Harbin Institute of Technology,Harbin,150001,China
2.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China
3.Inception Institute of Artificial Intelligence,Abu Dhabi,United Arab Emirates
4.CVTE Research,Guangzhou,510530,China

Fang，Jiansheng,Fu，Huazhu,Liu，Jiang. Deep triplet hashing network for case-based medical image retrieval[J]. MEDICAL IMAGE ANALYSIS,2021,69.

Fang，Jiansheng,Fu，Huazhu,&Liu，Jiang.(2021).Deep triplet hashing network for case-based medical image retrieval.MEDICAL IMAGE ANALYSIS,69.

Fang，Jiansheng,et al."Deep triplet hashing network for case-based medical image retrieval".MEDICAL IMAGE ANALYSIS 69(2021).