Abstract
Histopathology image analysis plays an important role in the treatment and diagnosis of cancer. However, analysis of whole slide images (WSI) with deep learning is challenging given that the duration of pixel-level annotations is laborious and time consuming. To address this, recent methods have considered WSI classification as a Multiple Instance Learning (MIL) problem often with a multi-stage process for learning instance and slide level features. Currently, most methods focus on either instance-selection or instance prediction-aggregation that often fails to generalize and ignores instance relations. In this work, we propose a MIL-based method to jointly learn both instance- and bag-level embeddings in a single framework. In addition, we propose a center loss that maps embeddings of instances from the same bag to a single centroid and reduces intra-class variations. Consequently, our model can accurately predict instance labels and leverages robust hierarchical pooling of features to obtain bag-level features without sacrificing accuracy. Experimental results on curated colon datasets show the effectiveness of the proposed methods against recent state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We used the publicly available implementations: https://github.com/MSKCC-Computational-Pathology/MIL-nature-medicine-2019.
- 2.
- 3.
References
Boland, C.R., Goel, A.: Microsatellite instability in colorectal cancer. Gastroenterology 138(6), 2073–2087 (2010)
Campanella, G.: Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25(8), 1301–1309 (2019)
Chen, H., et al.: Rectified cross-entropy and upper transition loss for weakly supervised whole slide image classifier. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 351–359. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_39
Cosatto, E., et al.: Automated gastric cancer diagnosis on h and e-stained sections; training a classifier on a large scale with multiple instance machine learning. In: Medical Imaging 2013: Digital Pathology, vol. 8676, p. 867605. International Society for Optics and Photonics (2013)
Hashimoto, N., et al.: Multi-scale domain-adversarial multiple-instance CNN for cancer subtype classification with non-annotated histopathological images. arXiv preprint arXiv:2001.01599 (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433 (2016)
Ilse, M., Tomczak, J.M., Welling, M.: Attention-based deep multiple instance learning. arXiv preprint arXiv:1802.04712 (2018)
Li, M., Wu, L., Wiliem, A., Zhao, K., Zhang, T., Lovell, B.: Deep instance-level hard negative mining model for histopathology images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, pp. 514–522. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_57
Li, S.: Multi-instance multi-scale CNN for medical image classification. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 531–539. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_58
Li, W., Nguyen, V.-D., Liao, H., Wilder, M., Cheng, K., Luo, J.: Patch transformer for multi-tagging whole slide histopathology images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 532–540. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_59
Lin, Y., Zhang, H.: Regularized instance embedding for deep multi-instance learning. Appl. Sci. 10(1), 64 (2020)
Lu, M.Y., Chen, R.J., Wang, J., Dillon, D., Mahmood, F.: Semi-supervised histology classification using deep multiple instance learning and contrastive predictive coding. arXiv preprint arXiv:1910.10825 (2019)
van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Muhammad, H., et al.: Unsupervised subtyping of cholangiocarcinoma using a deep clustering convolutional autoencoder. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 604–612. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_67
Nazeri, K., Aminpour, A., Ebrahimi, M.: Two-stage convolutional neural network for breast cancer histology image classification. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 717–726. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93000-8_81
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
Tellez, D., Litjens, G., van der Laak, J., Ciompi, F.: Neural image compression for gigapixel histopathology image analysis. IEEE Trans. Pattern Anal. Mach. Intell. (2019)
Wang, X.: Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans. Cybern. 50, 3950–3962 (2019)
Wang, X., Yan, Y., Tang, P., Bai, X., Liu, W.: Revisiting multiple instance neural networks. Pattern Recogn. 74, 15–24 (2018)
Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 499–515. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_31
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487 (2016)
Xu, G., et al.: CAMEL: a weakly supervised learning framework for histopathology image segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 10682–10691 (2019)
Yao, J., Zhu, X., Huang, J.: Deep multi-instance learning for survival prediction from whole slide images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 496–504. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_55
Zhu, X., Yao, J., Zhu, F., Huang, J.: WSISA: making survival prediction from whole slide histopathological images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7234–7242 (2017)
Acknowledgment
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (MSIT) (No. 2019R1C1C1008727), and the Grant of Artificial Intelligence Bio-Robot Medical Convergence Technology funded by the Ministry of Trade, Industry and Energy, the Ministry of Science and ICT, and the Ministry of Health and Welfare (No. 20001533).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
� 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chikontwe, P., Kim, M., Nam, S.J., Go, H., Park, S.H. (2020). Multiple Instance Learning with Center Embeddings for Histopathology Classification. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
