Project 2.1 – Weakly Labeled Learning

Gerda Bortsova, Cristina González-Gonzalo, Suzanne C. Wetstein, Florian Dubost, Ioannis Katramados, Laurens Hogeweg, Bart Liefers, Bram van Ginneken, Josien P.W. Pluim, Mitko Veta, Clara I. Sánchez, Marleen de Bruijne

Adversarial attack vulnerability of medical image analysis systems: Unexplored factors Journal Article

Medical Image Analysis, 73 , 2021.

Friso G. Heslinga, Ruben T. Lucassen, Myrthe A. van den Berg, Luuk van der Hoek, Josien P. W. Pluim, Javier Cabrerizo, Mark Alberti, Mitko Veta

Corneal pachymetry by AS-OCT after Descemet’s membrane endothelial keratoplasty Journal Article

Scientific Reports, 11 , pp. 13976, 2021.

Gerda Bortsova, Daniel Bos, Florian Dubost, Meike W. Vernooij, M. Kamran Ikram, Gijs van Tulder, Marleen de Bruijne

Automated Assessment of Intracranial Carotid Artery Calcification in Non-Contrast CT Using Deep Learning Journal Article Forthcoming

Forthcoming.

BibTeX

Coen de Vente, Pieter Vos, Matin Hosseinzadeh, Josien Pluim, Mitko Veta

Deep Learning Regression for Prostate Cancer Detection and Grading in Bi-parametric MRI Journal Article Forthcoming

IEEE Transactions on Biomedical Engineering, Forthcoming.

Suzanne C Wetstein, Nikolas Stathonikos, Josien PW Pluim, Yujing J Heng, Natalie D ter Hoeve, Celien PH Vreuls, Paul J van Diest, Mitko Veta

Deep Learning-Based Grading of Ductal Carcinoma In Situ in Breast Histopathology Images Journal Article Forthcoming

arXiv, Forthcoming.

@article{Wetstein2020deep,
title = {Deep Learning-Based Grading of Ductal Carcinoma In Situ in Breast Histopathology Images},
author = {Suzanne C Wetstein, Nikolas Stathonikos, Josien PW Pluim, Yujing J Heng, Natalie D ter Hoeve, Celien PH Vreuls, Paul J van Diest, Mitko Veta},
url = {https://arxiv.org/abs/2010.03244},
year = {2020},
date = {2020-10-15},
journal = {arXiv},
abstract = {Ductal carcinoma in situ (DCIS) is a non-invasive breast cancer that can progress into invasive ductal carcinoma (IDC). Studies suggest DCIS is often overtreated since a considerable part of DCIS lesions may never progress into IDC. Lower grade lesions have a lower progression speed and risk, possibly allowing treatment de-escalation. However, studies show significant inter-observer variation in DCIS grading. Automated image analysis may provide an objective solution to address high subjectivity of DCIS grading by pathologists.
In this study, we developed a deep learning-based DCIS grading system. It was developed using the consensus DCIS grade of three expert observers on a dataset of 1186 DCIS lesions from 59 patients. The inter-observer agreement, measured by quadratic weighted Cohen's kappa, was used to evaluate the system and compare its performance to that of expert observers. We present an analysis of the lesion-level and patient-level inter-observer agreement on an independent test set of 1001 lesions from 50 patients.
The deep learning system (dl) achieved on average slightly higher inter-observer agreement to the observers (o1, o2 and o3) (κo1,dl=0.81,κo2,dl=0.53,κo3,dl=0.40) than the observers amongst each other (κo1,o2=0.58,κo1,o3=0.50,κo2,o3=0.42) at the lesion-level. At the patient-level, the deep learning system achieved similar agreement to the observers (κo1,dl=0.77,κo2,dl=0.75,κo3,dl=0.70) as the observers amongst each other (κo1,o2=0.77,κo1,o3=0.75,κo2,o3=0.72).
In conclusion, we developed a deep learning-based DCIS grading system that achieved a performance similar to expert observers. We believe this is the first automated system that could assist pathologists by providing robust and reproducible second opinions on DCIS grade.},
keywords = {},
pubstate = {forthcoming},
tppubtype = {article}
}

Friso G. Heslinga, Mark Alberti, Josien P.W. Pluim, Javier Cabrerizo, Mitko Veta

Quantifying Graft Detachment after Descemet's Membrane Endothelial Keratoplasty with Deep Convolutional Neural Networks Journal Article

Translational Vision Science & Technology, 9 (48), 2020.

@article{heslinga2020quantifying,
title = {Quantifying Graft Detachment after Descemet's Membrane Endothelial Keratoplasty with Deep Convolutional Neural Networks},
author = {Friso G. Heslinga, Mark Alberti, Josien P.W. Pluim, Javier Cabrerizo, Mitko Veta},
url = {https://tvst.arvojournals.org/article.aspx?articleid=2770687},
year = {2020},
date = {2020-08-01},
journal = {Translational Vision Science & Technology},
volume = {9},
number = {48},
abstract = {Purpose: We developed a method to automatically locate and quantify graft detachment after Descemet's Membrane Endothelial Keratoplasty (DMEK) in Anterior Segment Optical Coherence Tomography (AS-OCT) scans. Methods: 1280 AS-OCT B-scans were annotated by a DMEK expert. Using the annotations, a deep learning pipeline was developed to localize scleral spur, center the AS-OCT B-scans and segment the detached graft sections. Detachment segmentation model performance was evaluated per B-scan by comparing (1) length of detachment and (2) horizontal projection of the detached sections with the expert annotations. Horizontal projections were used to construct graft detachment maps. All final evaluations were done on a test set that was set apart during training of the models. A second DMEK expert annotated the test set to determine inter-rater performance. Results: Mean scleral spur localization error was 0.155 mm, whereas the inter-rater difference was 0.090 mm. The estimated graft detachment lengths were in 69% of the cases within a 10-pixel (~150{mu}m) difference from the ground truth (77% for the second DMEK expert). Dice scores for the horizontal projections of all B-scans with detachments were 0.896 and 0.880 for our model and the second DMEK expert respectively. Conclusion: Our deep learning model can be used to automatically and instantly localize graft detachment in AS-OCT B-scans. Horizontal detachment projections can be determined with the same accuracy as a human DMEK expert, allowing for the construction of accurate graft detachment maps. Translational Relevance: Automated localization and quantification of graft detachment can support DMEK research and standardize clinical decision making.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

Kevin H. Kensler; Emily Z.F. Liu; Suzanne C. Wetstein; Allison M. Onken; Christina I. Luffman; Gabrielle M. Baker; Laura C. Collins; Stuart J. Schnitt; Vanessa C. Bret-Mounet; Mitko Veta; Josien P.W. Pluim; Ying Liu; Graham A. Colditz; A. Heather Eliassen; Susan E. Hankinson; Rulla M. Tamimi; Yujing J. Heng

Automated quantitative measures of terminal duct lobular unit involution and breast cancer risk Journal Article

Cancer epidemiology, biomarkers & prevention, 29 (11), 2020.

@article{kensler2020automated,
title = {Automated quantitative measures of terminal duct lobular unit involution and breast cancer risk},
author = {Kevin H. Kensler and Emily Z.F. Liu and Suzanne C. Wetstein and Allison M. Onken and Christina I. Luffman and Gabrielle M. Baker and Laura C. Collins and Stuart J. Schnitt and Vanessa C. Bret-Mounet and Mitko Veta and Josien P.W. Pluim and Ying Liu and Graham A. Colditz and A. Heather Eliassen and Susan E. Hankinson and Rulla M. Tamimi and Yujing J. Heng},
url = {https://cebp.aacrjournals.org/content/29/11/2358.abstract},
year = {2020},
date = {2020-11-01},
journal = {Cancer epidemiology, biomarkers & prevention},
volume = {29},
number = {11},
abstract = {Background: Manual qualitative and quantitative measures of terminal duct lobular unit (TDLU) involution were previously reported to be inversely associated with breast cancer risk. We developed and applied a deep learning method to yield quantitative measures of TDLU involution in normal breast tissue. We assessed the associations of these automated measures with breast cancer risk factors and risk.

Methods: We obtained eight quantitative measures from whole slide images from a benign breast disease (BBD) nested case–control study within the Nurses’ Health Studies (287 breast cancer cases and 1,083 controls). Qualitative assessments of TDLU involution were available for 177 cases and 857 controls. The associations between risk factors and quantitative measures among controls were assessed using analysis of covariance adjusting for age. The relationship between each measure and risk was evaluated using unconditional logistic regression, adjusting for the matching factors, BBD subtypes, parity, and menopausal status. Qualitative measures and breast cancer risk were evaluated accounting for matching factors and BBD subtypes.

Results: Menopausal status and parity were significantly associated with all eight measures; select TDLU measures were associated with BBD histologic subtype, body mass index, and birth index (P < 0.05). No measure was correlated with body size at ages 5–10 years, age at menarche, age at first birth, or breastfeeding history (P > 0.05). Neither quantitative nor qualitative measures were associated with breast cancer risk.

Conclusions: Among Nurses’ Health Studies women diagnosed with BBD, TDLU involution is not a biomarker of subsequent breast cancer.

Impact: TDLU involution may not impact breast cancer risk as previously thought.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

Suzanne C. Wetstein, Allison M. Onken, Christina Luffman, Gabrielle M. Baker, Michael E. Pyle, Kevin H. Kensler, Ying Liu, Bart Bakker, Ruud Vlutters, Marinus B. van Leeuwen, Laura C. Collins, Stuart J. Schnitt, Josien P. W. Pluim, Rulla M. Tamimi, Yujing J. Heng, Mitko Veta

Deep learning assessment of breast terminal duct lobular unit involution: Towards automated prediction of breast cancer risk Journal Article

PLoS ONE, 15 (4), pp. e0231653, 2020.

@article{TDLUpaper,
title = {Deep learning assessment of breast terminal duct lobular unit involution: Towards automated prediction of breast cancer risk},
author = {Suzanne C. Wetstein, Allison M. Onken, Christina Luffman, Gabrielle M. Baker, Michael E. Pyle, Kevin H. Kensler, Ying Liu, Bart Bakker, Ruud Vlutters, Marinus B. van Leeuwen, Laura C. Collins, Stuart J. Schnitt, Josien P. W. Pluim, Rulla M. Tamimi, Yujing J. Heng, Mitko Veta},
editor = {Ulas Bagci},
url = {https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0231653},
doi = {10.1371/journal.pone.0231653},
year = {2020},
date = {2020-04-15},
journal = {PLoS ONE},
volume = {15},
number = {4},
pages = {e0231653},
abstract = {Terminal duct lobular unit (TDLU) involution is the regression of milk-producing structures in the breast. Women with less TDLU involution are more likely to develop breast cancer. A major bottleneck in studying TDLU involution in large cohort studies is the need for labor-intensive manual assessment of TDLUs. We developed a computational pathology solution to automatically capture TDLU involution measures. Whole slide images (WSIs) of benign breast biopsies were obtained from the Nurses’ Health Study. A set of 92 WSIs was annotated for acini, TDLUs and adipose tissue to train deep convolutional neural network (CNN) models for detection of acini, and segmentation of TDLUs and adipose tissue. These networks were integrated into a single computational method to capture TDLU involution measures including number of TDLUs per tissue area, median TDLU span and median number of acini per TDLU. We validated our method on 40 additional WSIs by comparing with manually acquired measures. Our CNN models detected acini with an F1 score of 0.73±0.07, and segmented TDLUs and adipose tissue with Dice scores of 0.84±0.13 and 0.87±0.04, respectively. The inter-observer ICC scores for manual assessments on 40 WSIs of number of TDLUs per tissue area, median TDLU span, and median acini count per TDLU were 0.71, 0.81 and 0.73, respectively. Intra-observer reliability was evaluated on 10/40 WSIs with ICC scores of >0.8. Inter-observer ICC scores between automated results and the mean of the two observers were: 0.80 for number of TDLUs per tissue area, 0.57 for median TDLU span, and 0.80 for median acini count per TDLU. TDLU involution measures evaluated by manual and automated assessment were inversely associated with age and menopausal status. We developed a computational pathology method to measure TDLU involution. This technology eliminates the labor-intensiveness and subjectivity of manual TDLU assessment, and can be applied to future breast cancer risk studies.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}

Suzanne C. Wetstein, Cristina González-Gonzalo, Gerda Bortsova, Bart Liefers, Florian Dubost, Ioannis Katramados, Laurens Hogeweg, Bram van Ginneken, Josien P.W. Pluim, Marleen de Bruijne, Clara I. Sánchez, Mitko Veta

Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors Conference

2020.

Are adversarial attacks an actual threat for deep learning systems in real-world eye disease screening settings? Conference

C. González-Gonzalo, S. C. Wetstein, G. Bortsova, B. Liefers, B. van Ginneken, C. I. Sánchez

European Society of Retina Specialists, 2020.

@conference{Gonz20c,
title = {Are adversarial attacks an actual threat for deep learning systems in real-world eye disease screening settings?},
author = {C. González-Gonzalo, S. C. Wetstein, G. Bortsova, B. Liefers, B. van Ginneken, C. I. Sánchez},
url = {https://www.euretina.org/congress/amsterdam-2020/virtual-2020-freepapers/},
year = {2020},
date = {2020-10-02},
booktitle = {European Society of Retina Specialists},
abstract = {Purpose:

Deep learning (DL) systems that perform image-level classification with convolutional neural networks (CNNs) have been shown to provide high-performance solutions for automated screening of eye diseases. Nevertheless, adversarial attacks have been recently screening settings, where there is restricted access to the systems and limited knowledge about certain factors, such as their CNN architecture or the data used for development.

Setting:

Deep learning for automated screening of eye diseases.

Methods:

We used the Kaggle dataset for diabetic retinopathy detection. It contains 88,702 manually-labelled color fundus images, which we split into test (12%) and development (88%). Development data were split into two equally-sized sets (d1 and d2); a third set (d3) was generated using half of the images in d2. In each development set, 80%/20% of the images were used for training/validation. All splits were done randomly at patient-level. As attacked system, we developed a randomly-initialized CNN based on the Inception-v3 architecture using d1. We performed the attacks (1) in a white-box (WB) setting, with full access to the attacked system to generate the adversarial images, and (2) in black-box (BB) settings, without access to the attacked system and using a surrogate system to craft the attacks. We simulated different BB settings, sequentially decreasing the available knowledge about the attacked system: same architecture, using d1 (BB-1); different architecture (randomly-initialized DenseNet-121), using d1 (BB-2); same architecture, using d2 (BB-3); different architecture, using d2 (BB-4); different architecture, using d3 (BB-5). In each setting, adversarial images containing non-perceptible noise were generated by applying the fast gradient sign method to each image of the test set and processed by the attacked system.

Results:

The performance of the attacked system to detect referable diabetic retinopathy without attacks and under the different attack settings was measured on the test set using the area under the receiver operating characteristic curve (AUC). Without attacks, the system achieved an AUC of 0.88. In each attack setting, the relative decrease in AUC with respect to the original performance was computed. In the WB setting, there was a 99.9% relative decrease in performance. In the BB-1 setting, the relative decrease in AUC was 67.3%. In the BB-2 setting, the AUC suffered a 40.2% relative decrease. In the BB-3 setting, the relative decrease was 37.9%. In the BB-4 setting, the relative decrease in AUC was 34.1%. Lastly, in the BB-5 setting, the performance of the attacked system decreased 3.8% regarding its original performance.

Conclusions:

The results obtained in the different settings show a drastic decrease of the attacked DL system's vulnerability to adversarial attacks when the access and knowledge about it are limited. The impact on performance is extremely reduced when restricting the direct access to the system (from the WB to the BB-1 setting). The attacks become slightly less effective when not having access to the same development data (BB-3), compared to not using the same CNN architecture (BB-2). Attacks' effectiveness further decreases when both factors are unknown (BB-4). If the amount of development data is additionally reduced (BB-5), the original performance barely deteriorates. This last setting is the most similar to realistic screening settings, since most systems are currently closed source and use additional large private datasets for development. In conclusion, these factors should be acknowledged for future development of robust DL systems, as well as considered when evaluating the vulnerability of currently-available systems to adversarial attacks. Having limited access and knowledge about the systems determines the actual threat these attacks pose. We believe awareness about this matter will increase experts' trust and facilitate the integration of DL systems in real-world settings.},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}

Purpose:

Deep learning (DL) systems that perform image-level classification with convolutional neural networks (CNNs) have been shown to provide high-performance solutions for automated screening of eye diseases. Nevertheless, adversarial attacks have been recently screening settings, where there is restricted access to the systems and limited knowledge about certain factors, such as their CNN architecture or the data used for development.

Setting:

Deep learning for automated screening of eye diseases.

Methods:

We used the Kaggle dataset for diabetic retinopathy detection. It contains 88,702 manually-labelled color fundus images, which we split into test (12%) and development (88%). Development data were split into two equally-sized sets (d1 and d2); a third set (d3) was generated using half of the images in d2. In each development set, 80%/20% of the images were used for training/validation. All splits were done randomly at patient-level. As attacked system, we developed a randomly-initialized CNN based on the Inception-v3 architecture using d1. We performed the attacks (1) in a white-box (WB) setting, with full access to the attacked system to generate the adversarial images, and (2) in black-box (BB) settings, without access to the attacked system and using a surrogate system to craft the attacks. We simulated different BB settings, sequentially decreasing the available knowledge about the attacked system: same architecture, using d1 (BB-1); different architecture (randomly-initialized DenseNet-121), using d1 (BB-2); same architecture, using d2 (BB-3); different architecture, using d2 (BB-4); different architecture, using d3 (BB-5). In each setting, adversarial images containing non-perceptible noise were generated by applying the fast gradient sign method to each image of the test set and processed by the attacked system.

Results:

The performance of the attacked system to detect referable diabetic retinopathy without attacks and under the different attack settings was measured on the test set using the area under the receiver operating characteristic curve (AUC). Without attacks, the system achieved an AUC of 0.88. In each attack setting, the relative decrease in AUC with respect to the original performance was computed. In the WB setting, there was a 99.9% relative decrease in performance. In the BB-1 setting, the relative decrease in AUC was 67.3%. In the BB-2 setting, the AUC suffered a 40.2% relative decrease. In the BB-3 setting, the relative decrease was 37.9%. In the BB-4 setting, the relative decrease in AUC was 34.1%. Lastly, in the BB-5 setting, the performance of the attacked system decreased 3.8% regarding its original performance.

Conclusions:

The results obtained in the different settings show a drastic decrease of the attacked DL system's vulnerability to adversarial attacks when the access and knowledge about it are limited. The impact on performance is extremely reduced when restricting the direct access to the system (from the WB to the BB-1 setting). The attacks become slightly less effective when not having access to the same development data (BB-3), compared to not using the same CNN architecture (BB-2). Attacks' effectiveness further decreases when both factors are unknown (BB-4). If the amount of development data is additionally reduced (BB-5), the original performance barely deteriorates. This last setting is the most similar to realistic screening settings, since most systems are currently closed source and use additional large private datasets for development. In conclusion, these factors should be acknowledged for future development of robust DL systems, as well as considered when evaluating the vulnerability of currently-available systems to adversarial attacks. Having limited access and knowledge about the systems determines the actual threat these attacks pose. We believe awareness about this matter will increase experts' trust and facilitate the integration of DL systems in real-world settings.

F.G. Heslinga, J.P.W. Pluim, A.J. Houben, M.T. Schram, R.M. Henry, C.D. Stehouwer, M.J. Van Greevenbroek, T.T. Berendschot, M. Veta

Direct Classification of Type 2 Diabetes From Retinal Fundus Images in a Population-based Sample From The Maastricht Study Inproceedings

Medical Imaging 2020: Computer-Aided Diagnosis, pp. 113141N, International Society for Optics and Photonics, 2020.

Ruwan Tennakoon, Gerda Bortsova, Silas Ørting, Amirali K Gostar, Mathilde MW Wille, Zaigham Saghir, Reza Hoseinnezhad, Marleen de Bruijne, Alireza Bab-Hadiashar

Classification of Volumetric Images Using Multi-Instance Learning and Extreme Value Theorem Journal Article

IEEE Transactions on Medical Imaging, 39 (4), pp. 854-865, 2019.

Christina I. Luffman, Suzanne C. Wetstein, Allison M. Onken, Michael E. Pyle, Kevin H. Kensler, Ying Liu, Josien P. Pluim, Mitko Veta, Stuart J. Schnitt, Rulla M. Tamimi, Gabrielle M. Baker, Laura C. Collins, Yu Jing Heng

Assessing Breast Terminal Duct Lobular Unit Involution: A Computational Pathology Approach Conference

Abstracts and Case Studies From the College of American Pathologists 2019 Annual Meeting (CAP19), 143 (9), Archives of Pathology & Laboratory Medicine, 2019.

Allison M. Onken, Suzanne Wetstein, Michael Pyle, Josien Pluim, Stuart J. Schnitt, Gabrielle M Baker, Laura C. Collins, Rulla Tamimi, Mitko Veta, Yujing Jan Heng

Deep Learning Networks to Segment and Detect Breast Terminal Duct Lobular Units, Acini, and Adipose Tissue: A Step Toward the Automated Analysis of Lobular Involution as a Marker for Breast Cancer Risk Conference

United States and Canadian Academy of Pathology (USCAP), 2019.

Abstract | BibTeX

@conference{OnkenWetstein2019,
title = {Deep Learning Networks to Segment and Detect Breast Terminal Duct Lobular Units, Acini, and Adipose Tissue: A Step Toward the Automated Analysis of Lobular Involution as a Marker for Breast Cancer Risk},
author = {Allison M. Onken, Suzanne Wetstein, Michael Pyle, Josien Pluim, Stuart J. Schnitt, Gabrielle M Baker, Laura C. Collins, Rulla Tamimi, Mitko Veta, Yujing Jan Heng},
year = {2019},
date = {2019-03-16},
booktitle = {United States and Canadian Academy of Pathology (USCAP)},
abstract = {Background:
Terminal duct lobular unit (TDLU) involution is the physiological process whereby Type 2 and 3
lobules revert to Type 1 after child-bearing years. TDLU involution (quantitatively assessed by
TDLU count per mm2, TDLU span, and acini count per TDLU) is inversely associated with breast
cancer risk. The manual assessment of involution is time-consuming and subjective, making it
impractical to perform on large epidemiological studies. Deep learning algorithms such as
convolutional neural networks (CNNs) could be utilized for rapid and automated assessment of
TDLU involution. We designed two CNNs to segment TDLUs and detect acini as the first step
toward large-scale assessment of TDLU involution, and a third CNN to segment adipose tissue.
Design:
Whole slide images (WSIs; n=50) were obtained from the Nurses’ Health Study Incident Benign
Breast Disease Study. For each WSI, TDLUs, acini, and adipose tissue were annotated within a
region of interest comprising approximately 10% of the total tissue area. In order to assess
involution in histologically normal breast parenchyma only, TDLUs with proliferative or metaplastic
changes were excluded from manual evaluation. CNNs were engineered to recognize TDLUs, acini,
and adipose tissue using 60% of the WSIs for training, 20% as a test set, and 20% for validation. F1
and Dice scores were calculated as accuracy measures to compare CNN segmentation to manual
assessment.
Results:
Our CNNs detected acini, segmented TDLUs, and segmented adipose tissue with accuracy measures
of 0.73, 0.84, and 0.86, respectively. Two primary causes of discordance with manual assessment
were identified: 1) complex clustering of TDLUs where our CNN had difficulty predicting TDLU
boundaries and 2) acini with proliferative or metaplastic changes which our CNN frequently
detected as acini but which were intentionally excluded from manual annotation.
Conclusion:
We have developed a series of deep learning networks to segment and detect TDLUs, acini, and
adipose tissue on WSIs. With accuracy measures of >0.7, our CNNs are sufficiently robust to be
integrated into a computational pipeline for automated assessment of the quantitative features of
TDLU involution, and will be further refined to address sources of discordance with manual
assessment. This is the first step toward the large-scale quantification of TDLU involution which,
when applied to patient samples, could be used to better determine the breast cancer risk associated
with lobule type and degree of involution.},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}

Background:
Terminal duct lobular unit (TDLU) involution is the physiological process whereby Type 2 and 3
lobules revert to Type 1 after child-bearing years. TDLU involution (quantitatively assessed by
TDLU count per mm2, TDLU span, and acini count per TDLU) is inversely associated with breast
cancer risk. The manual assessment of involution is time-consuming and subjective, making it
impractical to perform on large epidemiological studies. Deep learning algorithms such as
convolutional neural networks (CNNs) could be utilized for rapid and automated assessment of
TDLU involution. We designed two CNNs to segment TDLUs and detect acini as the first step
toward large-scale assessment of TDLU involution, and a third CNN to segment adipose tissue.
Design:
Whole slide images (WSIs; n=50) were obtained from the Nurses’ Health Study Incident Benign
Breast Disease Study. For each WSI, TDLUs, acini, and adipose tissue were annotated within a
region of interest comprising approximately 10% of the total tissue area. In order to assess
involution in histologically normal breast parenchyma only, TDLUs with proliferative or metaplastic
changes were excluded from manual evaluation. CNNs were engineered to recognize TDLUs, acini,
and adipose tissue using 60% of the WSIs for training, 20% as a test set, and 20% for validation. F1
and Dice scores were calculated as accuracy measures to compare CNN segmentation to manual
assessment.
Results:
Our CNNs detected acini, segmented TDLUs, and segmented adipose tissue with accuracy measures
of 0.73, 0.84, and 0.86, respectively. Two primary causes of discordance with manual assessment
were identified: 1) complex clustering of TDLUs where our CNN had difficulty predicting TDLU
boundaries and 2) acini with proliferative or metaplastic changes which our CNN frequently
detected as acini but which were intentionally excluded from manual annotation.
Conclusion:
We have developed a series of deep learning networks to segment and detect TDLUs, acini, and
adipose tissue on WSIs. With accuracy measures of >0.7, our CNNs are sufficiently robust to be
integrated into a computational pipeline for automated assessment of the quantitative features of
TDLU involution, and will be further refined to address sources of discordance with manual
assessment. This is the first step toward the large-scale quantification of TDLU involution which,
when applied to patient samples, could be used to better determine the breast cancer risk associated
with lobule type and degree of involution.

Gerda Bortsova, Florian Dubost, Laurens Hogeweg, Ioannis Katramados, Marleen de Bruijne

Semi-supervised medical image segmentation via learning consistency under transformations Inproceedings

International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 810-818, Springer, Cham, 2019.

@inproceedings{Bortsova2019,
title = {Semi-supervised medical image segmentation via learning consistency under transformations},
author = {Gerda Bortsova, Florian Dubost, Laurens Hogeweg, Ioannis Katramados, Marleen de Bruijne},
url = {https://arxiv.org/abs/1911.01218},
doi = {https://doi.org/10.1007/978-3-030-32226-7_90},
year = {2019},
date = {2019-10-13},
urldate = {2020-08-20},
booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
pages = {810-818},
publisher = {Springer, Cham},
abstract = {The scarcity of labeled data often limits the application of supervised deep learning techniques for medical image segmentation. This has motivated the development of semi-supervised techniques that learn from a mixture of labeled and unlabeled images. In this paper, we propose a novel semi-supervised method that, in addition to supervised learning on labeled training images, learns to predict segmentations consistent under a given class of transformations on both labeled and unlabeled images. More specifically, in this work we explore learning equivariance to elastic deformations. We implement this through: (1) a Siamese architecture with two identical branches, each of which receives a differently transformed image, and (2) a composite loss function with a supervised segmentation loss term and an unsupervised term that encourages segmentation consistency between the predictions of the two branches. We evaluate the method on a public dataset of chest radiographs with segmentations of anatomical structures using 5-fold cross-validation. The proposed method reaches significantly higher segmentation accuracy compared to supervised learning. This is due to learning transformation consistency on both labeled and unlabeled images, with the latter contributing the most. We achieve the performance comparable to state-of-the-art chest X-ray segmentation methods while using substantially fewer labeled images.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

Suzanne C. Wetstein, Allison M. Onken, Gabrielle M. Baker, Michael E. Pyle, Josien P. W. Pluim, Rulla M. Tamimi, Yujing J. Heng, Mitko Veta

Detection of acini in histopathology slides: towards automated prediction of breast cancer risk Inproceedings

SPIE Medical Imaging, 2019.

@inproceedings{Wetstein2019,
title = {Detection of acini in histopathology slides: towards automated prediction of breast cancer risk},
author = {Suzanne C. Wetstein, Allison M. Onken, Gabrielle M. Baker, Michael E. Pyle, Josien P. W. Pluim, Rulla M. Tamimi, Yujing J. Heng, Mitko Veta},
url = {https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10956/109560Q/Detection-of-acini-in-histopathology-slides--towards-automated-prediction/10.1117/12.2511408.full},
year = {2019},
date = {2019-03-18},
booktitle = {SPIE Medical Imaging},
abstract = {Terminal duct lobular units (TDLUs) are structures in the breast which involute with the completion of childbearing and physiological ageing. Women with less TDLU involution are more likely to develop breast cancer than those with more involution. Thus, TDLU involution may be utilized as a biomarker to predict invasive cancer risk. Manual assessment of TDLU involution is a cumbersome and subjective process. This makes it amenable for automated assessment by image analysis. In this study, we developed and evaluated an acini detection method as a first step towards automated assessment of TDLU involution using a dataset of histopathological whole-slide images (WSIs) from the Nurses’ Health Study (NHS) and NHSII. The NHS/NHSII is among the world's largest investigations of epidemiological risk factors for major chronic diseases in women. We compared three different approaches to detect acini in WSIs using the U-Net convolutional neural network architecture. The approaches differ in the target that is predicted by the network: circular mask labels, soft labels and distance maps. Our results showed that soft label targets lead to a better detection performance than the other methods. F1 scores of 0.65, 0.73 and 0.66 were obtained with circular mask labels, soft labels and distance maps, respectively. Our acini detection method was furthermore validated by applying it to measure acini count per mm2 of tissue area on an independent set of WSIs. This measure was found to be significantly negatively correlated with age.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

Friso G. Heslinga, Josien P. W. Pluim, Behdad Dashtbozorg, Tos T. J. M. Berendschot, A. J. H. M. Houben, Ronald M. A. Henry, Mitko Veta

Approximation of a pipeline of unsupervised retina image analysis methods with a CNN Inproceedings

SPIE Medical Imaging, 2019.

@inproceedings{Heslinga2019,
title = {Approximation of a pipeline of unsupervised retina image analysis methods with a CNN},
author = {Friso G. Heslinga, Josien P. W. Pluim, Behdad Dashtbozorg, Tos T. J. M. Berendschot, A. J. H. M. Houben, Ronald M. A. Henry, Mitko Veta},
url = {https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10949/109491N/Approximation-of-a-pipeline-of-unsupervised-retina-image-analysis-methods/10.1117/12.2512393.full},
year = {2019},
date = {2019-03-15},
booktitle = {SPIE Medical Imaging},
abstract = {A pipeline of unsupervised image analysis methods for extraction of geometrical features from retinal fundus images has previously been developed. Features related to vessel caliber, tortuosity and bifurcations, have been identified as potential biomarkers for a variety of diseases, including diabetes and Alzheimer’s. The current computationally expensive pipeline takes 24 minutes to process a single image, which impedes implementation in a screening setting. In this work, we approximate the pipeline with a convolutional neural network (CNN) that enables processing of a single image in a few seconds. As an additional benefit, the trained CNN is sensitive to key structures in the retina and can be used as a pretrained network for related disease classification tasks. Our model is based on the ResNet-50 architecture and outputs four biomarkers that describe global properties of the vascular tree in retinal fundus images. Intraclass correlation coefficients between the predictions of the CNN and the results of the pipeline showed strong agreement (0.86 - 0.91) for three of four biomarkers and moderate agreement (0.42) for one biomarker. Class activation maps were created to illustrate the attention of the network. The maps show qualitatively that the activations of the network overlap with the biomarkers of interest, and that the network is able to distinguish venules from arterioles. Moreover, local high and low tortuous regions are clearly identified, confirming that a CNN is sensitive to key structures in the retina.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}

G. Bortsova, F. Dubost, S. Ørting, I. Katramados, L. Hogeweg, L. Thomsen, M. Wille, M. de Bruijne

Deep learning from label proportions for emphysema quantification Inproceedings

International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 768–776, Springer, Cham, 2018.