Segmentation of colon nuclei images using deep learning

Abstract

Precise detection and segmentation of cells in microscopy images is a crucial step in many processes in digital pathology and biomedical applications. Automating these processes remains challenging to this day due to complex morphology of the tissues and various staining methods. Current techniques used in nuclear segmentation, classification and composition prediction frequently face challenges leading to unsatisfactory results. In recent years, the application of advanced deep learning algorithms like U-Net and UNet++ in the field of computational pathology has demonstrated impressive outcomes by providing clinicians with faster and more reliable autonomous identification of diseases. Colorectal cancer (CRC) also known as bowel cancer or colon cancer, is a major healthcare problem that ranks as one of the main causes of cancer related deaths globally. Depending on the location of the lesion, colorectal cancer is diagnosed by sampling sections of the colon that are suspected of developing a tumor. This is usually done during a colonoscopy or sigmoidoscopy. Colorectal cancer detection and metastases determination can be done with various other medical imaging modalities such as CT scans and MRIs. Histopathological analysis of colorectal cancer can be done from tissues taken from a biopsy or surgery. Image segmentation is a fundamental step in computer vision that involves partitioning a digital image into discrete groups of pixels that are referred to as image segments for other tasks like object detection. Microscopy images of colorectal cancer can be very large in size, noisy or include overlapping nuclei or complex morphological tissues which can pose a challenge for image segmentation. Additionally, the complex anatomical structure of the colon and irregular shapes of tumors. To overcome these challenges, it is necessary to develop advanced algorithms and robust preprocessing techniques in image segmentation for clinical applications such as diagnosis and treatment planning of colorectal cancer. This study employed two convolutional neural networks (CNN) based models in order to segment nuclei in the colorectal cancer histopathology images presented in the Lizard dataset. The dataset consisted of 495,179 annotated nuclei across 251 patches of whole-slide images which then preprocessed to produce 1,616 training patches, 346 validation patches, and 47 test patches augmented with flipping, rotataion and color jittering. Both models are aimed to first perform instance segmentation of nuclei and classify them into six categories: neutrophil, epithelial, lymphocyte, plasma, eosinophil, and connective tissue. Results of U-Net showed superior performance with a Dice score of 0,7956, Intersection of Union (IoU) of 0,7213, precision of 0,7824, recall of 0,8087 and F1- score of 0,7952 excelling in the segmentation of the epithelial nuclei class. Although UNet++ showed improved results for specific classes it struggled with infrequent classes such as neutrophils as seen by its lower Dice score of 0,3831 for these nuclei. The small test set size (47 patches) limited the performance evaluation of this study indicating the necessity for larger datasets like TCGA-COAD and implementations of more advanced loss functions such as Tversky to improve model robustness for clinical applications.