Automated Bladder Cancer Screening with Deep Learning Algorithms
Ameya Rajeev, Abhinav Kakkera, Aqib Nawaz Shaik, Gagan Girish, David Ahn & Joycelyn Cheng
Lay Summary:
We compared the performance of two different methods of how each would model would segment cell images and diagnose it into positive, negative, suspicious, and atypical cells. We found that a newer deep learning method, U-net expectedly performed better than the traditional deep learning method.
Abstract:
Bladder cancer has one of the highest recurrence rates of any cancer, requiring lifelong monitoring and screening. Distinguishing between benign, atypical, and malignant cells is difficult due to overlapping cell morphology. While the nuclear-to-cytoplasmic (N/C) ratio is an important indicator used in screening, manual evaluation is extremely laborious. This study presents a U-Net-based deep-learning model that segments urothelial cells to calculate their N/C ratios used for diagnosis. We compared the deep learning approach with traditional methods and evaluated its predictive value of the N/C ratio through comparison with patient diagnostic categories to determine its potential for automated screening. Exactly 200 urothelial cell images with ground-truth masks were used to train and validate both methods. For the Otsu baseline, images were converted to grayscale, Gaussian-smoothed, and inverted so nuclei had higher intensity values. Three-class Multi-Otsu thresholding assigned pixels to backgrounds, cytoplasm, and the nucleus followed by morphological filling of small holes. For U-Net, images were resized to 256 x 256 pixels and normalized. U-Net was trained to segment background, cytoplasm, and nucleus using equally weighted cross-entropy and Dice losses. The evaluation dataset consisted of 100 cell images labeled into four categories- positive, negative, atypical, and suspicious. Segmentation performance was evaluated using Spearman’s correlation coefficient (SCC), and a box plot represented how well each method diagnosed the test set into four categories. U-Net achieved a SCC of 0.84, while Otsu’s performed at an SCC of 0.76. Similarly, U-Net produced a box plot closer to the ground-truth plot, than Otsu’s did. These results are statistically significant, and indicate that U-Net was able to diagnose and segment urothelial cell images with greater accuracy than Otsu’s method. This was expected, as Otsu’s is a traditional method, while U-Net is a recent architecture stated to perform well in segmentation tasks.
Q&A:
Bios: Ameya Rajeev,Abhinav Kakkera,Aqib Nawaz Shaik,Gagan Girish,David Ahn,Joycelyn Cheng
Program Track: Skills Development
GitHub Username:
ameyar880 -Ameya Rajeev
abhikakkera -Abhinav Kakkera
aqibshaik-dev -Aqib Nawaz Shaik
gagan_g -Gagan Girish
dahn1863 -David Ahn
JoyCheng3 -Joycelyn Cheng
What was your favorite seminar? Why?
My favorite seminar was the one on clinical perspective of digital and computational pathology by Mark Zarella. I really enjoyed this seminar because I was able to gain a much better understanding of the impact technology can have on pathology, and how it can be used in the real world. -Ameya Rajeev
My favorite seminar was the last seminar. I liked this seminar because it talked about presenting our groups work, and it helped me better present our findings that we concluded. I think this seminar was the most important to me, because I have a fear of presenting, and -Abhinav Kakkera
My favorite seminar was the one done by Elijah Renner and Ishan Ramrakhiani because I was very interested in their startup work, as well as Elijah’s work with medical encoders. I also liked the last seminar where we learned essential writing scores for formal research. -Aqib Nawaz Shaik
My favorite seminar was the last seminar because it helped me learn about the difference between a discussion and the results section. Also taught me on how to present engagingly. -Gagan Girish
My favorite seminar was Manuscript Preparation by Lou because I enjoyed the way he presented and how he conveyed the information. It was a very memorable experience to me and it has stuck with me ever since. -David Ahn
I really liked the seminar about the PREDICT model for breast cancer. I thought it was interesting how it can predict breast cancer outcomes, and it was inspiring to see how widely the model is used by both clinicians and patients. It was also interesting to see how large language models, such as ChatGPT, was able to follow the clinical guidelines very well. -Joycelyn Cheng
If you were to summarize your summer internship experience in one sentence, what would it be?
This summer, I gained experience applying AI and deep learning to real-world medical data while developing a potential approach to improve bladder cancer screening. -Ameya Rajeev
This summer internship was enlightening, it taught me how to collaborate with peers from different timezones, different backgrounds, and different interests. -Abhinav Kakkera
My summer internship was very fun and informative, I was able to strengthen my machine learning skills while also being able to work with an excellent group of people! -Aqib Nawaz Shaik
My summer was filled with lots of fun working with my team and learning new things along the way. It taught me how to communicate with others who were different ages, had different backgrounds, and were in different time zones. -Gagan Girish
A major learning experience and an opportunity to collaborate with others from differing backgrounds, interests, and parts of the world. -David Ahn
I thought my summer internship experience was fun and insightful, I was able to collaborate and meet with high schoolers across the country and was able to learn a lot from my project and the seminars. -Joycelyn Cheng
Blog Post
Team 6 2026
Skills Development Track
Automated Bladder Cancer Screening with Deep Learning Algorithms
Ameya Rajeev, Gagan Girish, Aqib Nawaz Shaik, Abhi Kakkera, David Ahn, Joycelyn Cheng
I. Introduction
Bladder cancer is the most common type of cancer found in the urinary tract, with approximately 74,000 new cases diagnosed annually in the United States [1]. Bladder cancer also has some of the highest recurrence rates of any cancer and requires lifelong monitoring and
screening [2]. An important indicator used in bladder cancer screening is the nuclear to cytoplasmic ratio, as cancerous cells have an increased nuclear size [3]. Manual screening of urine cytology specimens is time-consuming and laborious. Additionally, mixed cell types, debris, and variable stain/focus make boundaries unclear. This causes inconsistent N/C ratio estimates which hurts reproducibility. Here, we propose training a deep learning image segmentation model called U-Net, to segment the nucleus and cytoplasm while also computing the per cell N/C ratio. We benchmarked U-Net against interpretable segmentation baselines, including Otsu thresholding, and compared the resulting per-cell N/C ratio distributions across four diagnostic categories: negative, atypical, positive, and suspicious. Although models were evaluated on a limited dataset, the findings demonstrate the potential of automated image segmentation to generate objective and reproducible N/C ratio measurements in urine cytology.
II. Methods
1. Urothelial Cell Dataset
Cell Images: Use a pickle dataset containing 200+ urothelial cell images with ground-truth segmentation masks for the nucleus and cytoplasm.
Specimen Dataset: Analyze approximately 25 cells per patient with associated diagnostic categories to evaluate the N/C ratio as a potential biomarker.
2. Image Segmentation:
Otsu Thresholding: Applies multi-level, intensity-based thresholding as a traditional segmentation approach that does not require training.
UNET: Train a UNET deep learning model to automatically segment the nucleus and cytoplasm.
Model Evaluation: Compare segmentation results against ground-truth masks using metrics such as Dice coefficient and IoU.
3. Quantitative Analysis:
N/C Ratio: Calculate the nuclear-to-cytoplasmic ratio from segmented cell regions.
Diagnostic Analysis: Compare N/C ratios across patient diagnostic categories to evaluate their predictive value for bladder cancer.
4. Computational Tools:
Python: Use Python for data processing, image analysis, model development, and statistical analysis.
Libraries: Utilize PyTorch, OpenCV, NumPy, scikit-learn, and other image-processing libraries.
GPUs: Use GPU acceleration to train and evaluate the UNET model efficiently.
III. Results
UNET showed a Spearman correlation coefficient (SCC) higher than Otsu thresholding on our dataset, as UNET achieved a score of 0.844 compared to Otsu with 0.765. Additionally, UNET achieved a boxplot much more accurate to the ground truth plot compared to the Otsu method, as shown in Figure 1.
Figure. 1
{width=”5.796875546806649in”
height=”1.5712073490813647in”}
IV. Discussion
Our findings suggest that UNET has the potential for automated urothelial cell segmentation and N/C ratio measurement. Specifically, the segmentation model achieved a Spearman Correlation Coefficient of 0.84, indicating a strong positive correlation between automated and manual N/C ratio measurements. This high correlation shows that UNET is more effective in capturing and segmenting the nucleus and the cytoplasm of urothelial cells than the Otsu method. This can also be seen in the box plots, as there is a strong overlap in medians and interquartile ranges between automated and manual distributions showing that the model prediction closely matches human annotations. Accurate segmentation is important because errors in identifying the nucleus or cytoplasm can affect N/C ratio calculations. Automated segmentation could make these measurements more consistent and reproducible, potentially supporting quantitative analysis of urine cytology. However, our results have some limitations. For example, our dataset was relatively small and may not capture the full variation in clinical urine cytology images. Also, N/C ratio alone may not capture the full complexity of bladder cancer diagnosis, which depends on multiple cellular and morphological features. Image quality challenges including overlapping cells, debris and low contrast, can also lead to segmentation errors and affect N/C ratio measurements. In the future using a larger and more diverse dataset while considering additional morphological features alongside N/C ratio will determine whether the model can generalize to new specimens.
V. What We Learned
This project taught us how U-Net's encoder-decoder structure segments medical images, and why skip connections matter for preserving nuclear boundaries. Starting from a ResNet-50 encoder pretrained on ImageNet allowed the model to learn effectively from only a few hundred cells, and combining Dice loss with cross-entropy kept the model from ignoring nuclei, which occupy a small fraction of the pixels in each image. We also learned how much a simple baseline contributes. Otsu thresholding reached a mean Dice coefficient of 75.7% but produced no nuclear segmentation in 9 of 200 cells, because it separates regions purely by pixel intensity and breaks down on pale or heavily stained nuclei. Identifying that specific failure explained why the deep learning approach performed better, rather than leaving us to simply assert that it did, and it changed how we think about baselines: they are diagnostic tools that reveal what a model actually needs to handle, not just numbers to beat. Finally, we learned that accurate segmentation is only half the problem. Converting per-cell masks into N/C ratios and comparing them across diagnostic categories showed us what it takes for a model output to become a candidate biomarker, and working with roughly 200 training cells taught us to interpret our own results cautiously, since strong performance on a small dataset does not establish that a model will generalize to new specimens.
VI. Conclusion
During our time working on this project, we fulfilled the objective of learning more about automated bladder cancer screening by using deep learning algorithms and traditional methods to combat automated diagnosis. By comparing the success rate of segmentation using both Otsu’s method and UNET we managed to learn more about how both segmentations work and their strengths and weaknesses. We learned that UNET had higher segmentation accuracy than Otsu’s Method for urothelial cell segmentation, where UNET had an SCC of 0.844 while Otsu’s method had an SCC of 0.765. Having an accurate segmentation model would allow us to quickly calculate the N/C ratio for accurate diagnosis. The N/C ratio we obtain has potential to be used for automated bladder cancer screening. There is room for improvement as if we had a larger and more diverse dataset, we could validate the approach and improve reliability. In the future we hope to address these limitations by expanding and augmenting on the dataset, doing so to improve the accuracy metrics of our models.
VII. References
[1]: Kamat AM, Hahn NM, Efstathiou JA, Lerner SP, Malmström P-U, Choi W, Guo CC, Lotan Y, Kassouf W, Bladder cancer, The Lancet, 2016 Dec 3,
[https://doi.org/10.1016/S0140-6736(16)30512-8]{.underline}
[2]: Dyrskjøt L, Hansel DE, Efstathiou JA, Knowles MA, Galsky MD, Teoh J, Theodorescu D, Bladder cancer, Nature Reviews Disease Primers, 2023,
[https://doi.org/10.1038/s41572-023-00468-9]{.underline}
[3]: Bisharah S, Raap M, Abbas M, Urine Cytological Diagnostics: Possibilities and
Limitations—A 25-Year Review and Overview at Hannover Medical School, Clinical Practice, 2025, [https://doi.org/10.3390/clinpract15120234]{.underline}