Self Supervised Vision Transformers for Learning 3D Colorectal Cancer Tissue Architecture from Open-Top Light Sheet Microscopy

Prahlad Uchila, Gautham Agilan, Anirudh Maheshwari & Animesh Page

Lay Summary:

To find the root cause of cancer, docters often cut out tissue from your body and examine it. However, sometimes the thin slice doesn't take into account the whole cancer block so using a 3d scanner, we created a machine learning model to learn and understand the tissue.

Abstract:

Accurate categorization of tumor morphology is central to predicting the diagnosis and prognostication of colorectal cancer (CRC). However, conventional histopathology employs two-dimensional (2D) analysis of tissue samples that ignores crucial spatial context inherent in the volume itself, such as gland architecture of tumor budding; tissue features that are associated with CRC. Now, non-destructive imaging techniques such as Open-top light-sheet (OTLS) microscopy allow full 3D scanning of tumor samples, raising the question of whether volumetric self-supervised learning (SSL) confers direct advantages to categorization accuracy of 2D structure and across depth. In this study, we present a comparison between three SSL trained masked autoencoder (MAE) foundation models similar in all aspects except for input dimensionality- single-slice 2D, channel stacked 2.5D, and volumetric 3D. Analysis across six tissue classes with 5-fold class-based cross-validation revealed that the 2D model achieved the highest performance (83.50% average balanced accuracy, 66.25% macro F1), outperforming the 3D model (73.96%, 57.91%), and the less successful 2.5D (49.79%, 38.72%). However, the 3D model reached a lower flicker rate between neighboring Z-slices (1.36% vs. 1.79%), suggesting that spatial context was increased, improving consistency of structure categorization across depth. The results indicate that although a full volumetric model reaches a noisier, more limited accuracy level than a standard 2D MAE, it retains a level of spatial consistency.

Q&A:

Bios: Prahlad Uchila,Gautham Agilan,Anirudh Maheshwari,Animesh Page

Program Track: Advanced Research & Mentor

GitHub Username:

PUchila -Prahlad Uchila

Gautham-A10 -Gautham Agilan

maheshwari-aniruddh -Anirudh Maheshwari

AnimeshPage -Animesh Page

What was your favorite seminar? Why?

My favorite seminar was Thomas Cantore’s presentation because his company was making real, visible change in breast cancer and his presentation was very interesting. -Prahlad Uchila

My favorite seminar was Anvith Kakkera’s on using AI for thyroid cytopathology. I have been following his work for a while and he’s been a mentor of mine in the past, so it was nice to see its development. -Gautham Agilan

Mark Zarell, he seemed like the most cool and realistic person. He was very fun and interesting to listen to and i learned a lot from him. -Anirudh Maheshwari

The first one, since it introduced me to crucial concepts in pathology -Animesh Page

If you were to summarize your summer internship experience in one sentence, what would it be?