Estimating complex immune cell structures by AI tools for survival prediction in advanced melanoma
A conceptual illustration of the HookNet open-source deep learning model
Unique immune cell structures in tumors can boost immunotherapy effects and enhance survival, but testing is labor-intensive and not yet common. New AI-driven methods aim to hasten the standardization and adoption of testing for future melanoma patients.
Researchers from the ECOG-ACRIN Cancer Research Group (ECOG-ACRIN) have applied AI-driven processes for detecting tertiary lymphoid structures (TLS) in thousands of digital images of melanoma tumor tissue, significantly enhancing TLS identification and survival predictions for operable stage III/IV patients. The presence of TLS, a key biomarker for better prognosis and improved survival, is not yet a standard part of patients’ pathology reports, and manual detection is labor-intensive and can be variable. Lead investigators Ahmad A. Tarhini, MD, PhD, and Xuefeng Wang, PhD, will present the new approach at the American Association for Cancer Research 2025 Annual Meeting in Chicago.
“Our efforts reveal the potential of open-source AI tools to transform how we predict survival and immunotherapy benefits by detecting critical immune structures like TLS with unprecedented ease and accuracy,” said Dr. Tarhini, professor and senior member, cutaneous oncology and immunology, at the Moffitt Cancer Center and Research Institute in Tampa, Florida (pictured).
The study retrospectively analyzed thousands of archived digital images coupled with corresponding RNA sequencing data from 376 patients with advanced, high-risk melanoma, linking TLS presence to significantly better overall survival. The cohort had participated in a landmark US cooperative group trial led by ECOG-ACRIN called E1609 that tested immune check point blockade and cytokine therapy in high-risk melanoma (Tarhini A. J Clin Oncol. February 2020).
This analysis found TLS present in 55% of the E1609 cohort and predicted significantly better overall survival than those without TLS (36.23% vs. 29.59% at 5 years), especially in those with more than one TLS (38.04% in >1 TLS vs. 28.65%). TLS density was also significantly prognostic for overall survival (37.77% vs. 28.72% at 5 years for median cutoff). Survival also varied by AJCC stage group, age, sex, treatment type, and tumor ulceration, as shown in AACR Abstract 3358.
“These findings highlight the potential for AI-driven approaches to standardize TLS assessment using low-cost H&E-stained images, with the potential to improve prognostication and stratification within AJCC, and warrant further investigation,” said Dr. Tarhini.
Researchers first applied HookNet-TLS, an open-source deep learning algorithm, to measure TLS and germinal centers (GC) within the E1609 digitized H&E-stained slides. After reviewing the initial results, they retrained the model for better accuracy. They evaluated the prognostic value of TLS scores by correlating the presence of TLS and GC found in the digitized images with normalized TLS counts.
Next, the researchers applied Gigapth Whole-Slide Foundation Model for Digital Pathology feature extraction and investigated the potential in TLS detection in this cohort. Gigapth allowed for enhanced visualization of H&E image tiles through the generation of principal component analysis (PCA).
“Utilizing Gigapth Foundation Model, the generated PCA visualizations appear promising in enhancing TLS and GC detection. These are undergoing further fine-tuning, and the final results will be shared at a future meeting,” said Dr. Wang, chair of biostatistics and bioinformatics at Moffitt Cancer Center (pictured).
This research was supported by a grant from the National Cancer Institute, one of the National Institutes of Health.
"The new survival prediction methods leverage low-cost, easily accessible technologies. They have the potential to speed up TLS testing adoption for high-risk melanoma patients, which could in turn aid discussions between physicians and patients on potential immunotherapy benefits," added Dr. Tarhini.
About Tertiary Lymphoid Structures
Tertiary lymphoid structures (TLS) are clusters of immune cells (T cells, B cells, and dendritic cells) that can sometimes form in tissue outside of the body’s normal lymphatic organs, such as the lymph nodes or the spleen. They form in response to chronic inflammation or cancer. Not all cancerous tumors will develop TLS, but when they do, their presence can enhance immune cell infiltration and improve patient outcomes.
About HookNet
The developers of HookNet have made it publicly available by releasing the source code via the Grand Challenge platform for end-to-end development of machine learning solutions in biomedical imaging (Rijthoven M. Med Image Anal. February 2021). More recently, HookNet-TLS was introduced for the automated detection of TLS and germinal centers (Rijthoven M. Comm Nature. January 2024).
About ECOG-ACRIN
The ECOG-ACRIN Cancer Research Group (ECOG-ACRIN) is a membership-based scientific organization that includes approximately 1,400 cancer centers and community hospitals as members, and about 21,000 researchers and advocates in total. The group conducts studies spanning the cancer care spectrum, from early detection to management of advanced disease. To learn more, visit www.ecog-acrin.org and follow us on X @EAonc, Facebook, LinkedIn, YouTube, and Instagram.