Academics | The Hong Kong University of Science and Technology

Hebei Medical University Fourth Hospital and the Joint Intelligent Medical Data Experiment Center of the Hong Kong University of Science and Technology (Guangzhou) officially launched a joint research project in September 2024 to establish a multi-modal deep learning model based on HE images and HER2 immunohistochemistry images for predicting the prognosis of metastatic breast cancer. This project integrates multi-modal data and applies deep learning technology to provide a more accurate and efficient method for breast cancer prognosis prediction, demonstrating significant clinical value and advancing the integration of artificial intelligence with medicine.

Breast cancer is one of the most common malignant tumors in women globally, and HER2 is an important prognostic and predictive indicator for primary and metastatic breast cancer. Recently, novel antibody-drug conjugates (ADCs) have shown significant therapeutic effects in patients with late-stage HER2-low breast cancer, expanding treatment options beyond HER2-positive patients. However, HER2 expression in breast cancer exhibits spatial and temporal heterogeneity, and the discordance in HER2 expression between primary and metastatic lesions affects the efficacy and prognosis of patients receiving novel ADCs treatment. Accurate prognosis prediction for these patients is crucial for individualized treatment. Traditional prognosis prediction relies on clinical and pathological features, but these methods have limitations in accuracy and flexibility. The collaborative laboratory adopts a multi-modal deep learning approach, combining information from HE images and HER2 immunohistochemistry images to enhance the accuracy of predicting the prognosis of metastatic breast cancer. The core advantage of multi-modal deep learning models lies in their ability to integrate different types of data, automatically learn and extract key features from each modality, thus generating predictions with greater clinical relevance.

This project not only addresses key challenges in breast cancer prognosis prediction but also advances the development of intelligent healthcare through the integration of multi-modal data and deep learning technology. For both parties, this collaboration will enhance diagnostic efficiency and research impact, with the potential to achieve breakthroughs in more fields in the future, driving the mutual progress of medicine and artificial intelligence technology.