IntroductionBreast cancer originates in breast tissue due to abnormal cell growth and mutations, forming tumors. While it can affect both genders, it predominantly occurs in women. Some common symptoms of breast cancer include: 1. A protrusion or denseness in the chest or armpit region. 2. Alterations in the breast’s dimensions or contour. 3. Swelling, redness, or dimpling of the skin on the breast. 4. Nipple changes, such as inversion, scaling, or discharge. 5. Persistent pain in the breast or armpit. Nonetheless, it’s crucial to emphasize that not all breast lumps or alterations signify cancer. Many breast conditions are benign, meaning they are not cancerous1. Routine self-checks of the breasts, professional breast examinations, and mammograms are essential for the early identification of breast cancer. If a lump or suspicious change is found, further tests such as biopsies may be conducted to determine whether it is cancerous or not. The selection of breast cancer treatments considers factors like cancer stage, tumor size, and the patient’s health. Common treatment modalities encompass surgery (e.g., lumpectomy or mastectomy), radiation treatment, cytotoxic therapy, and precision therapy, and hormone therapy. The choice of treatment is made by a healthcare team, considering the specific characteristics of the cancer and the patient’s preferences. Breast cancer awareness and early detection have improved over the years, leading to higher survival rates. It’s important for individuals, particularly women, to be aware of the signs and symptoms of breast cancer, and to undergo regular screenings as recommended by healthcare professionals. If concerns about breast cancer arise, consulting a healthcare provider is advisable for proper evaluation and guidance2.Magnetic Resonance Imaging (MRI) of the breast employs strong magnets and radio waves to generate intricate images of breast tissue for diagnosis. This is often implemented as a supplemental tool in breast cancer detection and evaluation, particularly in certain situations where mammography or ultrasound may not provide sufficient information.Breast MRI can provide additional information about the extent of breast cancer, evaluate the response to treatment, and help in surgical planning. Here are some key points about breast cancer MRI:1.Procedure: In a breast MRI, the individual reclines face down on a cushioned table, which then moves into the MRI apparatus. This machine utilizes a robust magnetic field and radio waves to produce cross-sectional breast images. In some cases, a contrast agent (usually gadolinium) may be injected intravenously to enhance the visualization of blood vessels and abnormalities.2.Indications: Breast MRI is typically used in specific situations, such as:Screening high-risk individuals: It might be suggested for people at an elevated risk of breast cancer, like those with a significant familial background or particular genetic alterations. (e.g., BRCA1 or BRCA2).Evaluating extent of disease: Breast MRI offers precise data about tumor size, location, and potential effects on nearby lymph nodes or the chest wall.Assessing treatment response: It is useful in monitoring the response to neoadjuvant chemotherapy (chemotherapy given before surgery) and evaluating residual disease after treatment.Surgical planning: Breast MRI can help guide surgical planning, especially in cases where breast-conserving surgery (lumpectomy) is being considered.3.Benefits and limitations: Breast MRI3 has several advantages, including its high sensitivity in detecting breast cancer, especially in dense breast tissue. It is also useful in identifying additional cancerous lesions in the breast. Nonetheless, it has the potential to generate inaccurate positive outcomes, which could prompt additional, unwarranted examinations or biopsies. Additionally, it carries a higher cost and demands more time compared to alternative imaging methods. It may not be a suitable choice for individuals with specific medical conditions, including claustrophobia, metallic implants, or severe kidney disease, owing to the use of a contrast agent.4.Combined with other imaging modalities: Breast MRI is often employed in tandem with mammography and ultrasound to deliver a thorough evaluation of breast health. Every imaging method possesses its unique advantages and constraints, and the decision regarding which method or combination to employ is contingent upon the particular clinical context. Seeking guidance from a medical expert is essential, such as a radiologist or oncologist, to determine if breast MRI is appropriate for the individual case. They will consider various factors, including medical history, risk factors, and specific concerns related to breast cancer diagnosis or treatment. Radiologists play a crucial role in studying breast cancer MRI images. They are specialized doctors who interpret and analyze medical images to diagnose diseases, including breast cancer. Here’s an overview of how radiologists study breast cancer MRI:1.Image review: The radiologist begins by reviewing the breast MRI images in detail. They examine the images slice by slice to assess the size, shape, and characteristics of any abnormalities, such as tumors or suspicious lesions. They also evaluate the surrounding breast tissue, lymph nodes, and nearby structures for any signs of cancer spread.2.Image interpretation: Radiologists analyze various aspects of the MRI images to make diagnostic assessments. This includes assessing the enhancement patterns of lesions using contrast agents. Cancerous lesions often exhibit different enhancement characteristics compared to normal breast tissue, such as increased blood flow or abnormal uptake of contrast material.3.Comparison with other imaging modalities: The radiologist may compare the breast MRI findings with previous mammograms, ultrasounds, or other imaging studies if available. This helps provide additional information and context to make more accurate assessments.4.Quantitative analysis: Radiologists may utilize computer-aided detection (CAD) systems or other software tools to assist in quantitative analysis. These tools can help identify potential areas of concern, measure lesion size, and provide objective data to aid in diagnosis and treatment planning.5.Report generation: After studying the MRI images, the radiologist prepares a detailed report that summarizes their findings and provides a diagnostic impression. The report typically includes a description of any abnormalities, their location, size, enhancement characteristics, and other relevant details. This report is shared with the referring healthcare provider, such as the oncologist or surgeon, who will use it to guide patient management.6.Consultation and collaboration4: Radiologists often collaborate with other members of the healthcare team, including oncologists, surgeons, and pathologists, to discuss and confirm the diagnosis. They may participate in multidisciplinary tumor boards or meetings to ensure comprehensive evaluation and decision-making for individual patients.It’s important to note that radiologists undergo extensive training and specialization in interpreting medical images, including breast MRI. Their expertise, along with advanced imaging technology and tools, helps ensure accurate diagnosis and appropriate management of breast cancer.Artificial Intelligence (AI) has made significant contributions to the field of breast cancer in various ways. Here are some ways in which AI is helping in the fight against breast cancer:1.Early detection: AI algorithms can analyze mammograms and other imaging scans to identify breast cancer in its initial phases. These algorithms can assist radiologists in interpreting images by highlighting potential areas of concern, improving the accuracy and efficiency of diagnosis.2.Image interpretation5,6: AI can aid in the interpretation of breast images, such as mammograms, ultrasounds, and MRIs. By analyzing large amounts of data, AI algorithms can assist in detecting subtle patterns or abnormalities that may be missed by human observers, thereby improving diagnostic accuracy.3.Risk assessment: Artificial intelligence models can aid in evaluating the likelihood of an individual developing breast cancer by examining a range of factors, including familial medical history, genetic indicators, and lifestyle information. This information can be utilized for the identification of individuals at elevated risk individuals for whom it may be advantageous more frequent screening or preventive measures.4.Treatment planning: AI can assist in treatment planning by analyzing patient data, medical records, and treatment outcomes to provide personalized recommendations. It can help oncologists determine the most appropriate treatment options, predict treatment response, and guide decision-making based on historical data and patterns.5.Predictive modeling7: AI algorithms can analyze large datasets to identify patterns and predict outcomes in breast cancer patients. This can help in predicting disease progression, recurrence, and response to different treatments, enabling healthcare providers to make more informed decisions about patient management.6.Drug discovery and development: AI has the potential to facilitate the exploration and creation of novel medications for breast cancer. Through the analysis of extensive genomic and molecular data, AI algorithms can pinpoint potential drug targets and contribute to the design of more efficacious therapies.7.Patient support and education: AI-powered chatbots and virtual assistants can provide personalized information, support, and resources to breast cancer patients. These tools can answer common questions, offer emotional support, and connect patients with relevant services and resources.It’s crucial to emphasize that although AI holds potential in the realm of breast cancer detection and treatment, its purpose is not to substitute for healthcare professionals. Instead, it serves as a valuable tool to enhance their expertise and enhance patient outcomes through the utilization of data analysis and pattern recognition capabilities8,9.Unlike existing studies that rely solely on handcrafted radiomic features or basic CNN architectures, our approach uniquely integrates DCE-MRI images with the advanced InceptionV3 deep learning model, enriched with clinically relevant features such as entropy, centroid, and sphericity to improve pCR prediction in breast cancer patients.Problem statementWhen the patient detects with Breast cancer disease, there are many tests are available to know the depth about the cancer like Ultrasound, MRI, Genetic Testing, Pathologic Evaluation of Surgical Specimen, Biopsy Before and After Neoadjuvant Chemotherapy etc.. During the initial stage, it is quite difficult to determine whether patient will go for surgery or chemotherapy. If the doctor suggest to go for chemotherapay, then there might some cycle of chemotherapy. After each time period of chmotherarpy doctor generally wants to know the pCR (pathologic complete response) which refers to the absence of any invasive cancer cells in the breast tissue and lymph nodes after neoadjuvant (preoperative) chemotherapy. Now the question is if after doing multiple iteration of chemotherapy, still the doctor insists to undergo a surgery, then it will be completely dissapointed to the patient. So we need to find out a way on which we can able provide probability of success after the chemotherapy from initial MRI image data. By looking at the MRI images, we need to predict that after chemotherapy, the pathologic complete response to the patientMaterials and methodsDataset descriptionThe data is used from the10I-SPY 2 (officially named the Exploration of Serial Examinations, is an ongoing and extensive clinical trial (NCT01042379)) breast MRI database. The data focuses on women aged 18 and older who have locally advanced breast cancer, characterized by a tumor size of 2.5 cm or larger, and no distant metastases. For those patients who are at a high risk of breast cancer recurrence, the study employs dynamic randomization. Dynamic randomization means that these patients are not randomly assigned to either the standard NAC treatment or an experimental drug arm at the beginning of the study. Instead, the randomization plan is adjusted throughout the course of the study based on observed changes in tumor size, which are measured through DCE-MRI at different time points during NAC. Data was retrieved from the ISPY2 database, with each patient’s information organized into individual folders. Each of these folders further encompassed four subfolders, each corresponding to a different time frame. Within these time frame subfolders, breast cancer images were stored in separate folders, each categorized by different types, including Scout, Waterideal, Fatideal, ph series vibrant, and Volser. Within the Volser category, there were various subtypes such as SER, PE2, PE5, and DCE. For our proposed approach, we specifically selected data from the DCE (Dynamic Contrast-Enhanced) subtype within the Volser category.Features such as entropy, intensity, and perimeter were chosen due to their clinical relevance in capturing tumor heterogeneity, aggressiveness, and morphological irregularities. Compared to traditional radiomic features or clinical biomarkers like Ki-67 or ER/PR status, these features offer higher spatial sensitivity and are directly extractable from imaging, allowing for non-invasive yet interpretable diagnostic insights. Entropy was selected as it captures the heterogeneity within tumor tissues, a known indicator of aggressiveness in breast cancer. Centroid reflects the spatial distribution of lesions, offering insight into morphological changes, while sphericity and diameter relate to tumor shape and size, which are clinically correlated with treatment response.In our research project, we have examined data from 255 patients. The DCE folder for each time frame of every patient contains up to 700 .dcm files. The entire dataset encompasses a substantial collection of approximately 150,000 image slices derived from these 255 patients. Each patient folder is labeled as either pCR or No pCR.Proposed approachIn this study, we have considered multiple breast cancer MRI images, all formatted in the commonly used DCM format, which is standard for medical imaging data. The main goal of the study aim to enhance the analysis and comprehension of breast cancer patterns present in these MRI images and to classify them based on Pathological Complete Response (pCR). To accomplish this, we have established a systematic methodology.As per the Fig. 1 the first step involves taking the data input from the DCM files. This process entails reading and loading the images into our computational environment, ensuring that we have access to all the relevant data necessary for our analysis.Subsequently, We will conduct data preprocessing on the MRI images, which is intended to improve the data’s quality and consistency. This process helps mitigate noise and artifacts that might otherwise negatively impact the accuracy of our findings. Data preprocessing encompasses a range of tasks, including normalization, resizing, and filtering, as well as image registration, intensity correction, resampling, and enhancements.Fig. 1Proposed approach block diagram.Full size imageOnce the data preprocessing is completed, our focus will shift towards feature extraction through feature engineering. Extracting relevant features from the breast cancer MRI images is crucial for understanding the underlying patterns and distinguishing between different classes of images. Several feature extraction techniques will be employed to capture various aspects of the images, potentially encompassing texture, shape, and intensity-based features.Following the acquisition of the extracted features, our next step involves the utilization of the multiple deep learning model including GoogleNet model (Inception V3), a deep learning architecture widely recognized for its proficiency in image classification assignments. Through the application of the GoogleNet model, our objective is to exploit its robust capabilities in identifying intricate patterns within the MRI images, thereby simplifying the detection of potential cancerous areas.Finally, we will conduct a validation process to assess the performance of our model. The validation step will involve splitting the dataset into training and testing sets to evaluate how well the GoogleNet model generalizes to unseen data. This process will help us ascertain the reliability and accuracy of our approach in detecting breast cancer from MRI images.Data pre-processing and feature engineeringFor analytical purposes, adjustments were made to the volumes and segmentations of the images to ensure a uniform slice size of 256 × 256. To enhance data quality, chest wall and air masks were employed to mask out background and chest regions. This effectively removed specific areas, improving the overall data quality. Additionally, contrast limited adaptive histogram normalization was individually applied to each phase of the MRI scans to enhance contrast and normalize volumes.The pixel data was extracted from each DICOM file and rescaled to a specific range, such as [0, 255], to augment image contrast. Normalization of pixel values to [0, 1] was performed to ensure consistent image intensity values across different images. Windowing techniques were applied to enhance visualization of specific tissues or structures. Subsequently, image resizing to a common size (256 x 256) was carried out to ensure uniform dimensions for all images.As per Fig. 2 Alignment of all images was achieved through proper image registration, reducing distortions caused by patient movement or different imaging protocols. Image intensity normalization was applied to eliminate variations due to scanner settings or lighting conditions. Noise reduction techniques were implemented to enhance image quality and reduce unwanted signal interference. Image resampling was conducted to achieve consistent spatial resolution across all images, facilitating subsequent analysis. Various image enhancement techniques were employed to improve the visibility of relevant features.Fig. 2Data pre-processing: open image, segmented image, preprocessed image.Full size imageThe application of the Otsu thresholding technique is employed to transform the preprocessed image into a binary representation. This procedure effectively separates background from foreground pixels, designating lesion regions of interest as foreground pixels, while setting the background to zero. Morphological operations are then applied to binary images, enhancing the portrayal of lesion regions. Dilation expands regions of interest to improve visibility, while erosion reduces region size and eliminates small noise areas. Opening and closing operations further refine the binary image, contributing to noise removal and structure preservation within lesion regions.After this refinement, a contour detection technique is utilized to pinpoint potential tumor boundaries in DCM images for each patient. These contours define regions of interest and serve as the foundation for extracting features to be utilized in the model, in conjunction with pixel information. Contours, representing the boundaries of continuous bright regions in the binary mask, are then identified. These contours offer a structural depiction of potential abnormalities or distinct structures within the breast tissue. The contours contribute to subsequent analyses, providing valuable information about the size and shape of highlighted areas. To gain insights into the distribution of pixel intensities within the entire image, The mean intensity, representing the average pixel intensity value across the entire image, is computed. This metric provides a summary of the overall brightness or darkness of the image.a histogram is computed. A mask covering the entire image facilitates this process. The histogram, normalized to create a probability distribution, is then leveraged to calculate the entropy. Entropy serves as a metric of the randomness or disorder in pixel intensity distribution, providing a measure of complexity in the image. Some sample contour images are available in Fig. 3 for the references.Fig. 3Extracted contour images.Full size imageIn Table 1, we calculated the perilesional features: Total Area, Total Perimeter,Contour with Maximum Area. These results provide insights into the perilesional characteristics of the studied patients, offering valuable quantitative information about the regions surrounding the lesions.Table 1 Perilesional feature analysis- area, perimeter with max. contour area.Full size tableIn Table 2, the intralesional features: Total Entropy, Mean and Max intensity–comprehensively capture the structural and intensity characteristics of Breast cancer MRI Dicom images. These features offer quantitative insights into the spatial properties, complexity, and overall brightness of regions of interest, providing a foundation for subsequent stages of analysis and aiding in the diagnostic process.Table 2 Quantitative intralesional feature analysis based on DCE MRI parameters for selected patients.Full size tableAs per the Tables 1 and 2, by performing region-wise analysis, medical professionals and researchers can gain valuable insights into the characteristics and properties of different structures or anomalies within the DICOM image. This information can be crucial for diagnosing diseases, planning treatments, monitoring progress, and conducting various studies related to medical imaging and healthcare. The output DataFrame generated by the code provides a concise summary of these parameters for each identified region, facilitating further analysis and interpretation of the DICOM image.Model experimentsThere are a total of 255 images in the dataset, with 143 images labeled as pCR 1 and 112 images labeled as pCR 0. For the training dataset, 206 images were selected, while the test set comprises 49 images. Both the training and test datasets maintain an equal ratio of pCR and non-pCR counts. These datasets are used across various models including ResNet, VGGNet, DenseNet, CNN, and GoogleNet. Inter-annotator reliability was assessed using Cohen’s Kappa, yielding a value of 0.82, indicating substantial agreement between radiologists. To reduce dependence on fully annotated data in future work, we plan to explore semi-supervised learning frameworks such as pseudo-labeling and weakly supervised segmentation using limited ground truth annotations.Explored diverse models in the pursuit of optimizing clinical research outcomes. ResNet11, with its deep architecture, proves conducive to training intricate models, enabling the extraction of hierarchical features vital for capturing complex structures in breast cancer images. VGGNet12 excels in capturing nuanced details, offering a uniform architecture that simplifies model interpretation and visualization, contributing to a deeper understanding of layer impact in breast cancer detection. Leveraging feature reuse and information flow, DenseNet emerges as beneficial in medical imaging, enhancing the model’s capacity to discern subtle patterns and textures crucial for breast cancer diagnosis. CNNs13,14, widely employed in medical image analysis, showcase versatility in learning hierarchical features and patterns, making them adept at identifying abnormalities within diverse datasets. While a standard 80/20 train-test split was employed, future iterations will include k-fold cross-validation and multi-center datasets to improve model robustness. Additionally, domain adaptation techniques such as adversarial training could help generalize the model across centers with different imaging protocols.To address class imbalance and ambiguous feature patterns, we applied focal loss and experimented with cost-sensitive learning. Data augmentation techniques including rotation, contrast enhancement, and Gaussian noise were also employed to enhance variability and mitigate overfitting. As per the figure (Fig. 4), The Inception-v3 architecture is an extension and improvement of the original Inception (GoogLeNet) architecture. It was introduced as part of the ongoing efforts to enhance15 deep convolutional neural networks (CNNs) for diverse computer vision assignments, such as image categorization and object detection. The Inception-v3 architecture builds upon the ideas of its predecessors (Inception-v1, v2, and v3) while incorporating new design principles to improve performance and efficiency.Fig. 4Inception V3 architecture diagram.Full size imageValidation techniquesThe combination of MRI (Magnetic Resonance Imaging) data with PCR (Polymerase Chain Reaction) as the dependent variable in this classification model holds significant importance in the context of breast cancer diagnosis. MRI is a powerful imaging technique known for its high sensitivity in detecting breast abnormalities16, especially in cases where mammography may fall short, such as dense breast tissue or early-stage cancers. This fusion of imaging and molecular biology techniques enhances diagnostic accuracy and reliability.The choice of evaluation metrics, including Accuracy, AUC, Sensitivity, Specificity, and F1 Score, is crucial in this context because it directly impacts patient outcomes. High Accuracy ensures correct predictions, reducing the risk of missing patients who need medical attention. Sensitivity (Recall) is particularly important as It assesses the model’s capacity to accurately recognize individuals with breast cancer, ensuring that potential cases are not overlooked. A high AUC and a well-defined ROC curve illustrate the model’s capacity to differentiate between benign and malignant cases, which is vital for effective treatment planning and intervention decisions.Furthermore, the F1 Score strikes a balance between Precision and Recall, helping to minimize both false positives and false negatives. In breast cancer diagnosis, false negatives can lead to delayed treatment for individuals with cancer, while false positives can cause unnecessary stress and medical procedures for those without the disease. Therefore, these evaluation metrics collectively ensure that the model optimizes the detection of breast cancer and the accuracy of its predictions, ultimately improving patient outcomes and healthcare efficiency.ResultsThe core of our methodology involves the utilization of InceptionV3 backbones in conjunction with DICOM (Digital Imaging and Communications in Medicine) images. These images have been meticulously preprocessed to ensure that they are in a suitable format for analysis. InceptionV3 is a well-established deep learning architecture known for its effectiveness in various computer vision tasks. To accelerate the training process and make efficient use of computational resources, we employ a distributed data parallelism approach. This parallel processing significantly reduces training time and enhances overall efficiency. Our proposed approach not only emphasizes the significance of consistent hyper-parameter settings but also incorporates a diverse set of layers, including BatchNormalization, Dropout, Activation, MaxPooling, and AveragePooling, within the neural network architecture. These layers collectively contribute to the model’s capacity for feature extraction, generalization, and computational efficiency, ultimately enhancing its performance in various experiments. The proposed InceptionV3-based model achieved the highest predictive performance with an accuracy of 92.0%, AUC of 0.91, sensitivity of 91.0%, and specificity of 93.0%. These results indicate a robust and reliable capability in predicting pCR status compared to other baseline models such as ResNet, VGGNet, and DenseNet. The reported accuracy of 92.0% is accompanied by a 95% confidence interval of [88.4%, 95.6%], and the AUC of 0.91 is statistically significant (p< 0.01), confirming the model’s robustness.An ablation study was conducted to assess the contribution of each preprocessing step. Removal of normalization reduced AUC by 4.2%, absence of resizing caused unstable input flow resulting in 3.8% accuracy loss, and skipping noise reduction led to a 2.9% drop in specificity. These findings highlight the necessity of a structured preprocessing pipeline for stable performance. In addition to accuracy and AUC, we report the Matthews Correlation Coefficient (MCC) of 0.78 and a Precision-Recall AUC of 0.89, which provide further insight into model reliability, especially in cases with ambiguous imaging features or class imbalance.pCR prediction with T1 images with perilesional featureThe classification performance of deep learning models utilizing breast cancer MRI images, coupled with T1 images and Perilesional features, is notably robust across various architectures—ResNet, VGGNet, DenseNet, InceptionV3, and a conventional CNN. As per Table 3, The models exhibit high accuracy (ranging from 0.84 to 0.87) and discriminatory power, as reflected by impressive Area Under the Curve (AUC) values ranging from 0.85 to 0.88. Sensitivity, measuring the models’ ability to correctly identify positive cases (PCR), ranges from 0.83 to 0.86, indicating effectiveness in capturing actual positive instances. Specificity, measuring the models’ proficiency in identifying negative cases (NoPCR), is consistently high, ranging from 0.86 to 0.89. The F1 Score, reflecting a balance between precision and recall, ranges from 0.81 to 0.85. These results collectively suggest the models’ competence in distinguishing between patients with and without breast cancer.Table 3 Classification performance of deep learning models using perilesional features (T1 + region area and parameters).Full size tablepCR prediction with T1 images with intralesional featuresThe presented results showcase the performance of various deep learning models trained on breast cancer MRI images, utilizing T1 images and Intralesional features for classification. Among the models evaluated—ResNet, VGGNet, DenseNet, InceptionV3, and a conventional CNN—distinct patterns emerge in terms of accuracy, sensitivity, specificity, and the F1 Score. Notably, InceptionV3 attains the highest accuracy at .89, demonstrating its exceptional ability to discriminate between positive and negative cases. ResNet and DenseNet also exhibit strong overall performance, boasting high accuracy and balanced sensitivity and specificity. VGGNet follows closely, showcasing solid discrimination capabilities. The conventional CNN, while having the lowest accuracy at .83, still provides reasonable performance. Sensitivity and specificity are well-maintained across models, reflecting their proficiency in correctly identifying positive and negative cases. The F1 Score further illuminates the precision-recall trade-offs made by each model. These findings underscore the potential of deep learning models trained on T1 images and intensity-based features for breast cancer classification, emphasizing the need for thorough validation and collaboration with medical professionals for real-world applicability. The nuanced insights provided by each metric contribute to a comprehensive understanding of the models’ strengths and considerations in the context of breast cancer diagnosis. As per Table 4, All models show promising results, with accuracies ranging from .83 to .89, indicating their effectiveness in classifying breast cancer cases based on T1 images and intensity features. InceptionV3 stands out with the highest accuracy, while ResNet and DenseNet also perform very well. Sensitivity and specificity are well-balanced in most models, demonstrating their ability to identify both positive and negative cases effectively. The F1 Score provides insights into the precision and recall balance, highlighting the trade-offs made by each model.Table 4 Classification performance using intralesional features extracted from T1-weighted DCE-MRI.Full size tableResults with numerous loss function, activation function and optimizer functionWithin the domain of deep learning, the selection of an appropriate loss function holds a critical position in the training process of neural networks, especially in the context of Breast cancer MRI and PCR classification. This metric serves as a measure of the model’s efficacy by evaluating the difference between its predicted outcomes and the actual PCR labels. Throughout the training phase, the primary objective revolves around the minimization of the loss function value. This endeavor is undertaken with the aim of refining the model’s performance, ultimately leading to improved accuracy and reliability in discerning between benign and malignant breast cancer cases in MRI scans. The three loss functions–binary crossentropy, hinge, and Kullback-Leibler divergence–can be expressed as equations as depicted below:Binary Crossentropy Loss (also known as Log Loss or Negative Log Likelihood Loss):$$\begin{aligned} \text {Binary Crossentropy Loss} = - (y * log(p(a)) + (1 - y) * log(1 - p(a))) \end{aligned}$$(1)where:- y is the true label (0 or 1) of the sample.- p(a) represents the estimated probability of the positive class. (between 0 and 1).Hinge Loss (used for Support Vector Machine classifiers):$$\begin{aligned} \text {Hinge Loss} = max(0, 1 - y * f(a)) \end{aligned}$$(2)where:- The true label of the sample, denoted as y assumes values of either 1 or -1.- f(a) is the decision function output for the sample.Kullback-Leibler Divergence (also known as Relative Entropy):$$\begin{aligned} \text {Kullback-Leibler Divergence} = sum(p(x) * log(p(a) / q(a))) \end{aligned}$$(3)where:- p(a) is the actual probability distribution.- q(a) is the estimated probability distribution.Table 5 InceptionV3 classification performance with different loss functions.Full size tableTable 5 summarizes the outcomes of our Breast cancer MRI classification study, where we assessed the impact of different loss functions within a neural network model on the discrimination of patients with and without pathologic complete response (PCR). Each patient’s data includes a uni-lateral cropped original dynamic contrast-enhanced (DCE) folder, consisting of a sequence of .dcm images capturing the temporal evolution of contrast enhancement. Our findings reveal that the “hinge” loss function achieved an AUC of 0.90, demonstrating a robust ability to distinguish between patients with varying PCR statuses. The accompanying accuracy of .91, sensitivity of .92, specificity of .89, and F1-score of .90 underscore its promising performance in identifying subtle patterns indicative of PCR in the MRI images. In the case of the “binary crossentropy” loss function, we observed an AUC of 0.91, reflecting the model’s effective discrimination between PCR and NoPCR cases. With an accuracy of .92, sensitivity of .93, specificity of .90, and F1-score of .90, this loss function exhibited a noteworthy capability to capture nuanced features associated with PCR status in breast cancer patients.For the “kullback leibler” loss function, our model achieved an AUC of 0.89, highlighting its consistent discriminative ability in the context of Breast cancer MRI classification. The corresponding accuracy of .90, sensitivity of .91, specificity of .87, and F1-score of .87 collectively indicate its potential utility in distinguishing between patients with diverse PCR outcomes.These results emphasize the critical role of loss functions in optimizing the neural network’s ability to identify subtle patterns within dynamic contrast-enhanced MRI sequences, providing valuable insights for clinicians in predicting pathologic complete response in breast cancer patients.In the study related to breast cancer MRI and PCR classification, various activation functions were investigated. These functions’ performance metrics, including minimum and maximum AUC (Area Under the Curve) values, were analyzed. Activation functions are pivotal in influencing how a neural network model processes MRI data to predict PCR results. One of the frequently utilized activation functions in deep learning models is ReLU. ReLU is a piecewise linear function that retains positive or zero input values and converts any negative input to zero. In mathematical notation, the ReLU function is defined as:$$\begin{aligned} ReLU(x) = max(0, x) \end{aligned}$$(4)ReLU allows faster convergence during training and is computationally efficient since it only involves simple thresholding. The logistic function, which is alternatively referred to as the sigmoid function, finds widespread usage as an activation function in neural networks designed for tasks involving binary classification. It maps any real-valued number to the range [0, 1]. The sigmoid function is given by the formula:$$\begin{aligned} sigmoid(x) = 1 / (1 + exp(-x)) \end{aligned}$$(5)Sigmoid activation is useful when we want the output of the neural network to represent probabilities or when dealing with binary classification problems. It squashes the input values between 0 and 1, making it suitable for producing a probability-like output.The tanh activation function bears similarities to the sigmoid function, yet it maps the input within the range of [-1, 1]. Its mathematical definition is as follows:$$\begin{aligned} tanh(x) = (2 / (1 + exp(-2x))) - 1 \end{aligned}$$(6)Tanh is a zero-centered function, meaning its output has a mean of zero for both positive and negative inputs. This property helps mitigate the vanishing gradient problem compared to the sigmoid function. Tanh is useful in scenarios where we need the output to be centered around zero, such as in the hidden layers of a neural network. However, like sigmoid, tanh can also suffer from the vanishing gradient issue, especially for extreme input values.Table 6 InceptionV3 classification performance with different activation functions.Full size tableTable 6 presents pivotal insights from our Breast cancer MRI classification study, examining the influence of different activation functions within a neural network model. Each patient’s breast cancer MRI images, stored in a uni-lateral cropped original dynamic contrast-enhanced (DCE) folder with a sequence of .dcm images, are meticulously categorized into pathologic complete response (PCR) or NoPCR.The study investigates three activation functions: relu, tanh, and tanh, each shaping the model’s performance. For the “relu” activation function, the model achieved AUC values ranging from 0.90 to 0.92. These results suggest that the relu activation function contributes to a model with moderate discriminatory power, and the maximum AUC of 0.92 underscores its effectiveness in distinguishing diverse breast cancer cases in MRI scans.In the case of the “tanh” activation function, the model demonstrated AUC values ranging from 0.89 to 0.91. These outcomes suggest that the tanh activation function exhibits a varying impact on model performance, with the maximum AUC of 0.91 indicating a reasonably good discriminatory capability in identifying breast cancer cases within MRI scans. The accompanying accuracy, sensitivity, specificity, and F1-score for the tanh activation function are 0.92, 0.91, 0.93, and 0.90, respectively.These findings contribute valuable clinical insights, providing a nuanced understanding of how activation functions influence the model’s capacity to discern subtle patterns indicative of pathologic complete response in breast cancer patients undergoing MRI assessments.In the scope of your breast cancer MRI and PCR classification model, various optimizer functions were evaluated. Their performance metrics, specifically the minimum and maximum AUC values, were scrutinized. These optimizer functions play a critical role in refining the neural network model’s parameters to enhance its predictive accuracy for PCR outcomes based on breast cancer MRI data.One commonly employed optimization algorithm in the realm of machine learning, including neural networks, is Stochastic Gradient Descent (SGD). In SGD, the model’s parameters are adjusted incrementally, typically after processing each individual data point or a small batch of data points. The primary objective is to iteratively minimize the loss function by fine-tuning the model’s parameters in the direction of steepest descent, which is opposite to the gradient of the loss concerning the parameters. The update rule for SGD is given by:$$\begin{aligned} \theta {(t+1)} = \theta (t) - \text {learning rate} * \nabla J(\theta (t)) \end{aligned}$$(7)where \(\theta (t)\) is the model’s parameters at time step t, J(\(\theta\)) is the loss function, and \(\nabla J(\theta (t))\) is the gradient of the loss function with respect to the parameters. The learning rate determines the step size for parameter updates and is a hyperparameter that needs to be tuned.SGD is simple and computationally efficient, making it a popular choice for optimization. However, it may suffer from slow convergence, especially when the learning rate is not appropriately tuned, and it may oscillate around the minimum of the loss function.RMSprop is an adaptive learning rate optimization algorithm that addresses some of the limitations of traditional SGD. It maintains a moving average of the squared gradients for each parameter. The update rule for RMSprop is given by:$$\begin{aligned} \begin{aligned} \text {moving avg squared grad}&= \text {decay rate} * \text {moving avg squared grad} \\&\quad + (1 - \text {decay rate}) * (\nabla J(\theta (t)) ^ 2) \end{aligned} \end{aligned}$$(8)$$\begin{aligned} \begin{aligned} \theta (t+1)&= \theta (t) - \\&\quad \left( \frac{\text {learning rate}}{\sqrt{\text {moving avg squared grad} + \epsilon }} \right) * \nabla J(\theta (t)) \end{aligned} \end{aligned}$$(9)where the decay rate serves as a hyperparameter, governing the rate at which the moving average diminishes. Meanwhile, epsilon represents a small constant added to the denominator, primarily as a precautionary measure to avoid division by zero17. RMSprop modifies the learning rate individually for each parameter, taking into account the recent gradient magnitudes. It helps to speed up convergence and can handle different learning rates for different parameters.Adam is an extension of RMSprop that also includes momentum. It combines the benefits of both RMSprop and momentum optimization techniques. Adam computes an exponentially decaying average of the gradients themselves (similar to momentum). The update rule for Adam is given by:$$\begin{aligned} & m = \beta _1 \cdot m + (1 - \beta _1) \cdot \nabla J(\theta (t)) \end{aligned}$$(10)$$\begin{aligned} & v = \beta _2 \cdot v + (1 - \beta _2) \cdot (\nabla J(\theta (t))) ^ 2 \end{aligned}$$(11)$$\begin{aligned} & m_{\text {hat}} = \frac{m}{(1 - \beta _1^t)} \end{aligned}$$(12)$$\begin{aligned} & v_{\text {hat}} = \frac{v}{(1 - \beta _2^t)} \end{aligned}$$(13)$$\begin{aligned} & \theta (t+1) = \theta (t) - \left( \frac{\text {learning\_rate}}{\sqrt{v_{\text {hat}}} + \text {epsilon}} \right) \cdot m_{\text {hat}} \end{aligned}$$(14)where m and v are the initial and secondary moments of the gradients, respectively, and beta1 and beta2 are hyperparameters that control the decay rates of the moments. The t denotes the time step (i.e., the number of updates performed), and epsilon is a small constant added to the denominator for numerical stability.Table 7 InceptionV3 classification performance with different optimizer functions.Full size tableTable 7 presents insights for a Breast cancer MRI classification task, comparing the performance of different optimizers used in a neural network model. The evaluated optimizers are Stochastic Gradient Descent (SGD), RMSprop, and Adam.In the context of our Breast cancer MRI classification study, the choice of optimizer plays a crucial role in shaping the model’s performance. When employing the “sgd” (Stochastic Gradient Descent) optimizer, AUC values exhibited a range from 0.9 to an impressive maximum of 0.92. These outcomes underscore the significant impact of the sgd optimizer on the model, with a maximum AUC of 0.92, indicating a robust capability to accurately classify breast cancer cases based on MRI data and PCR results. The “rmsprop” (Root Mean Square Propagation) optimizer yielded AUC values ranging from 0.91 to a maximum of 0.92. These results suggest that the rmsprop optimizer had a moderate impact on model performance, with a maximum AUC of 0.92, signifying reasonable discriminatory ability in identifying breast cancer cases in MRI scans. Lastly, the “adam” (Adaptive Moment Estimation) optimizer produced AUC values spanning from 0.89 to a maximum of 0.91. These outcomes suggest that the choice of the adam optimizer led to varying model performance, with a maximum AUC of 0.91, indicating a reasonable capability to classify breast cancer cases based on MRI data and PCR results. These nuanced findings provide valuable insights into the interplay between optimizer selection and the model’s ability to discern subtle patterns indicative of pathologic complete response in breast cancer patients undergoing MRI assessments.Conclude pCR classificationIn this proposed approach, we conducted a thorough evaluation of five deep learning models–ResNet, VGGNet, DenseNet, InceptionV3, and CNN–for the classification of breast cancer using MRI images. All models demonstrated commendable performance, with high overall accuracy, AUC, sensitivity, specificity, and F1 score. However, InceptionV3 emerged as the standout performer across all metrics. Notably, it achieved the highest accuracy at 92%, showcasing its proficiency in correctly categorizing patients as either positive or negative for breast cancer. The superior discriminatory ability of InceptionV3 was further emphasized by its leading AUC of 0.91, underscoring its effectiveness in distinguishing between positive and negative cases. Moreover, As per Table 8, InceptionV3 exhibited an impressive balance between sensitivity (91%) and specificity (93%), surpassing the other models. The elevated F1 score of 0.9 highlighted InceptionV3’s capacity for precision and recall, emphasizing its robustness in breast cancer classification. In summary, our results underscore InceptionV3 as the optimal choice among the evaluated models, offering a compelling foundation for its potential clinical integration due to its superior overall performance in breast cancer detection using MRI images.Table 8 Model(s) accuracy.Full size tableDiscussionWomen seek breast cancer MRI for various reasons, including having BRCA1 or BRCA2 gene mutations, having first-degree relatives with these mutations, having a high lifetime risk of breast cancer, receiving chest radiation therapy during their youth, or having certain genetic disorders or relatives with these disorders. These factors increase their need for breast cancer screening through MRI. Instead of progressing directly to surgery following the identification of breast cancer through a breast cancer MRI, the standard approach often involves initiating neoadjuvant chemotherapy. Neoadjuvant chemotherapy is a treatment regimen that entails the administration of chemotherapy drugs before surgical intervention. This strategy serves multiple crucial purposes in the realm of breast cancer management. Firstly, neoadjuvant chemotherapy aims to reduce the size of the tumor or tumors within the breast. By shrinking the cancerous tissue, it makes surgical removal more effective and potentially less invasive. In some cases, it may even facilitate breast-conserving surgeries, such as lumpectomies, which can preserve a significant portion of the breast tissue. Moreover, neoadjuvant chemotherapy provides a valuable opportunity to assess how the tumor responds to the chemotherapy drugs. This response evaluation guides subsequent treatment decisions. A favorable response, characterized by significant tumor shrinkage, may indicate a more optimistic prognosis and influence the choice of surgical procedure. Following the completion of the prescribed neoadjuvant chemotherapy cycle, patients typically undergo a follow-up breast cancer MRI. This post-treatment MRI plays a pivotal role in determining if a pathologic complete response has been achieved. A pathologic complete response signifies that there is no detectable cancer remaining in the breast tissue18. This milestone is significant, as it is associated with improved long-term outcomes.This study advances the current state of breast cancer response prediction by introducing a multi-modal approach that not only achieves superior accuracy but also incorporates clinically interpretable features. Compared to traditional methods, our model bridges the gap between black-box deep learning and domain-specific radiomics, offering a balanced trade-off between performance and interpretability. The results of the post-chemotherapy breast cancer MRI contribute to tailoring the subsequent steps in the treatment plan. If a pathologic complete response is confirmed, it may impact the decision regarding whether to proceed with surgery or consider alternative treatments, such as radiation therapy or hormonal therapy.Neoadjuvant chemotherapy and the subsequent breast cancer MRI assessments are integral components of a comprehensive breast cancer treatment strategy. They enable healthcare providers to customize treatment plans based on individual patient responses to chemotherapy and specific tumor characteristics. This personalized approach aims to optimize treatment effectiveness and long-term survival while minimizing the impact on breast tissue and overall quality of life. For real-world deployment, the model could be integrated into radiology applications or clinical decision support systems , subject to compliance with data privacy laws. Implementation will require federated learning frameworks, secured patient consent protocols, and interoperability with hospitals.Instead of relying solely on traditional chemical processes to assess Pathologic Complete Response (pCR) in breast cancer treatment, a cutting-edge approach involves the utilization of deep learning techniques. In this innovative method, deep learning models are trained to analyze breast cancer MRI images, providing predictions regarding the likelihood of achieving a PCR. These MRI images capture critical information about the tumor and surrounding tissues, and deep learning algorithms excel at extracting intricate patterns and features that may not be apparent through visual inspection alone. This approach is advantageous as it offers non-invasive assessment, eliminating the need for invasive tissue biopsies. Furthermore, it takes advantage of temporal data, considering MRI images taken at different points during neoadjuvant chemotherapy to assess the dynamic nature of tumor response. The abundance of available MRI data from breast cancer patients allows for robust model training, making the predictions more accurate. Overall, this integration of deep learning into breast cancer diagnosis and treatment represents a promising avenue for enhancing the precision and effectiveness of patient care. Our approach is the combination of advanced feature extraction, leveraging the InceptionV3 model with customized hyperparameters. And rigorous validation was a crucial component of our study, and we used state-of-the-art techniques to accomplish this. This meticulous validation process significantly enhanced both the accuracy and the credibility of our study, particularly in the context of pCR classification19. The primary aim of the previous study was to enhance the precision of predicting pathologic complete response (pCR). This prediction pertained to the application of neoadjuvant systemic therapy (NAST) among patients diagnosed with triple-negative breast cancer (TNBC)20. A predictive model was meticulously constructed by integrating two key components. These components were the pretreatment MRI radiomic features (MRIRF) and the levels of tumor-infiltrating lymphocytes (TIL). The amalgamated model demonstrated enhanced prognostic accuracy in contrast to the standalone models. The integration of MRIRF and TIL resulted in a higher positive predictive value (PPV) and negative predictive value (NPV), indicating its potential for more accurate pCR prediction in TNBC patients undergoing NAST21,22. A different research endeavor sought To assess the presence of intramammary edema (intra-E) and intratumoral necrosis (intra-N) as observed in T2-weighted magnetic resonance imaging (T2WI) could function as indicators of pathological complete response (pCR) in patients with triple-negative breast cancer (TNBC). Nevertheless, the findings indicated that the existence of intra-E or intra-N before neoadjuvant chemotherapy (NAC) did not exhibit any significant correlation with the pCR rate. The only positive correlation observed was between intra-E and tumor size. But in our sophisticated approach we have a clear prdiction of pCR23 Identify breast cancer patients achieving pathologic complete response (pCR) after neoadjuvant chemotherapy (NAC) by analyzing dynamic contrast-enhanced (DCE) MRI images acquired before treatment. The study used radiomic analysis to extract features from tumor24 and peritumoral regions. Machine learning logistic regression models were trained and validated on DCE-MRI images, achieving significant area under the receiver operating characteristics (ROC) curves for different cancer subtypes. In this proposed study, we have used all the features along with some additional features like mean intensity, centriod etc. for better analysis and classification25. Theere are analysis like SVM which achieved an area under the curve (AUC) of 0.92 for distinguishing between pCR and non-pCR. Although there were some misclassified cases, the survival curve for the pCR group exhibited a more elevated position when compared to the non-pCR group, though not statistically significant (p=0.2). Here is the difference is only quantifiable variales considered instead image related features26. RUSBoosted Tree machine-learning classification which combining pre- and early treatment data with molecular subtypes and Ki67, and highlights the potential of using texture features and molecular subtypes in MRI analysis to predict PCR outcomes.This study has certain limitations. First, the dataset is limited to 384 patients from a single public source, which may impact the model’s generalizability. Second, the model has not been externally validated on independent cohorts. Third, potential class imbalance and variations in imaging protocols across centers could introduce bias27.There are study which assessed the women who completed NAC for breast cancer and compared 68Ga-FAPI uptake and FAP immunoreactivity in patients with or without pathologic complete response (pCR). The proposed approach has not focus to a particular type of patient, but yes it can be checked the accuracy28. There are prvious study aimed to predict pathologic complete response (pCR) to neo-adjuvant systemic therapy (NST) in breast cancer patients using pretreatment MRI-based radiomics analysis. They developed and validated radiomics, clinical, and combined models based on MRI data from 320 tumors. However, the clinical models outperformed the radiomics models significantly in predicting pCR29, and the combined models showed similar or better performance. This suggests that radiomics features did not add significant clinical models demonstrated superior predictive value. The lack of reproducibility data hindered a deeper analysis, highlighting the need for reproducibility studies to better assess the potential of radiomics30 in this context. The approach is not only contains the radiomics feature but features.have taken intralesional features.We also analyzed multiple masses using MRI segmentation and pCR classification with different VGG16 and ResNet networks31. But accuracy was comparatively lower. The proposed model could be integrated into clinical decision-making pipelines by assisting oncologists in early identification of patients likely to achieve pCR, thereby potentially reducing unnecessary chemotherapy cycles and improving personalized treatment planning.InceptionV3 was selected for its superior ability to capture multi-scale spatial features using factorized convolutions. Compared to ResNet and VGGNet, InceptionV3 showed better robustness to overfitting with a 9.5% lower variance between training and test accuracy and 15% faster convergence during training. The dataset includes 255 patients with diverse profiles: age ranged from 23 to 73 years, tumor sizes ranged from 0.5 cm to 5.4 cm, and included various histological subtypes such as IDC, ILC, and mixed types. Future work will incorporate external validation using multi-center datasets to assess model generalizability across broader and more heterogeneous populations. The proposed model is computationally efficient, requiring approximately 2.4 seconds per inference on a mid-range GPU. This supports its feasibility for integration into real-time clinical settings, including resource-constrained environments.ConclusionIn conclusion, This method has a specific design, and its primary goal is the identification of patients. It seeks to identify those patients who achieve a pathologic complete response (pCR). This response is obtained as a direct outcome of neoadjuvant therapy administered for breast cancer treatment. We achieved this by leveraging advanced feature extraction techniques, incorporating radiomic features such as region centroid, Entropy, Sphericity, and more, which provided deeper insights into the MRI data and enhanced the discriminative power of the classification model. This study presents a novel integration of InceptionV3 with DCE-MRI and radiomic features, offering a more comprehensive approach to predicting pathological complete response compared to conventional methods.To further optimize the model’s performance, we utilized the InceptionV3 (GoogleNet) model and carefully tailored its hyperparameters. Through rigorous experimentation with different combinations of loss functions, optimizer functions, and activation functions, we identified the most effective configuration for the task at hand.To ensure the reliability of the findings, it subjected the classification results to thorough validation using state-of-the-art techniques, including Matthews Correlation Coefficient, Cohen’s Kappa, and Jaccard Index. The advanced evaluation metrics were instrumental in offering a comprehensive evaluation of the model’s performance. They played a crucial role in confirming the credibility of the obtained results.By adopting a collaborative approach involving both expert radiologists and a computer-aided system, we achieved superior predictive performance for pCR. This model exhibited impressive values for essential parameters like the area under the curve (AUC), Matthews Correlation Coefficient, Cohen’s Kappa, and Jaccard Index.Future work will focus on validating the model on larger, more diverse datasets, including multi-center cohorts, and evaluating its applicability to other cancers and imaging modalities beyond DCE-MRI.Overall, this study highlights the importance of combining advanced feature extraction techniques, leveraging sophisticated deep learning models, and utilizing rigorous validation methods to enhance the accuracy and credibility of pCR classification in breast cancer patients undergoing neoadjuvant therapy. Our findings have significant implications for personalized treatment planning and can potentially contribute to improved patient outcomes and the advancement of breast cancer care.