Durvalumab with Chemotherapy for Stage II-III Breast Cancer

Inclusion Criteria: * STEP 1: REGISTRATION (SCREENING): Participants must have histologically confirmed estrogen receptor (ER) positive and/or progesterone receptor (PR) positive (hormone receptor positive) and HER2 negative breast cancer, as per American Society of Clinical Oncology (ASCO) College of American Pathologists (CAP) guidelines * NOTE: Participants with HER2 positive disease by ASCO CAP guidelines are ineligible. HER2 negative and HER2 low or equivocal cases as per ASCO CAP guidelines that do not receive HER2 targeted therapy are eligible * STEP 1: REGISTRATION (SCREENING): Participants must have clinical stage II or III breast cancer * NOTE: Participants with inflammatory breast cancer are eligible * NOTE: Participants with occult (i.e. undetectable) primary breast cancer with axillary nodal involvement are not eligible, as MammaPrint testing has not been validated on tissue obtained from an axillary lymph node * STEP 1: REGISTRATION (SCREENING): Participants must not have metastatic disease (i.e., must be clinically M0 or Mx) Systemic staging studies with imaging should follow routine practice as per National Comprehensive Cancer Network (NCCN) and ASCO guidelines * STEP 1: REGISTRATION (SCREENING): Participants must not have locally recurrent breast cancer * STEP 1: REGISTRATION (SCREENING): Participants with multifocal disease in the same breast or synchronous bilateral primary tumors are eligible, however, all tumors that are biopsied must be hormone receptor positive and HER2 negative per ASCO CAP guidelines and at least one of the tumors must be MammaPrint High-2. MammaPrint can be performed sequentially on biopsies as it is sufficient to have MammaPrint High 2 status on at least one of the lesions * NOTE: Biopsy of multiple lesions in the same breast is not required if the clinical presentation is consistent with a single disease process that is multifocal in nature. However, if there is clinical suspicion of two distinct primary breast malignancies, additional biopsies should be pursued * STEP 1: REGISTRATION (SCREENING): Participants must have either adequate tissue available to submit on-study or a prior known MammaPrint Index Score that is MP2 status * Submitting tissue for on-study MammaPrint testing: * Participants must have a minimum of ten, unstained formalin-fixed paraffin-embedded (FFPE) slides (4-5 micron thickness) available from initial tumor biopsy for MammaPrint assessment * NOTE: Participants must agree to have this tissue submitted to Agendia for MammaPrint Index Scoring and to have subsequent results disclosed to Southwest Oncology Group (SWOG) Cancer Research Network OR * Submitting prior known MammaPrint Index Score: * If a MammaPrint Index Score report from within the last 12 weeks is already known and is MP2 status, the participant must be registered to Step 2 immediately following Step 1 registration provided they meet all other criteria. MP2 status is defined as a MammaPrint Index score between negative 1.0 and negative 0.57 (-1.0 to -0.57, including negative 1.0 and negative 0.57) tested from initial tumor biopsy * NOTE: Participants must agree to have their commercial MammaPrint Index Score disclosed to SWOG Cancer Research Network * NOTE: Participants with prior known MammaPrint result that is not MP2 status should not be enrolled to either step of this study * NOTE: Participants enrolling with known MP2 status (i.e. MP already obtained as routine care) must only sign the treatment informed consent form. Screening consent is not required when MP2 status is known prior to study enrollment * STEP 1: REGISTRATION (SCREENING): Participants must not have received any prior treatment for their current breast cancer, including chemotherapy, immunotherapy, biologic or hormonal therapy, and must be candidates for doxorubicin, paclitaxel, and durvalumab therapy * STEP 1: REGISTRATION (SCREENING): Participants must be \>= 18 years old at the time of registration * STEP 1: REGISTRATION (SCREENING): Participants must have body weight \> 30 kg * STEP 1: REGISTRATION (SCREENING): Participants must have Zubrod Performance Status of 0-2 * STEP 1: REGISTRATION (SCREENING): Participants with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial * STEP 1: REGISTRATION (SCREENING): Participant must not have medical contraindications to receiving immunotherapy, including history of non-infectious pneumonitis that required steroids or active autoimmune disease that has required systemic treatment with disease modifying agents, corticosteroids or immunosuppressive drugs in the past two years. Replacement therapy (e.g. thyroxine for pre-existing hypothyroidism, insulin for type I diabetes mellitus, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment. Intra-articular steroid injections are allowed * STEP 1: REGISTRATION (SCREENING): NOTE: As a part of the Oncology Patient Enrollment Network (OPEN) registration process the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system * Participants must be informed of the investigational nature of this study and must sign and give informed consent in accordance with institutional and federal guidelines * For participants with impaired decision-making capabilities, legally authorized representatives may sign and give informed consent on behalf of study participants in accordance with applicable federal, local, and Central Institutional Review Board (CIRB) regulations * STEP 2: RANDOMIZATION: Participants must have met all eligibility criteria for Step 1 Registration * STEP 2: RANDOMIZATION: Participants must have MammaPrint High Risk 2 result * For participants submitting tissue for on-study MammaPrint testing: * Participants must be registered to Step 2: Randomization within 84 calendar days (12 weeks) after receiving an MP2 status from the MammaPrint Index score. MP2 status is defined as a MammaPrint Index score between negative 1.0 and negative 0.57 (-1.0 to -0.57, including negative 1.0 and negative 0.57) from initial tumor biopsy OR * Submitting commercial MammaPrint Index Score: * If a MammaPrint Index Score report from within the last 12 weeks is already known and is MP2 status, the participant must be registered to Step 2 immediately following Step 1 registration provided they meet all other criteria. MP2 status is defined as a MammaPrint Index score between negative 1.0 and negative 0.57 (-1.0 to -0.57, including negative 1.0 and negative 0.57) tested from initial tumor biopsy * NOTE: Participants without a MammaPrint High-Risk 2 score must not be registered to Step 2 Randomization * STEP 2: RANDOMIZATION: Participants must not have received live vaccines within 28 days prior to study Step 2: Randomization. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, chicken pox, shingles, yellow fever, rabies, Bacillus Calmette-Guerin (BCG), and typhoid (oral) vaccine. Seasonal influenza vaccines and coronavirus disease 2019 (COVID-19) vaccines are allowed; however, intranasal influenza vaccines (e.g. Flu-Mist) are live attenuated vaccines, and are not allowed * STEP 2: RANDOMIZATION: Participants must not be planning to receive any concurrent non-protocol directed chemotherapy, immunotherapy, biologic or hormonal therapy for cancer treatment while receiving treatment on this study * STEP 2: RANDOMIZATION: Participant must have Zubrod Performance Status of 0-2 * STEP 2: RANDOMIZATION: Participants must not have a history of (non-infectious) pneumonitis that required steroids or evidence of active pneumonitis within two years prior to Step 2: Randomization * STEP 2: RANDOMIZATION: Participants must not have active autoimmune disease that has required systemic treatment in the past two years (i.e., with use of disease modifying agents, corticosteroids or immunosuppressive drugs) prior to Step 2: Randomization. Replacement therapy (e.g. thyroxine for pre-existing hypothyroidism, insulin for type I diabetes mellitus, or physiologic corticosteroid replacement therapy for adrenal or pituitary insufficiency, etc.) is not considered a form of systemic treatment. Intra-articular steroid injections are allowed * STEP 2: RANDOMIZATION: Participant must have a complete medical history and physical exam within 28 days prior to Step 2: Randomization * STEP 2: RANDOMIZATION: Leukocytes \>= 3 x 10\^3/uL (within 28 days prior to Step 2: Randomization) * STEP 2: RANDOMIZATION: Absolute neutrophil count \>= 1.5 x 10\^3/uL (within 28 days prior to Step 2: Randomization) * STEP 2: RANDOMIZATION: Platelets \>= 100 x 10\^3/uL (within 28 days prior to Step 2: Randomization) * STEP 2: RANDOMIZATION: Total bilirubin =\< institutional upper limit of normal (ULN) unless history of Gilbert's disease. Participants with history of Gilbert's disease must have total bilirubin =\< 5 x institutional ULN (within 28 days prior to Step 2: Randomization) * STEP 2: RANDOMIZATION: Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) =\< 3 × institutional ULN (within 28 days prior to Step 2: Randomization) * STEP 2: RANDOMIZATION: Participants must have a calculated creatinine clearance \>= 50 mL/min using the Cockcroft-Gault Formula. This specimen must have been drawn and processed within 28 days prior to Step 2: Randomization * STEP 2: RANDOMIZATION: Participants must have adequate cardiac function. Participants with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, must have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, participants must be class 2B or better * STEP 2: RANDOMIZATION: Participants must not have uncontrolled diabetes defined as hemoglobin A1c of 9.0% or greater, within 28 days prior to Step 2: Randomization * STEP 2: RANDOMIZATION: Participants with history of human immunodeficiency virus (HIV)-infection must be on effective anti-retroviral therapy at registration and have an undetectable viral load on the most recent test results obtained within 6 months prior to Step 2: Randomization * STEP 2: RANDOMIZATION: Participants with history of chronic hepatitis B virus (HBV) infection must have undetectable HBV viral load on the most recent test results obtained while on suppressive therapy within 6 months prior to Step 2: Randomization, if indicated * STEP 2: RANDOMIZATION: Participants with a history of hepatitis C virus (HCV) infection must have been treated and cured. Participants currently being treated for HCV infection must have undetectable HCV viral load on the most recent test results obtained within 6 months prior to Step 2: Randomization, if indicated * STEP 2: RANDOMIZATION: Participants must not be pregnant or nursing. Individuals who are of reproductive potential must have agreed to use an effective contraceptive method during protocol therapy and for 6 months following completion of protocol therapy with details provided as a part of the consent process and must have a negative pregnancy test at screening. A person who has had menses at any time in the preceding 12 consecutive months or who has semen likely to contain sperm is considered to be of "reproductive potential." In addition to routine contraceptive methods, "effective contraception" also includes refraining from sexual activity that might result in pregnancy and surgery intended to prevent pregnancy (or with a side-effect of pregnancy prevention) including hysterectomy, bilateral oophorectomy, bilateral tubal ligation/occlusion, and vasectomy with testing showing no sperm in the semen. Participants should not breastfeed during protocol therapy and for 6 months following completion of protocol therapy * STEP 2: RANDOMIZATION: Participants must be offered the opportunity to participate in specimen banking. With participant consent, specimens must be collected and submitted via the SWOG Specimen Tracking System * STEP 2: RANDOMIZATION: Participants who can complete questionnaires in English, or Spanish must be offered the opportunity to participate in the Quality of Life study * STEP 2: RANDOMIZATION: NOTE: As a part of the OPEN registration process the treating institution's identity is provided in order to ensure that the current (within 365 days) date of institutional review board approval for this study has been entered in the system * STEP 2: RANDOMIZATION: Participants must be informed of the investigational nature of this study and must sign and give informed consent in accordance with institutional and federal guidelines