Enfortumab Vedotin and Pembrolizumab Combined With Radiotherapy in Muscle Invasive Bladder Cancer

Inclusion Criteria: * Biopsy-confirmed muscle-invasive bladder cancer (cT2,T3,T4a). (Note: Tissue samples are required.) (Participants with cT3/T4a staged disease will be capped at 25% of patients treated at RP2D). * Urothelial histology present. Mixed histologies other than small cell/neuroendocrine are allowed as long as some urothelial histology is present. Neuroendocrine histology of any component and pure variant (non-urothelial) histology tumors will be excluded. (Patients with \< 50% urothelial histology will be capped at 25% of patients treated at RP2D). * Must be judged by the investigator to be ineligible for radical cystectomy or electing not to undergo radical cystectomy. * Must be eligible for and agree to receive bladder irradiation as determined by the treating investigator. * Must have a TURBT within 8 weeks of combination treatment start with viable tumor content. If no viable tumor content is present on TURBT, the patient will be replaced in the study. * Patients who have autoimmune disease will be evaluated on a case-by-case basis and can only enroll so long as participants are not on active immunosuppression with a corticoid steroid allowance exceeding 10mg of prednisone or equivalent per day. * Age \>= 18 years. * Eastern Cooperative Oncology Group (ECOG) performance status \<= 1. * Absolute neutrophil count ≥ 1,500/microliter (mcL). * Platelets \>= 100,000/mcL. * Hemoglobin \>= 9.0 g/dL or ≥ 5.6 mmol/L. \* Criteria must be met without erythropoietin dependency and without packed red blood cell (pRBC) transfusion within last 2 weeks * Total bilirubin \<= 1.5 × upper limit of normal, unless elevated due to Gilbert's syndrome and direct bilirubin is within normal limits. * Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase (SGOT)) \<= 2.5 X institutional upper limit of normal. * Alanine aminotransferase (ALT) (serum glutamic pyruvic transaminase (SGPT)) \<= 2.5 X institutional upper limit of normal. * Creatinine clearance glomerular filtration rate (GFR) \>= 30 mL/min/1.73 m\^2, calculated by Cockcroft-Gault or measured using 24-hour creatinine clearance. * International normalized ratio (INR) OR prothrombin time (PT) \<= 1.5 × upper limit of normal (ULN). \* If participant is receiving anticoagulant therapy, as long as PT or activated partial thromboplastin time (aPTT) is within therapeutic range of intended use of anticoagulants, participant is eligible. * Activated partial thromboplastin time (aPTT) \<= 1.5 × ULN. \* If participant is receiving anticoagulant therapy, as long as PT or aPTT is within therapeutic range of intended use of anticoagulants, participant is eligible. * Ability to understand and the willingness to sign a written informed consent document. * Human immunodeficiency virus (HIV)-infected individuals on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial. * Participants who are hepatitis B virus surface antigen (HBsAg) positive are eligible if they have received hepatitis B virus (HBV) anti-viral therapy for at least 4-weeks, and have undetectable HBV viral load prior to randomization. Participants should remain on anti-viral therapy throughout study intervention and follow local guidelines for HBV anti-viral therapy post completion of study intervention. \* Note: Hepatitis B screening tests are not required unless patients have a known history of HBV infection. * Participants with a history of hepatitis C virus (HCV) infection are eligible if HCV viral load is undetectable at screening. \* Note: Participants must have completed curative anti-viral therapy at least 4 weeks prior to cycle 1 day 1. * Women of child-bearing potential and men with sexual partners of childbearing potential must agree to use adequate contraception for the duration of study participation. Enfortumab vedotin (EV) may cause fetal harm. Women of child-bearing potential must use contraception during treatment with EV and for 120 days after the last dose. Men with female partners who are women of child-bearing potential must use contraception during treatment with EV and for 120 days after the last dose. Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Acceptable methods include barrier method, hormonal method, as well as intrauterine devices * Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 8 weeks after last administration of study treatment. Exclusion Criteria: * Presence of distant metastases on imaging (M1 disease). * Presence of ≥ N2 disease on imaging (N1 disease allowed, but participants with N1 disease will be capped at 25% of patients treated at RP2D). * Presence of small cell / neuroendocrine histology in tumor sample (any content). * Absence of urothelial histology in TURBT tumor sample (pure variant histology). * Presence of untreated upper tract urothelial cancer. * Presence of severe hydronephrosis precluding therapy in the judgement of the treating physician. * Presence of extensive carcinoma in situ (CIS) is exclusionary; moderate CIS that could still benefit from radiation treatment in the judgement of treating physician is allowed. * Baseline neuropathy grade 2 (G2) or greater. * Baseline uncontrolled diabetes mellitus.Uncontrolled diabetes is defined as hemoglobin A1c (HbA1c) ≥ 8% or HbA1c 7 to \< 8% with associated diabetes symptoms (polyuria or polydipsia) that are not otherwise explained. \* Note: Patients with prior diagnosis but with disease under control are eligible * Prior treatment with systemic immunotherapy or chemotherapy for urothelial cancer. (with the exception of prior systematic therapy treatment \>12 months prior). Note: Prior bacillus calmette-guerin (BCG) and intravesical treatments are allowed * Has received prior systemic anti-cancer therapy including investigational agents within 4 weeks prior to cycle 1 day 1. * Has received prior radiotherapy within 2 weeks of cycle 1 day 1 or had radiation-related toxicities requiring corticosteroids. * Has received a live vaccine or live-attenuated vaccine within 30 days before the first dose of study intervention. \* Note: Administration of killed vaccines is allowed. Any licensed coronavirus 2019 (COVID-19) vaccine (including for emergency use) is allowed in the study as long as they are messenger ribonucleic acid (mRNA) vaccines, replication-incompetent adenoviral vaccines, or inactivated vaccines. * Major surgery within 2 weeks prior to first dose of EV. \* Note: Cataract surgery, standard tissue biopsies, and standard of care cardiac devices, such as a pacemaker or stent placed on an elective basis, are allowable procedures. * Has a diagnosis of immunodeficiency or is receiving chronic systemic steroid therapy (in dosing exceeding 10 mg daily of prednisone equivalent) or any other form of immunosuppressive therapy within 7 days prior to the first dose of study drug. \* Note: Inhaled or topical steroids are permitted. * HIV-infected participants with a history of Kaposi's sarcoma and/or Multicentric Castleman's Disease. * Has known active CNS metastases and/or carcinomatous meningitis. * History of another significant life-limiting malignancy requiring systematic treatment within 2 years prior to the first dose of study drugs, or any evidence of residual disease from a previously diagnosed malignancy. \* Note: Patients with nonmelanoma skin cancer, curatively treated localized prostate cancer, curatively treated upper tract urothelial cancer, or carcinoma in situ of any type (if complete resection was performed) are allowed. * Hypersensitivity to pembrolizumab or enfortumab vedotin, or any of their excipients. * Prior allogeneic stem cell or solid organ transplant. * Has a history of (non-infectious) pneumonitis/interstitial lung disease that required steroids or has current pneumonitis/interstitial lung disease. * Has an active infection requiring systemic therapy. * Has a history or current evidence of any condition, therapy, or laboratory abnormality or other circumstance that might confound the results of the study, interfere with the participant's participation for the full duration of the study, such that it is not in the best interest of the participant to participate, in the opinion of the treating investigator. * History of Hepatitis B with detectable HBV viral load (participants who are HBsAg positive are eligible if they have received HBV anti-viral therapy for at least 4 weeks, and/or have undetectable HBV viral load prior to randomization) or known active hepatitis C virus (defined as detectable HCV RNA .\[qualitative\]) infection. \* Note: Testing for Hepatitis B or C is not required unless clinically indicated or if there is a known history of hepatitis infection. Has known psychiatric or substance abuse disorders that would interfere with cooperation with the requirements of the trial. * Pregnant and breast feeding participants are excluded from this study because targeted chemotherapy and radiation have the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with EV+pembrolizumab, breastfeeding should be discontinued if the mother is treated with these investigational products.