Phase I/II Study of the Combination Immunotherapy Regimen: SX-682, TriAdeno Vaccine, Retifanlimab and IL-15 Agonist N-803 (STAR15) for Metastatic Colorectal Cancer (mCRC)

* INCLUSION CRITERIA: * Participants with histologically confirmed colorectal cancer and evidence of metastatic disease. * Participants must have received, been ineligible to receive, or refused to receive two lines of standard systemic therapy i.e., a fluoropyrimidine with oxaliplatin or irinotecan with bevacizumab, regorafenib, trifluridine, and (if history of RAS wild-type) EGFR-targeted therapy. Participants must have received one line of systemic checkpoint inhibitor if history of advanced microsatellite instability-high \[MSI-H/dMMR\]) metastatic colon cancer. * Participants who had progressive disease within 6 months before study treatment following standard adjuvant therapy are eligible if they have not received systemic therapy for metastatic disease. Participants with a history of MSI-H/dMMR must have also received one line of checkpoint inhibitor therapy. * Age \>= 18 years. * Measurable disease per RECIST 1.1. * ECOG performance status \<= 2. * Adequate organ and marrow as a function defined below: * absolute neutrophil count (ANC) \>= 1,500 cells/mm\^3 * platelet count \>= 100,000 cells/mm\^3 * hemoglobin (Hgb) \>= 9 g/dL * total bilirubin level \< 1.5 x upper limit of normal (ULN) * alanine aminotransferase (ALT) \<= 2.5 x ULN OR \<= 5 x ULN for participants with liver metastases * aspartate aminotransferase (AST) level \<= 2.5 x ULN OR \<= 5 x ULN for participants with liver metastases * creatinine clearance (CrCl) calculated by Cockroft-Gault formula \>= 50 mL/min * Resolution of toxic effect(s) of prior anti-cancer therapy (except alopecia and neuropathy) to Grade \<=1 or to \<=2 if effective medical management of those toxicities is in place such that they are controlled per standard of care (e.g., grade 2 hypothyroidism requiring oral thyroid replacement). * Participants with treated brain metastases are eligible if clinically appropriate follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression. * Participants positive for human immunodeficiency virus (HIV) are eligible if they are compliant with appropriate anti-retroviral therapy for at least 6 months, have HIV viral load \<400 copies/mL, and a CD4 count \> 350 cells/microliter at screening. * Participants positive for Hepatitis C virus (HCV) are eligible if they have completed definitive anti-viral therapy and have an undetectable viral load. * Individuals of child-bearing potential (IOCBP) and individuals who can father children must agree to use an effective method of contraception (barrier, hormonal, intrauterine device \[IUD\], surgical sterilization) at study entry and up to 6 months after the last dose of the study drug(s). * Breastfeeding participants must be willing to discontinue breastfeeding from study treatment initiation through 6 months after study treatment discontinuation. * Participants must have lesion(s) accessible for biopsy (other than used for measurement of disease) and be willing to undergo mandatory study biopsies in the phase 1 portion of the study. Lesions to be biopsied will be determined safely accessible by the provider performing the biopsy (e.g. interventional radiology if a liver or lung biopsy) prior to performing the biopsy. Note: If Phase 2 opens, per the investigator s discretion, if biopsy is strenuous to obtain, e.g., no easily accessible lesions are available or the subject condition is not amenable for biopsy, then the subject will be eligible for enrollment without biopsy at screening and for continuation of treatment without on-treatment study biopsy. -Participants must be able to understand and willing to sign a written informed consent document. EXCLUSION CRITERIA: * Participants with prior investigational drug, chemotherapy, immunotherapy, or any prior therapeutic radiotherapy within 14 days prior to study treatment initiation. * Participants with palliative radiotherapy performed within 7 days prior to study treatment initiation. * Active autoimmune disease requiring systemic immunosuppression in excess of physiologic maintenance doses of corticosteroids (\> 10 mg/day of prednisone or equivalent) with the exception of: * intermittent use of bronchodilators, inhaled corticosteroids, or local corticosteroid injections in participants with asthma * using topical, ocular, intra-articular, or intranasal corticosteroids (with minimal systemic absorption * brief courses of corticosteroids for prophylaxis (e.g., contrast dye allergy). * Evidence of interstitial lung disease, history of interstitial lung disease, or active, noninfectious pneumonitis. Participants with chronic post-radiation pulmonary changes/scarring that is asymptomatic are eligible. * Active infections requiring systemic antibiotics or antifungal or antiviral treatment within 8 days prior to treatment initiation. Participants who have had appropriate antibiotics initiated but are still completing the treatment course are eligible if clinically improved or had minimal symptoms at presentation (e.g., urinary tract infection or pharyngeal streptococcal infection without evidence of systemic inflammatory response). * History of organ transplant, including allogeneic stem cell transplantation. * Participants who experienced immune-related toxicity during prior checkpoint inhibitor therapy for which permanent discontinuation of therapy was recommended (per product label or consensus guidelines) or any immune-related toxicity requiring systemic corticosteroids (with the exception of endocrinopathy that is well controlled on replacement hormones). * History of allergic reactions attributed to compounds of similar chemical or biologic composition to the study drugs. * Receipt of a live vaccine within 28 days prior to treatment initiation. Note: Examples of live vaccines include but are not limited to measles, mumps, rubella, varicella-zoster (chickenpox), yellow fever, rabies, BCG, and typhoid vaccines. Seasonal influenza vaccines for injection are generally killed-virus vaccines and are allowed; however, intranasal influenza vaccines are live, attenuated vaccines and are not allowed. * History of infection with Hepatitis B virus (HBV) unless on suppressive therapy. Individuals with serologic evidence of a resolved prior HBV infection (i.e., HBsAgnegative and anti-HBc positive) are eligible. * Pregnancy confirmed with Beta-human chorionic gonadotropin (Beta-HCG) serum or urine pregnancy test performed in IOCBP at screening. * Uncontrolled intercurrent illness that would limit compliance with study requirements.