ctHPVDNA Response-Adapted Chemoradiation +/- Retifanlimab Treatment in Locally-Advanced Anal Cancer

Inclusion Criteria: * Participants must have histologically proven stage T1-4N+M0 or T3-T4N0M0 anal canal or anal margin squamous cell carcinoma. This may include tumors of non-keratinizing histology such as basaloid, transitional cell or cloacogenic histology. Special considerations include the following: * Participants with excision of the primary tumor but with node positive disease or residual disease at the primary if T3-T4N0 will be eligible. * Age ≥18 years * ECOG performance status 0-2 * Creatinine clearance \>30 ml/min by Cockcroft-Gault Equation. * HIV-infected participants are eligible if they meet the following eligibility criteria: * A CD4 T-cell count \>= 200/mm3 and a viral load \< 200 copies/mm3 * No history of AIDS-related complications within past year other than history of low CD4+ T-cell count (\>200/mm3) prior to initiation of combination antiretroviral therapy. * Participant must be healthy on the basis of HIV disease with high likelihood of near normal life span were it not for the anal cancer. * Participant MUST receive appropriate care and treatment for HIV infection, including antiretroviral medications when clinically indicated, and should be under the care of a physician experienced in HIV management. Participants will be eligible regardless of antiretroviral medication provided the regimen has been stable for at least 4 weeks. * Participants must be PPD negative. Alternatively, the QuantiFERON-TB assay can be used. An individual is considered positive for M. tuberculosis infection if the IFN-γ response to TB antigens is above the test cut-off (after subtracting the background IFN-γ response in the negative control). The result must be obtained within 20 weeks prior to enrollment. PPD positive (or Quantiferon assay positive) participants are permitted if prophylaxis has been completed prior to enrollment. * Tumor size must be documented based on physical examination including digital rectal exam and/or anoscopy/proctoscopy within 4 weeks prior to enrollment. * Staging imaging studies must include a PET scan AND either a CT with contrast of the abdomen/pelvis or an MRI with contrast of the pelvis. It is preferred that participants receive contrast. For participants with an allergy who cannot receive pre-medication, or any other reason they can't receive IV contrast, it is recommended that they undergo an MRI of the pelvis. * Participant must have no history of prior chemotherapy for anal cancer. * Participant must not have had prior potentially curative surgery (i.e. abdominal-perineal resection) for carcinoma of the anus. However, participants who undergo local excision or excisional biopsy are eligible provided there was tumor involvement of the anal canal and/or anal verge prior to the resection, if the margins were positive, and/or if the stage is T2N0 based on tumor size before the procedure. This means that participants with T1N0M0 anal margin squamous cell carcinoma who underwent surgical excision with negative margins and no involvement of the anal verge and/or anal canal are not eligible. * Participant must not be receiving any other standard anti-cancer therapy or experimental agent. * Participant must not have intercurrent illness including, but not limited to, ongoing or active infection or psychiatric/social situations that, in the judgement of the investigator, would limit compliance with study requirements. * Participant must not have had significant cardiovascular disease within 6 months prior to enrollment that has not been treated/controlled in the opinion of the treating investigators including myocardial infarction, unstable angina, stroke, transient ischemic attack, symptomatic coronary artery disease, symptomatic congestive heart failure, or uncontrolled cardiac arrhythmia. * Participant must not have a history of a different malignancy unless they are deemed by the investigator to be at low risk of recurrence. * Participants who are on anti-coagulation with warfarin within 2 weeks prior to enrollment must use an alternative anti-coagulant if planned to receive Capecitabine, otherwise, they must receive infusional 5-FU. * NOTE: Low molecular weight heparin is permitted provided the participant's PT/INR is \< 1.5. Participants who will received capecitabine and are on Dilantin for a seizure disorder must have Dilantin levels checked weekly. * Participants must have normal organ and marrow function as defined below: * Hemoglobin ≥ 10.0 g/dl * Platelet count ≥ 100,000/mcL * Absolute neutrophil count ≥ 1,500/mcL * Total bilirubin must be \<1.5 X institutional ULN OR conjugated bilirubin \<= institutional ULN if total bilirubin \> 1.5 X ULN. Note that conjugated bilirubin only needs to be tested if total bilirubin \>1.5 X ULN. Participants with a known history of Gilbert's disease are eligible without regard to bilirubin and liver function tests at the discretion of the treating physician. * AST/ALT must be \</= 2.5 X institutional ULN (participants with a known history of Gilbert's disease are eligible without regard to bilirubin and liver function tests at the discretion of the treating physician). * Albumin \>/= 3.0 g/dL * Women must not be pregnant or breast-feeding because the study treatment may cause harm to an unborn fetus or breastfeeding child. A female of childbearing potential is defined as any woman, regardless of sexual orientation, or whether they have undergone a tubal ligation, who meets the following criteria: 1) has achieved menarche at some point, 2) has not undergone a hysterectomy or bilateral oophorectomy; or 3) has not been naturally postmenopausal (amenorrhea following cancer therapy does not rule out childbearing potential) for at least 24 consecutive months (i.e., has had menses at any time in the preceding 24 consecutive months). Women of childbearing potential and sexually active males must agree to use accepted and effective method(s) of contraception or to abstain from sexual intercourse for the duration of their participation in the study and for at least six months after the completion of treatment. * Participants must have the ability to understand and the willingness to sign a written informed consent document. * Participants must have testing DPYD deficiency per institutional standards and must not be homozygous for DPYD deficiency. Exclusion Criteria: * Any prior pelvic radiation or previous radiation that would result in overlapping radiation fields. * History of allergic reactions to compounds similar to capecitabine, * Participants with a prior or concurrent malignancy whose natural history or treatment has the potential to interfere with the safety or efficacy assessment of the investigational regimen in the opinion of the investigator. * Participants with inflammatory bowel disease, scleroderma, or known homozygosity for DPYD deficiency. * Participants with a fistula between the tumor and invaded organ * Participant must not have active autoimmune disease or inflammatory bowel disease that has required systemic treatment in past 2 years * No prior treatment with an immune checkpoint inhibitor (anti-PD-1, anti-PD-L1, anti-PD-L2, anti-CTLA4 monoclonal antibody) * No participants with immunodeficiency or receiving systemic steroid therapy equivalent to \> 10 mg prednisone per day or any other form of immunosuppressive therapy within 7 days prior to the first dose of study medication. Topical corticosteroid or occasional inhaled corticosteroids are allowed. * No live vaccines within 30 days prior to the first dose of trial treatment and while participating in the trial. Examples of live vaccines include, but are not limited to, the following: measles, mumps, rubella, chicken pox, yellow fever, rabies, BCG, and typhoid (oral) vaccine. Seasonal influenza vaccines for injection are generally killed virus vaccines and are allowed; however, intranasal influenza vaccines (e.g., Flu-Mist®) are live attenuated vaccines and are not allowed. * Participants must not have known interstitial lung disease that is symptomatic or may interfere with the detection or management of suspected drug-related pulmonary toxicity * Participants must not have a history of allergic reactions attributed to compounds of similar chemical or biologic composition to retifanlimab. * Participants are excluded if known to be homozygous for Dihydropyrimidine Dehydronase