Nivolumab Addition for Advanced Esophageal or Stomach Cancer

Inclusion Criteria: * Participants must have advanced or locally unresectable gastric, gastroesophageal junction or esophageal adenocarcinoma * Participants must have PD-L1 CPS (Combined Positive Score) ≥ 1. This test would have been performed as part of standard of care (SOC) pathology testing, using tissue obtained within two years prior to registration and collected prior to or after a frontline regimen * Participants must have a histologically confirmed diagnosis of microsatellite stable (MSS) and HER2 negative gastric, gastroesophageal junction, or esophageal adenocarcinoma * Participants must have documented unresectable and/or metastatic disease on CT or MRI imaging completed prior to registration. Imaging must have been completed within 28 days prior to registration for participants with measurable disease. CT scans or MRIs used to assess non-measurable disease must have been completed within 42 days prior to registration. All disease must be assessed and documented on the Baseline Tumor Assessment Form * Participants with treated brain metastases must have no evidence of progression on the follow-up brain imaging after central nervous system (CNS)-directed therapy. All treatment for brain metastases must have been completed at least 28 days prior to registration * Participants must have disease progression or intolerance to frontline standard of care (SOC) chemotherapy plus either nivolumab, pembrolizumab or any other PD-1 or PD-L1 inhibitor. Peri-operative chemotherapy plus nivolumab, pembrolizumab or any other PD-1 or PD-L1 inhibitor will count as one line if disease progression occurs while on the therapy or within 6 months of completing the chemotherapy plus nivolumab or pembrolizumab or other PD-1/PD-L1 inhibitor cycle * Participants must not have received more than one prior line of systemic therapy defined as chemotherapy plus either nivolumab, pembrolizumab, or any other PD-1 or PD-L1 inhibitor, in the stage IV or unresectable setting. Peri-operative or adjuvant nivoluamb or other PD-1/PD-L1 inhibitors would count as one prior line of systemic therapy if patients progressed while on nivolumab (or other PD-1/PD-L1 inhibitors) or within 6 months of stopping it * Note: Radiation or any other regional therapy options done to address local residual disease or metastatic disease would not count as a line of therapy. Maintenance therapy with a different form of fluoropyrimidine (i.e. switching from capecitabine to fluorouracil \[5FU\]) would not count as another line of therapy * Participants must not have a condition requiring systemic treatment with either corticosteroids (\> 10 mg daily prednisone equivalents) or other immunosuppressive medications within 14 days of registration. Inhaled or topical steroids and adrenal replacement doses \< 10 mg daily prednisone equivalents are permitted in the absence of active autoimmune disease. Participants are permitted to use topical, ocular, intra-articular, intranasal, and inhalational corticosteroids (with minimal systemic absorption). Physiologic replacement doses of systemic corticosteroids are permitted, even if \< 10 mg/day prednisone equivalents. A brief course of corticosteroids for prophylaxis (e.g., contrast dye allergy) or for treatment of non-autoimmune conditions (e.g., delayed-type hypersensitivity reaction caused by contact allergen) is permitted, as long as there has been a washout period for corticosteroids of ≥ 7 days prior to registration * Participants must not have prior significant immunotherapy related adverse events requiring permanent discontinuation of the immunotherapy agent including events like pneumonitis, myocarditis, renal failure, Guillain barre syndrome, or myasthenia gravis. Participants with endocrinopathy events leading or not to replacement steroids, thyroid hormone, insulin, or cortisol are eligible * Participants must not have received a live attenuated vaccination within 28 days prior to registration * Participants must not have had a major surgery within 28 days or subcutaneous venous access device placement within 7 days prior registration * Participants must have fully recovered from the effects of prior surgery in the opinion of the treating investigator. Any participants with postoperative bleeding complications or wound complications from a surgical procedure performed in the last eight weeks should be excluded * Participants must not have plans to undergo elective or planned major surgery during the clinical trial * Participants must not have active bleeding or prior history of gastrointestinal (GI) perforation, fistula or significant GI bleeding (requiring transfusion, endoscopic or surgical intervention) within 84 days prior to registration * Participants must not be planning to receive any concurrent chemotherapy, immunotherapy, investigational agents, biologic or hormonal therapy for cancer treatment while receiving treatment on this study * Participants must not have a history of a grade 3 or 4 allergic reaction attributed to humanized or human monoclonal antibody therapy * Participants must not have a history of grade 3 or 4 immunotherapy related toxicities with the exception of hormonal abnormalities like thyroiditis or thyroid derangements * Participants must be ≥ 18 years old * Participants must have Zubrod Performance Status of 0-2 * Participants must have a complete medical history and physical exam within 28 days prior to registration * Leukocytes ≥ 2 x 10\^3/uL (within 28 days prior to registration) * Absolute neutrophil count ≥ 1.2 x 10\^3/uL (within 28 days prior to registration) * Hemoglobin ≥ 9.0 g/dL (within 28 days prior to registration) * Platelets ≥ 100 x 10\^3/uL (within 28 days prior to registration) * Total bilirubin ≤ 1.5 x institutional upper limit of normal (ULN) (within 28 days prior to registration) unless history of Gilbert's disease. Participants with history of Gilbert's disease must have total bilirubin ≤ 5 x institutional ULN * Aspartate aminotransferase (AST)/alanine aminotransferase (ALT) ≤ 3 × institutional ULN (within 28 days prior to registration) (unless liver metastases are present, in which case they must be ≤ 5 x ULN) * Participants must have a calculated creatinine clearance ≥ 40 mL/min using the following Cockcroft-Gault Formula. This specimen must have been drawn and processed within 28 days prior to registration * Participants' urinary protein must be ≤ 1+ on dipstick or routine urinalysis (UA) within 28 days of registration. Random analysis of urine protein with a normal value is sufficient. If urine dipstick or routine analysis indicated proteinuria ≥ 2+, then a 24-hour urine is to be collected and demonstrate \< 1000 mg of protein in 24 hours to allow participation in the study * Participants must have adequate cardiac function. Participants with known history or current symptoms of cardiac disease, or history of treatment with cardiotoxic agents, must have a clinical risk assessment of cardiac function using the New York Heart Association Functional Classification. To be eligible for this trial, participants must be class 2 or better * Participants must have recovered to baseline or \< grade 2 CTCAE version (v) 5.0 from toxicities related to any prior treatments, unless AE(s) are clinically stable on supportive therapy * Participants must not have experienced arterial thromboembolic events, including but not limited to myocardial infarction, transient ischemic attack, cerebrovascular accident, or unstable angina, within 6 months prior to registration * Participants must not have uncontrolled blood pressure within 28 days prior to registration as determined by the treating investigator * Participants with known human immunodeficiency virus (HIV)-infection must be on effective anti-retroviral therapy at registration and have undetectable viral load on the most recent test results obtained within 6 months prior to registration * Participants with evidence of chronic hepatitis B virus (HBV) infection must have undetectable HBV viral load while on suppressive therapy on the most recent test results obtained within 6 months prior to registration, if indicated * Participants with a history of hepatitis C virus (HCV) infection must have been treated and cured. Participants currently being treated for HCV infection must have undetectable HCV viral load on the most recent test results obtained within 6 months prior to registration, if indicated * Participants must not have a prior or concurrent malignancy whose natural history or treatment (in the opinion of the treating physician) has the potential to interfere with the safety or efficacy assessment of the investigational regimen * Participants must not be pregnant or nursing (nursing includes breast milk fed to an infant by any means, including from the breast, milk expressed by hand, or pumped). Individuals who are of reproductive potential must have agreed to use an effective contraceptive method with details provided as a part of the consent process. A person who has had menses at any time in the preceding 12 consecutive months or who has semen likely to contain sperm is considered to be of "reproductive potential." In addition to routine contraceptive methods, "effective contraception" also includes refraining from sexual activity that might result in pregnancy and surgery intended to prevent pregnancy (or with a side-effect of pregnancy prevention) including hysterectomy, bilateral oophorectomy, bilateral tubal ligation/occlusion, and vasectomy with testing showing no sperm in the semen * Participants must not have a history of inflammatory bowel disease, (including ulcerative colitis and Crohn's disease), symptomatic autoimmune disease (e.g., rheumatoid arthritis, systemic progressive sclerosis \[scleroderma\], systemic lupus erythematosus, autoimmune vasculitis \[e.g., Wegener's Granulomatosis\]); CNS or motor neuropathy considered of autoimmune origin (e.g., Guillain-Barre Syndrome and myasthenia gravis, multiple sclerosis). Note: Participants with Graves' disease will be allowed * Participants must not have a history of pneumonitis that has required oral or IV steroids within the last 12 months prior to registration * Participants must be offered the opportunity to participate in specimen banking. With participant consent, specimens must be collected and submitted via the Southwest Oncology Group (SWOG) Specimen Tracking System * Participants who can complete patient reported outcomes (FACT-Ga and PRO-CTCAE) questionnaires in English or Spanish must participate in the quality of life studies * Participants must be informed of the investigational nature of this study and must sign and give informed consent in accordance with institutional and federal guidelines. For participants with impaired decision-making capabilities, legally authorized representatives may sign and give informed consent on behalf of study participants in accordance with applicable federal, local, and Central Institutional Review Board (CIRB) regulations