Tumor Infiltrating Lymphocytes for Metastatic Cancer

* INCLUSION CRITERIA: * Measurable (per RECIST v1.0 criteria), metastatic cancer of one of the following types: upper or lower gastrointestinal, hepatobiliary, genitourinary, breast, ovarian/endometrial, or endocrine tumors including neuroendocrine tumors. Patients must have at least one lesion that is resectable for TIL generation with minimal morbidity, preferentially using minimal invasive laparoscopic or thoracoscopic surgery for removal of superficial tumor deposit. * Confirmation of diagnosis of metastatic cancer by the NCI Laboratory of Pathology. * Refractory to approved standard systemic therapy. Specifically: * Patients with metastatic colorectal cancer must have received oxaliplatin or irinotecan. * Patients with hepatocellular carcinoma must have received sorafenib (Nexavar(R)), since level 1 data support a survival benefit with this agent. * Patients with breast and ovarian cancer must be refractory to both first- and second-line treatments and must have received at least one second-line chemotherapy regimen. * Patients with 3 or fewer brain metastases that are \< 1 cm in diameter and asymptomatic are eligible. Lesions that have been treated with stereotactic radiosurgery must be clinically stable for one month after treatment for the patient to be eligible. Patients with surgically resected brain metastases are eligible. * Age greater than or equal to 18 years and less than or equal to 72 years. * Clinical performance status of ECOG 0 or 1. * Patients of both sexes must be willing to practice birth control from the time of enrollment on this study and 12 months after the last dose of combined chemotherapy for individuals of child-bearing potential (IOCBP) and for four months after treatment for individuals that can father children. * IOCBP must have a negative pregnancy test be a pregnancy test prior to the start of treatment because of the potentially dangerous effects of the treatment on the fetus. Serology * Seronegative for HIV antibody. (The experimental treatment being evaluated in this protocol depends on an intact immune system. Patients who are HIV seropositive may have decreased immune-competence and thus may be less responsive to the experimental treatment and more susceptible to its toxicities.) * Seronegative for hepatitis B antigen, and seronegative for hepatitis C antibody. If hepatitis C antibody test is positive, then the patient must be tested for the presence of antigen by RT-PCR and be HCV RNA negative. Hematology * ANC \> 1000/mm\^3 without the support of filgrastim * WBC greater than or equal to 2500/mm\^3 * Platelet count greater than or equal to 80,000/mm\^3 * Hemoglobin \> 8.0 g/dL. Subjects may be transfused to reach this cut-off. Chemistry * Serum ALT/AST less than or equal to 5.0 x ULN * Serum creatinine less than or equal to 1.5 x ULN * Total bilirubin less than or equal to 2.0 mg/dL, except in patients with Gilbert s Syndrome, who must have a total bilirubin \< 3.0 mg/dL. * Patients must have completed any prior systemic therapy at the time of enrollment. Note: Patients may have undergone minor surgical procedures or limited field radiotherapy within the four weeks prior to enrollment, as long as related major organ toxicities have recovered to less than or equal to grade 1. * Ability of subject to understand and the willingness to sign a written informed consent document. * Willing to sign a durable power of attorney. * Subjects must be co-enrolled on protocol 03-C-0277. EXCLUSION CRITERIA: * Participants who are pregnant or nursing because of the potentially dangerous effects of the treatment on the fetus or infant. * Concurrent systemic steroid therapy. * Active systemic infections requiring anti-infective treatment, coagulation disorders, or any other active or uncompensated major medical illnesses. * Advanced primary with impeding occlusion, perforation or bleeding, dependent on transfusion. * Any form of primary immunodeficiency (such as Severe Combined Immunodeficiency Disease and AIDS). * History of major organ autoimmune disease. * Grade 3 or 4 major organ irAEs clinically attributed to anti-PD-1/PD-L1 therapy. * Concurrent opportunistic infections (The experimental treatment being evaluated in this protocol depends on an intact immune system. Patients who have decreased immunecompetence may be less responsive to the experimental treatment and more susceptible to its toxicities.) * History of severe immediate hypersensitivity reaction to cyclophosphamide, fludarabine, or aldesleukin. * History of coronary revascularization or ischemic symptoms. * For select patients with a clinical history prompting cardiac evaluation: last known LVEF less than or equal to 45%. * Documented Child-Pugh score of B or C for hepatocellular carcinoma patients with known underlying liver dysfunction. * For select patients with a clinical history prompting pulmonary evaluation: known FEV1 less than or equal to 50%. * Patients who are receiving any other investigational agents.