Lu-177-DOTATATE (Lutathera) in Combination With Olaparib in Inoperable Gastroenteropancreatico Neuroendocrine Tumors (GEP-NET)

* INCLUSION CRITERIA: * Clinical diagnosis of GEP-NET disease, histologically consistent with neuroendocrine tumor. * Inoperable disease (metastatic, non-candidate for surgery with curative intent, locally advanced into vessels or other critical structures, etc.) NOTE: Presence of at least one non-irradiated index lesion (Phase II only). * Patients on somatostatin analogue therapy (e.g., but not only limited to sandostatin or lanreotide therapy) must have initiated and been on a consistent dose of therapy for at least 3 months prior to study enrollment. * Patients on short-term octreotide must have dose held for 24 hours without octreotide because this is necessary for study Lu-177-DOTATATE therapy. * Age \>=18 years. Because no dosing or adverse event data are currently available on the use of Lu-177-DOTATATE in combination with olaparib in patients \<18 years of age, children are excluded from this study, but may be eligible for future pediatric trials. * Must have presence of somatostatin receptors (SSTR) positive disease as documented by positive Ga-68-DOTATATE PET scan within 12 weeks prior to enrollment. NOTE: Positivity of Ga-68-DOTATATE PET scan is defined as having at least one RECIST 1.1 measurable lesion that has an SUV higher than or equal to liver and is qualitatively higher and distinguishable from background activity. * Known BRCA mutation status (Cohort 3 only). * Progressive disease by RECIST 1.1, as compared to previous anatomic imaging no more than 36 months from the date of study enrollment, with at least 1 measurable lesion by RECIST 1.1. * ECOG Performance Status of \<=1. * Patients must have normal organ and bone marrow function measured within 28 days prior to enrollment as defined below: * Hemoglobin \>= 10.0 g/dL with no blood transfusion in the past 28 days * Absolute neutrophil count (ANC) \>= 1.5 x 10\^9/L * Platelet count \>= 100 x 10\^9/L * Total bilirubin \<= 1.5 x institutional upper limit of normal (ULN) * Aspartate aminotransferase (AST) (Serum Glutamic Oxaloacetic Transaminase (SGOT)) / Alanine aminotransferase (ALT) (Serum Glutamic Pyruvate Transaminase (SGPT)) \<= 2.5 x institutional upper limit of normal unless liver metastases are present in which case they must be \<= 5x ULN * Patients must have creatinine clearance estimated of \>= 51 mL/min using the Modification of Diet in Renal Disease (MDRD) study equation or based on a 24 hour urine test: eGFR = 175 x (SCr)\^-1.154 x (age)\^-0.203 x 0.742 \[if female\] x 1.212 \[if Black\] * Ability to understand and willingness to sign informed consent. * Postmenopausal or evidence of non-childbearing status. For individuals of childbearing potential (IOCBP): negative urine or serum pregnancy test within 28 days of study enrollment. Postmenopausal is defined as: * Amenorrheic for 1 year (12 months in a row) or more without an alternative medical cause; if the individual received exogenous hormonal treatments, must be amenorrheic for 1 year or more following cessation of the same * Luteinizing hormone (LH) and Follicle stimulating hormone (FSH) levels in the post-menopausal range for individuals under 50 * Radiation-induced oophorectomy with last menses \>1 year ago * Chemotherapy-induced menopause with \>1 year interval since last menses * Surgical sterilization for IOCBP participants (bilateral oophorectomy or hysterectomy) and/or participants with partners that can bear children. * NOTE: An individual is not of childbearing potential if a prior history of hysterectomy with bilateral oophorectomy or other procedure has rendered the patient surgically sterile, or \>1 years since last menstruation. Must have outside endocrinologist/medical oncologist who can follow the patient for standard of care follow-ups after receiving PRRT at the NIH. * Study drugs can have adverse effects on embryofetal survival and development. It is further not known whether olaparib or its metabolites are found in seminal fluid. For these reasons: * Individuals of childbearing potential and their partners, who are sexually active, must agree to the use of 2 highly effective forms of contraception in combination (male condom plus one of the methods listed below) or must totally/truly abstain from any form of sexual intercourse. This should be started from the signing of the informed consent, throughout study treatment and for at least 7 month for individuals of childbearing potential after the last dose of the study drugs. * Patients with partners that can bear children must use a male condom during treatment and for 4 months after the last dose of study drugs when having sexual intercourse with a pregnant individual or with an individual of childbearing potential. Partners of patients should also use a highly effective form of contraception (see below) if they are of childbearing potential. Patients should not donate sperm throughout study treatment and for 4 months following the last dose of study drugs. * Acceptable birth control methods include: * Total sexual abstinence i.e., refrain from any form of sexual intercourse in line with the patients' usual and/or preferred lifestyle. Abstinence must be for the total duration of the study treatment and for at least 7 months (for IOCBP) or 4 months (for patients with partners of child bearing potential) after the last dose of study treatment. Periodic abstinence (e.g., calendar ovulation, symptothermal, post-ovulation methods) and withdrawal are not acceptable methods of contraception. * Vasectomised sexual partner PLUS male condom (with participant assurance that partner received post-vasectomy confirmation of azoospermia) * Tubal occlusion PLUS male condom * Intrauterine Device (provided coils are copper-banded) PLUS male condom * Mini pill PLUS male condom: Progesterone-based oral contraceptive pill using desogestrel. Cerazette (Merck Sharp \& Dohme) is currently the only highly efficacious progesterone- based pill available * Combined pill PLUS male condom: Normal and low-dose combined oral pills * Injection PLUS male condom: Medroxyprogesterone injection (eg, Depo-Provera \[Pfizer\]) * Implants PLUS male condom: Etonorgestrel-releasing implants (eg, Nexplanon \[Merck Sharp \& Dohme\]) * Patch PLUS male condom: Norelgestromin/ethinyl estradiol transdermal system (eg, Xulane) * Intravaginal device (eg, ethinyl estradiol-/etonogestrel-releasing intravaginal devices such as NuvaRing \[Merck Sharp \& Dohme\]) PLUS male condom * Levonorgestrel-releasing intrauterine system (eg, Mirena \[Bayer\]) PLUS male condom. EXCLUSION CRITERIA: * Patients who have any GEP-NET lesions that are negative by Ga-68-DOTATATE-PET imaging but positive by FDG-PET imaging, unless they have progressed on at least one other line of prior systemic treatment (such as chemotherapy or tyrosine kinase inhibitor) and the majority of their tumor lesions are Ga-68-DOTATATE-avid. * Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with study drugs, breastfeeding should be discontinued if the mother is treated with study drugs. * Other known co-existing malignancies except non-melanoma skin cancer and carcinoma in situ of the uterine cervix, unless definitively treated and proven no evidence of recurrence for 5 years. * Patients who are receiving any other investigational agents. * Patients receiving any systemic chemotherapy or radiotherapy (except for palliative reasons) within 4 weeks prior to study enrollment. * Patients with persistent toxicities (\>= CTCAE grade 2) with the exception of alopecia, caused by previous cancer therapy and toxicities deemed irreversible/stable expected to interfere with study drug administration in the opinion of the Principal Investigator. * Patient's weight exceeding PET table tolerance (\> 400 lbs). * Uncontrolled inter-current illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, hypertension (\>180/110), arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements. * Patients with symptomatic, uncontrolled brain metastases. NOTE: Patients with previously treated brain metastases are eligible if asymptomatic and may be on a stable dose of corticosteroids as long as these were started at least 4 weeks prior to treatment. * Patients with spinal cord compression unless considered to have received definitive treatment for this and evidence of clinically stable disease by imaging and clinical assessment as assessed by the treating investigator for 28 days before enrollment. * Concomitant use of known strong or moderate CYP3A inhibitors within 2 weeks before enrollment. * Concomitant use of known strong or moderate CYP3A inducers within 5 weeks (for enzalutamide or phenobarbital) and 3 weeks for other agents before enrollment. * Patients that have had major surgery within 4 weeks prior to study enrollment and have not recovered from any effects of any major surgery. * Patients unable to swallow orally administered medication and patients with gastrointestinal disorders likely to interfere with absorption of the study medication. * Previous allogeneic hematopoietic stem cell transplant, allogeneic bone marrow transplant or double umbilical cord blood transplant (duCBT). * Patients with a known hypersensitivity to olaparib or Lutathera or any excipients of these products. * Resting ECG indicating uncontrolled cardiac conditions, as judged by the investigator (e.g., unstable ischemia, uncontrolled symptomatic arrhythmia, congestive heart failure, QTcF prolongation \>500 ms, electrolyte disturbances, etc.), or patients with congenital long QT syndrome. * Patients with myelodysplastic syndrome/acute myeloid leukemia or with features suggestive of MDS/AML. * Immunocompromised patients, e.g., patients who are known to be serologically positive for human immunodeficiency virus (HIV). HIV-positive patients on combination antiretroviral therapy are ineligible because of the potential for pharmacokinetic interactions with study drugs. In addition, these patients are at increased risk of lethal infections when treated with marrow-suppressive therapy. Appropriate studies will be undertaken in patients receiving combination antiretroviral therapy when indicated. * Patients with known active hepatitis (i.e., Hepatitis B or C). * Any previous treatment with PARP inhibitor, including olaparib and/or any previous treatment with any systemic radionuclide agents. * Involvement in the planning and/or conduct of the study. * Previous treatment with Lu-177-DOTATATE.