Testing the Safety and Tolerability of the Anti-cancer Drugs Trastuzumab Deruxtecan and Neratinib for Cancers With Changes in the HER2 Gene

Inclusion Criteria: * Patients must have histologically confirmed malignancy that is metastatic or unresectable with participation in this clinical trial determined to be the best option for next treatment in the opinion of the investigator, and meet the following specific criteria: * Patients enrolling in Part 1 (Dose Escalation) may have a diagnosis of any solid tumor * Patients enrolling in the Part 2 Pharmacodynamic Cohort may have a diagnosis of any solid tumor except pancreas cancer (NOTE: pancreatic cancer excluded from this cohort after Revision 11 activation due to the addition of the pancreatic specific cohort) * Patients enrolling in the Part 2 Pancreatic Cohort must have a diagnosis of pancreatic adenocarcinoma (PDAC) and meet the HER2 positivity guidance * Patients must have a solid tumor with HER2-positivity as determined by any one or more of the following: * HER2 overexpression defined by immunohistochemistry (IHC) 3+ * ERBB2 amplification by in situ hybridization (ISH) or next-generation sequencing as determined by any Clinical Laboratory Improvement Act (CLIA) certified lab * A known HER2 activating mutation * HER2 overexpression by IHC/ISH will follow histology specific American Society of Clinical Oncology (ASCO)-College of American Pathologists (CAP) guidelines for breast and gastric cancers. HER2 overexpression by IHC/ISH for the Pancreatic Cohort will follow ASCO-CAP guidelines for gastric cancers. For tumor histologies without specific guidelines the following criteria will apply: * HER2 IHC should be performed first, followed by ISH methods in cases showing 2+ (equivocal) expression by IHC. Positive (IHC 3+) or negative (IHC 0 or 1+) do not require further ISH testing. Cases with HER2:CEP17 ratio ≥ 2 or an average HER2 copy number ≥ 6.0 signals per cell are considered positive by ISH * Known HER2 activating mutations: * G309A/E * S310F/Y * S653C * V659E * G660D * R678Q * E693K * Q709L * L755S/P * Del. 755-759 * D769Y/H * G776V/C * V777L * V842I * T862A * L869R * H878Y * All exon 20 insertions, including: * A771\_Y772insYVMA * A775\_G776insYVMA * Y772\_A775dup * P780\_Y781insGSP * G778\_P780dup * V697L * T733I * D769N * L841V * L866M * R896C * If a different mutation is identified, contact the study chair for conferral. Synonymous mutations are not eligible * Patients must have received at least 1 prior line of therapy in the advanced/metastatic setting. No limitation on number of prior therapies; however, patients may not have received neratinib or DS-8201a previously. Prior HER2-targeted therapy other than neratinib or DS-8201a is allowed (e.g., trastuzumab, pertuzumab, TDM-1, lapatinib, etc.) * Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of neratinib in combination with DS-8201a in patients \< 18 years of age, children are excluded from this study * Patients must have Eastern Cooperative Oncology Group (ECOG) performance status =\< 1 (Karnofsky \>= 70%) * Hemoglobin \>= 9.0 g/dL (\>= 8.0 g/dL for gastric cancer \[GC\] only) (within 14 days of enrollment) * No transfusions with red blood cells or platelets are allowed within 1 week prior to screening assessment * Leukocytes \>= 3.0 K/cumm (within 14 days of enrollment) * Absolute neutrophil count \>= 1.5 K/cumm (within 14 days of enrollment) * No administration of granulocyte colony-stimulating factor (G-CSF) is allowed within 1 week prior to screening assessment * Platelets \>= 100 K/cumm (within 14 days of enrollment) * No transfusions with red blood cells or platelets are allowed within 1 week prior to screening assessment * Serum albumin \>= 2.5 g/dL (within 14 days of enrollment) * Total bilirubin =\< 1.5 × institutional upper limit of normal (ULN), (\< 3 × ULN in the presence of documented Gilbert's syndrome or liver metastases at baseline) (within 14 days of enrollment) * Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x ULN (if liver metastases are present =\< 5 x ULN) (within 14 days of enrollment) * International normalized ratio (INR)/prothrombin time (PT) and activated partial thromboplastin time (aPTT) =\< 1.5 x institutional ULN (within 14 days of enrollment) * This applies only to patients who are not receiving therapeutic anticoagulation that may affect INR. Those who are on therapeutic anticoagulation, should be on a stable dose for 4 weeks and should be considered within therapeutic range * Creatinine =\< 1.5 x institutional ULN OR Glomerular filtration rate (GFR) \>= 30 mL/min/1.73 m\^2 (using the Cockcroft-Gault equation) (within 14 days of enrollment) * Patients who are human immunodeficiency virus (HIV)-positive may participate IF they meet the following eligibility requirements: * They must be stable on their anti-retroviral regimen, and they must be healthy from an HIV perspective * They must have a CD4 count of greater than 250 cells/mcL over the past 6 months on this same anti-retroviral regimen and must not have had a CD4 count \< 200 cells/ul over the past 2 years, unless it was deemed related to THE CANCER AND/OR CHEMOTHERAPY-induced bone marrow suppression * For patients who have received chemotherapy in the past 6 months, a CD4 count \< 250 cells/ul during chemotherapy is permitted as long as viral loads were undetectable during this same chemotherapy * They must have an undetectable viral load and a CD4 count \>= 250 cells/uL within 7 days of enrollment * They must not be currently receiving prophylactic therapy for an opportunistic infection and must not have had an opportunistic infection within the past 6 months. HIV-infected patients should be monitored every 12 weeks for viral load and CD4 counts * For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated * Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load * Patients with treated brain metastases are eligible if the following criteria are met: 1) follow-up brain imaging done at least in 4 weeks after central nervous system (CNS)-directed therapy shows no evidence of progression and 2) the patient no longer requires steroids, or is on a stable steroid dose \> 4 weeks * Patients with radiographically new or progressive brain metastases (active brain metastases) or leptomeningeal disease are eligible only if has no progressive clinical symptoms and if the treating physician determines that immediate CNS specific treatment is not required and is unlikely to be required during the first cycle of therapy * Patients should be New York Heart Association functional classification of class 2B or better * Patients must have left ventricular ejection fraction (LVEF) \>= 50% by either an echocardiogram (ECHO) or multigated acquisition (MUGA) scan within 28 days before randomization/enrollment * Part 2, Pancreatic cohort ONLY: Patients must have disease that is measurable by Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1. * Part 2, PD cohort ONLY: Patients must have disease that is evaluable or measurable by Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 * Part 2, PD cohort ONLY: Patients must have at least one lesion suitable for biopsy without significant risk to the patient. The biopsiable lesion can be the same as the evaluable lesion for response by RECIST 1.1 * HER2 antibody conjugated to a topoisomerase 1 inhibitor agents as well as other therapeutic agents used in this trial are known to be teratogenic; thus, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry, for the duration of study participation, and for at least 1 month after the last dose of neratinib, or at least 7 months after the last dose of DS-8201a, whichever is longer (women of childbearing potential \[WOCBP\] only). Should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately. Men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 3 months after the last dose of neratinib, or 4 months after completion of DS-8021a administration, whichever is longer * Women of non-child-bearing potential defined as pre-menopausal females with a documented tubal ligation or hysterectomy; or postmenopausal defined as 12 months of spontaneous amenorrhea (in questionable cases, a blood sample with simultaneous follicle-stimulating hormone \[FSH\] \> 40 mIU/mL and estradiol \< 40 pg/mL \[\< 147 pmol/L\] is confirmatory) are eligible. Females on hormone replacement therapy (HRT) and whose menopausal status is in doubt will be required to use one of the contraception methods outlined for women of child-bearing potential if they wish to continue their HRT during the study. Otherwise, they must discontinue HRT to allow confirmation of post-menopausal status prior to study enrollment. For most forms of HRT, at least 2-4 weeks will elapse between the cessation of therapy and the blood draw; this interval depends on the type and dosage of HRT. Following confirmation of their post-menopausal status, they can resume use of HRT during the study without use of a contraceptive method * Male subjects must not freeze or donate sperm starting at screening and throughout the study period, and at least 4 months after the final study drug administration. Preservation of sperm should be considered prior to enrollment in this study * Female subjects must not donate, or retrieve for their own use, ova from the time of screening and throughout the study treatment period, and for at least 7 months after the final study drug administration * Ability to understand and the willingness to sign a written informed consent document. Participants with impaired decision-making capacity who have a legally-authorized representative (LAR) and/or family member available will also be eligible Exclusion Criteria: * With the exception of medications that are under investigation in the study (e.g., standard of care, comparators, or combination therapies), the following medications, treatment, and procedures will be prohibited during the treatment period: * Other anticancer therapy, including small-molecule targeted agents within 2 weeks or five half-lives, whichever is longer; chemotherapy otherwise not specified (including, but not limited to cytotoxic chemotherapy, antibody drug conjugates, retinoid therapy, hormonal therapy) within 3 weeks; immunotherapy or monoclonal antibody within 4 weeks; and nitrosoureas or mitomycin C within 6 weeks (concurrent use of hormones for noncancer-related conditions \[e.g., insulin for diabetes and hormone replacement therapy\] is acceptable) * Other investigational therapeutic agents * Patients who have had major surgery or radiation within 4 weeks; palliative stereotactic radiation within 2 weeks (except for palliative radiation to known metastatic sites as long as it does not affect assessment of response or interrupt treatment for more than the maximum time specified in dose modification section) * Radiotherapy to the thorax (palliative radiation to known metastatic sites in the thoracic spine is permitted in this study) * Concomitant use of chronic systemic (IV or oral) corticosteroids or other immunosuppressive medications except for managing adverse events (inhaled steroids or intra-articular steroid injections are permitted in this study); chronic replacement dose steroids (e.g., for those with adrenal insufficiency) are permitted in this study * Subjects with bronchopulmonary disorders who require intermittent use of bronchodilators (such as albuterol) will not be excluded from this study * Concomitant treatment with chloroquine or hydroxychloroquine is not allowed during the study treatment due to concern for overlapping toxicities. If treatment with chloroquine and hydroxychloroquine treatment is absolutely required, study treatment must be interrupted. If chloroquine or hydroxychloroquine is administered, then a wash-out period of more than 14 days is required before restarting study treatment * Receipt of live, attenuated vaccine (messenger ribonucleic acid \[mRNA\] and replication deficient adenoviral vaccines are not considered attenuated live vaccines) within 30 days prior to the first dose of study drug * Patients with a history of (non-infectious) interstitial lung disease (ILD)/pneumonitis that required steroids, has current ILD/pneumonitis, or where suspected ILD/pneumonitis cannot be ruled out by imaging at screening * Patients with clinically severe pulmonary compromise resulting from intercurrent pulmonary illnesses including, but not limited to, any underlying pulmonary disorder (i.e. pulmonary emboli within three months of the study enrollment, severe asthma, severe chronic obstructive pulmonary disease (COPD), restrictive lung disease, pleural effusion, etc.), and any autoimmune, connective tissue or inflammatory disorders with potential pulmonary involvement (i.e. rheumatoid arthritis, Sjogren's, sarcoidosis, etc.), or prior pneumonectomy * Patients with history of allergic reactions attributed to compounds of similar chemical or biologic composition to DS-8201a, the inactive ingredients in the drug product, or neratinib * Patients who have a history of severe hypersensitivity reactions to other monoclonal antibodies * Patients receiving any medications or substances that are moderate or strong inhibitors or inducers of CYP3A4 and P-glycoprotein are ineligible. Avoid concomitant use with proton pump inhibitors and P-glycoprotein substrates. Because the lists of these agents are constantly changing, it is important to regularly consult a frequently-updated medical reference. As part of the enrollment/informed consent procedures, the patient will be counseled on the risk of interactions with other agents, and what to do if new medications need to be prescribed or if the patient is considering a new over-the-counter medicine or herbal product * Patients with a medical history of myocardial infarction within 6 months before enrollment, or symptomatic congestive heart failure (CHF) (New York Heart Association class II to IV) * Patients with a corrected QT interval (QTc) prolongation to \> 470 ms (females) or \> 450 ms (males) based on average of the screening triplicate 12-lead electrocardiogram (ECG) * Patients with clinically significant corneal disease in the opinion of the investigator * Patients with a pleural effusion, ascites, or pericardial effusion that requires drainage, peritoneal shunt, or cell-free and concentrated ascites reinfusion therapy (CART). (Drainage and CART are not allowed within 2 weeks prior to screening assessment) (GC indication) * Patients with spinal cord compression * Patients with an uncontrolled infection requiring IV antibiotics, antivirals, or antifungals * Patients with unresolved toxicities from previous anticancer therapy, defined as toxicities (other than alopecia) not yet resolved to grade =\< 1 or baseline. Subjects with chronic grade 2 toxicities may be eligible per the discretion of the investigator after consultation with the sponsor medical monitor or designee (e.g., grade 2 chemotherapy-induced neuropathy) * Patients with substance abuse or any other medical conditions such as clinically significant cardiac or psychological conditions, that may, in the opinion of the investigator, interfere with the subject's participation in the clinical study or evaluation of the clinical study results * Pregnant women are excluded from this study because DS-8201a is a HER2 antibody conjugated to a topoisomerase 1 inhibitor agent with the potential for teratogenic or abortifacient effects. Because there is an unknown but potential risk for adverse events in nursing infants secondary to treatment of the mother with DS-8201a, breastfeeding should be discontinued if the mother is treated with DS-8201a. These potential risks may also apply to other agents used in this study * Prior treatment with neratinib or DS-8201a * Clinically significant chronic gastrointestinal disorder with diarrhea as a major symptom; grade 2 (G2) or greater diarrhea at baseline. Please contact the study principal investigator (PI) for any patient with more than two episodes of diarrhea per day averaged over at least a 7 day period at time of screening to determine whether the diarrhea would be considered clinically significant * Inability to swallow tablets * Patients with active additional malignancy or a personal history of additional malignancy that may affect outcome of disease under treatment (patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen at the discretion of the treating investigator are allowed) * Patients with prior allogeneic organ transplantation including allogeneic stem cell transplantation