Tissue-resident memory T cells (TRM) provide durable, localized anti-pathogen and anti-tumor immunity, but the microenvironmental cues that govern their formation, maintenance, and function remain poorly defined. Leveraging a single-cell foundation model developed at Genentech (SCimilarity) across ~24 million cells from healthy and diseased tissues, we delineated TRM gene programs and uncovered robust associations with fibroblast abundance across multiple indications, including cancer. In silico ligand-receptor analysis nominated TGF-β3 as a key fibroblast-derived mediator. Newly developed human and mouse fibroblast–T cell co-culture systems, combined with genetic deletion and isoform-specific antibody blockade, demonstrated that fibroblasts drive TRM-like phenotypes in-vitro via TGF-β3, motivating the generation of the first fibroblast-specific knockout model to ablate Tgfb3 in fibroblasts in vivo (DptCreTgfb3fl/fl). Across steady-state barrier tissues, localized skin viral infection (MVA), and subcutaneous tumor models (KPR.OVA), fibroblast-specific Tgfb3 loss reduced polyclonal and antigen-specific CD4 and CD8 TRM. Single-cell proteogenomic profiling revealed that TGF-β3 deficiency induced a previously undescribed integrated T-cell stress program, indicating a primary pro-survival role for TGF-β3 in sustaining TRM. Functionally, genetic or antibody-mediated TGF-β3 blockade accelerated tumor growth, which revealed a previously underappreciated tumor-suppressive fibroblast axis in cancer. These findings established fibroblast-derived TGF-β3 as a critical paracrine stromal cue that sustains TRM and their anti-tumor function. Importantly, these findings challenge canonical views dominated by the immunosuppressive TGF-β1 isoform and provide a mechanistic rationale for isoform-selective, TGF-β3–sparing modulation over pan–TGF-β blockade in cancer therapy, contextualizing the limited clinical success of past therapeutic strategies targeting the TGF-β pathway.