Electron-phonon coupling, i.e., the scattering of lattice vibrations by electrons and vice versa, is a common phenomenon in solids and can lead to emergent ground states such as superconductivity and charge-density wave order. Signatures of strong electron-phonon coupling, e.g. softening and broadening of phonons on cooling, are typically assigned to the presence of nested parts of the Fermi surface or lattice anharmonicity. Here, we unravel a third scenario in the seminal strong-coupling material YNi$_2$B$_2$C. The three-dimensional Fermi surface features a large value of the electronic joint density-of-states but only fro a particular value of the electron out-of-plane momentum $k_z$. Using a combination of inelastic neutron scattering and angle-resolved photoemission spectroscopy analyzed based on ab-initio lattice dynamical calculations we show that this peak of the electronic joint density-of-states as function of $k_z$ is likely the origin for the spectacular phonon renormalization in YNi$_2$B$_2$C. Thus, our study rationalizes strong phonon anomalies in the absence of both classic, i.e. phonon-momentum dependent nesting and anharmonicity.