Single quantum dot controls a plasmonic cavity’s scattering and anisotropy