Really the only solid case for switching to a “balanced” headphone amp and cable - beyond the “specs look better” argument, which can just as easily point you to an SE amp - is if you have such low-impedance headphones that the crosstalk remains disturbingly high no matter what high-quality 4-wire SE cable you get. (The SE cable voltage-divider crosstalk problem is outlined here: Cables: Do they make a difference, and if so, what do you like?)
So for something like a Verum 1 (8-ohm drivers) or in general for headphones with driver impedance of 16-ohm-or-lower, I can see the point of “going balanced”, because the circuit math shows the crosstalk might indeed never be satisfactory with any SE cable.