First of all, the nominal impedance of 300 ohms is simply an average; it is not inclusive of the variations in impedance at different frequencies. The HD600 shows very substantial peaks in impedance, especially near 100 Hz, where it may rise above 600-700 ohms. This forces the amplifier to provide higher voltage for the same current and, consequently, more power to drive these effective frequencies. Additionally, though a sensitivity rating of 97 dB SPL/mW is calculated at one frequency (1KHz for HD600), real music spans a continuous spectrum of frequencies, all at different levels of driver efficiency. Lower frequencies, where the driver might be less efficient, will require more power to reproduce in balance and with accuracy.
Higher power is also needed due to the dynamic range and transients in music. Also, the rated power values you’re seeing are continuous (2.5V RMS and 20mW for 90dB SPL) but music possesses transient peaks that may be several times the rated values; thus, a relatively large amount of peak power is required from the amplifier. For instance, ten times the rated power may be required by transient peaks, so the amplifier should be able to deliver such bursts to avoid any distortion and keep the sound clean. In addition, the quality and capability of the amplifier are very important. High impedance headphones require more voltage swing, current delivery on a consistent basis from an amplifier. Some amplifiers do struggle with this kind of headphone, having either a high output impedance or being limited in some way regarding voltage swing capability.