Currently it has to be installed via pip install mpldatacursor in the "WinPython Command Prompt" (Spyder menu Tools->Open command prompt), which is annoying when I want to convince people to switch from Matlab.
It should be included to reduce the functionality differences to Matlab.