If somebody else has your passphrase, they can do anything you can do. This includes rerouting your paycheck to their bank account, busting up your relationships with malicious emails, and stealing your identity to masquerade as your evil twin.
IU will never ask for your passphrase or login credentials via email or over the phone. If you get an email that looks like it came from IU and it’s asking for that information, report it immediately to the University Information Policy Office (UIPO) (firstname.lastname@example.org) or contact the Support Center (http://kb.iu.edu/data/abxl.html). Be sure to include the full email headers (https://kb.iu.edu/data/akij.html).