Fixing stuck Exchange delegated access

Recently ran into an issue where an admin account had stuck delegated access to user accounts.  Even after removing the access the admin would still see the user account showing in Outlook.  Force updating the Offline Address Book and others didn’t fix it.  PowerShell showed that the admin account was still there with deny permissions …