I have had this annoying issue for a while now and decided to fix it.
After installing Evernote (and Toggl also), I wasn't able to delete the shortcut from my desktop.
I found some hints in the right direction on the Evernote support site, but they didn't quite solve the issue.
Apparently it resides in the User\Public folder, but I couldn't Del or Shift+Del it there either. However, when I right click and choose Delete (as Admin) I was finally able to delete the shortcut.
Hope this helps..