Due to a SMTP failure, some of the mail and receipt were stuck in Outlook. However, these are never shown in the Outbox or Deleted Folders. I keep seeing some mail trying to send out every few minutes, but I was never able to pin point that was wrong.
So, here is the solution to remove those failed acknowledgment and stuff :
You should download this utility from Microsoft :