You may have discovered that deleting an Azure Active Directory is a particularly frustrating experience that ultimately ends in failure. The new portal have improved things a bit, by running through a series of check before the delete button is enabled.
You may need to go back to the Classic portal (https://manage.windowsazure.com) to see some of the objects/resources to delete.
However, although this will help you remove ‘most’ of what you need to, unfortunately NOT all!
In this case I got a “Unable to delete directory”