You can permanently delete a tag from the dashboard. This removes the tag itself, not just its association with one link.
To delete a tag
Go to your links list.
Hover over a link that has the tag you want to delete, and select the Edit icon.
In the edit area, find the tag and select the trash can icon next to it.
The tag is deleted permanently and no longer appears as an option for any link.