When you press the Delete button on the Administration|Templates screen, a confirmation window will appear.

If the selected Template is not in use and can be deleted, the window will simply ask you to confirm the delete. Press OK to confirm or Cancel to abort.

If it cannot be deleted because it is in use, the text will be "xxxx has relationships and cannot be deleted. Do you want to deactivate it instead?"(where xxx is the name of the item you are trying to delete). Pressing OK will unset its Active flag, Cancel will abort.