The delete button or "(X)" is not offered in list view when the user hovers the mouse over a binary field.  A workaround is to switch to table view, where it is possible to delete a binary field.