How do I remove automatic formatting of new columns in Excel?

When I delete columns i a spreadsheet, the last (new) columns appear with
borders around some cells.
This makes it difficult to insert new columns in the middle of the sheet.

How do I remove this automatic formatting?