I have data in E4, E12, E20... that I want to copy into a new column without
the extra rows in between.

Here's one way.

If the new column starts at row 1, put this at the top and copy
downward:
=OFFSET($E$4,(ROW()-1)*8,0)

If it starts at row 2, put this instead:
=OFFSET($E$4,(ROW()-2)*8,0)

Some of the data cells might be blank, in which case the formula
produces zero. If this is troublesome, try
=IF(OFFSET($E$4,(ROW()-1)*8,0)="","",
OFFSET($E$4,(ROW()-1)*8,0))

Modify to suit.