After exporting,

ActiveWorkbook.SaveAs FileName:=X, _
FileFormat:=xlCSV, CreateBackup:=False
ActiveWindow.Close , savechanges:=False

Then you would be where you were before you added this workbook. But if you
must go someplace else, you could explicitly return somewhere else:

Windows("FirstFile.xls").Activate