Improving File Writing speed
I would like to improve the speed of my code. It generates an output file
for the results in CSV format which may be very large (10-200MB). Each line of the output file contains a record which is calculated and written sequentially using WRITE #. I would like to improve the speed of this process by improving the speed of the File Output which seems to be the main bottleneck. I thought about "writing to memory" and then dumping results in big chunks at regular intervals could improve the speed but I am not sure how to implement it to respect the CSV formatting that I am getting using the WRITE # command. Any suggestions, ideas? |
Improving File Writing speed
Care to share some code and/or timings? You don't say how long the process
takes right now, so it's difficult to imagine how much improvement could be made. I seem to recall comparing file creation speeds between either building up the file as a huge string in memory and writing it in one shot, versus writing out each line directly to the file. Writing line-by-line was faster. Note if you want to stry building up a large string before writing it out then you should avoid repeated concatenations and instead use a "string builder" class to handle it. Eg: http://www.vbaccelerator.com/home/Vb...er/article.asp It's not so difficult to create a CSV-formatted line: just separate your fields with a comma and wrap any content containing a literal comma in quotes. Tim "FCS" wrote in message ... I would like to improve the speed of my code. It generates an output file for the results in CSV format which may be very large (10-200MB). Each line of the output file contains a record which is calculated and written sequentially using WRITE #. I would like to improve the speed of this process by improving the speed of the File Output which seems to be the main bottleneck. I thought about "writing to memory" and then dumping results in big chunks at regular intervals could improve the speed but I am not sure how to implement it to respect the CSV formatting that I am getting using the WRITE # command. Any suggestions, ideas? |
All times are GMT +1. The time now is 12:07 PM. |
Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
ExcelBanter.com