We have an application that is planning to store around 1.1TB of XML files which average 8.5kb in size.
These represent a rolling 18 months of data, with around 200,000 new files being created every day.
Each file will be written only once, and then has a 3% chance of being read a small number (<10) of times over the following 18 months.
What NTFS options are open to us that will help with performance?
Current ones on our list are:
- Disabling 8.3 name creation
- Limiting the number of files in a directory (number still under debate...)
Regarding fragmentation: We are planning to use 2k cluster sizes for disk space usage efficiency. Each file will be written only once (i.e. no file edits). Files will be deleted after 18 months on a dayby-day basis.
Therefore we don't believe that fragmentation will be a significant issue.