Minimizing Hard Disk Drive Failure and Data Loss/Reference

Managing backups and revisions
If multiple generations of backups or revisions are stored, as they usually are, older generations must eventually be deleted due to storage capacity limitations.

FIFO approach
A naive approach to deleting past generations would be to always delete the oldest generations until there is sufficient capacity for the upcoming generations. This is analogous to FIFO. This approach, however, is naive and can lead to data loss. To understand why, consider a file in which an an error is introduced. Several generations of backups and revisions have since occurred. The error is then detected. At this time, it would be pointless to have all of the most recent generations because all of them have the error. It would instead be beneficial to have at least one of the older generations, as it would not have the error.

DWMY approach
A better approach is to keep generations distributed across all points in time. This can be done by storing daily backups for the past one to two weeks, weekly backups for the past one to two months, monthly backups for the past six months to a year, and yearly backups. Quarterly and/or biannual backups can also be kept.

Weighted random approach
An alternative approach to keeping generations distributed across all points in time is to delete past generations (except the first and the last generation) when necessary in a weighted-random fashion. For each desired deletion, the weight assigned to each of the past generations signifies the probability of it being deleted. One acceptable weight is a constant exponent (possibly the square) of the multiplicative inverse of the duration (possibly expressed in the number of days) between the date of the generation and the generation available before it.

Using a larger exponent leads to a more uniform distribution of generations, whereas a smaller exponent lead to a distribution with more recent and less older generations. While a proof is not offered for this assertion, empirical results suggest it to be true. This technique thus ensures that past generations are always distributed across all points in time as desired.

Physical reasons for hard drive failure
There are several different physical causes of hard drive failure.

Related reading

 * Popdisk backup system
 * Case study of one company's methods
 * Recovering data after a hard disk drive failure
 * How to backup an operating system
 * Choosing the right file format
 * ../Self-Monitoring, Analysis, and Reporting Technology/