Picture Archival Scheme
Digital photos should be treated like any other precious data. Companies go through extensive, sophisticated and sometimes highly complex methods of protecting their data. We as individuals can learn a lot from them in terms of disaster recovery and business continuance plans. While they generally have a lot more financial resources to throw at the problem, some of their solutions can be scaled down and made feasible for individuals to implement.
For internal drives, I've had pretty good luck with Seagate, IBM and Hitachi. I have had a bunch of 18GB and 9GB SCSI Hitachi drives that had been spinning pretty much for 10 years straight in one of my old servers. For Seagate drives, I recommend the AS type drives for workstations, ES for servers and NS for storage arrays.
As for external storage, it depends on the requirements. Do you need it to be highly portable, luggable or fixed? I have all three.
[0] Portable - This drive is meant to go where I go and provides a means of extended normally acquiesced storage for in-the-field and on-the-road backups to data normally on my laptop. I'm still using an old Apricorn Aegis Bio 160GB for this solution. It has built-in security that's biometrically protected for access as well as data encryption. The biometrics is all on the drive unit itself so it's not tied to any specific laptop or operating system. It appears as a USB mass storage device to the host. It is powered through the USB connection. I need the security because I also have sensitive data for work. I'll probably snag myself a larger drive at some point but honestly, I don't need much carry-around storage since I regularly sync back to the "mothership" anyways.
[1] Luggable - This type of drive is externally powered and is usually a bit larger. It may have several ways to connect to the host including Firewire, USB and SATA. I have a couple of Western Digital MyBook drives for this. I don't carry them around. I actually use them as backup media for my network storage array. Basically they've replaced my tape-backup solution. I simply hook up the drive, start my backups and then unhook the drive and store them away. I have several of them so I can rotate them to adhere to my backup and retention strategy.
[2] Network Attached Storage (NAS) - This is a fixed location, typically multidrive array that is accessed and accessible over the network. I run a large number of computers at home and so I needed some kind of fileserver. For this, I use a QNAP TS-869 Pro. It has eight drive bays and is populated with 3TB drives configured into RAID-5 volumes. I also have a second NAS which acts as my backup NAS that mirrors my primary NAS. In the event of a total primary NAS failure, I can shift mounts to my back NAS. The backup NAS is a Netgear ReadyNAS NV+ (formerly Infrant before Netgear bought them). It holds 4 drives and is currently configured with 2TB drives. It supports several forms of RAID from simple striping to mirroring to RAID-5 with hot-standby. It also supports its own proprietary X-RAID. Both NAS appliances allow for organic growth of the array without incurring substantial rebuilding and downtime of the volumes. You simply add drives as you need and the thing grows in size... even while hot. Files are accessible via many forms of transport protocol including NFS (my main priority), CIFS, AFS, Windows sharing, HTTP/HTTPS and FTP. They provide fault-tolerant highly redundant solutions. They also include a built-in backup system which is what I use with the WD MyBooks to achieve nightly incremental and weekly full backups. They don't come with a card reader but they do act as a USB-host so a USB card reader can be attached to it. Any USB mass storage devices (including my DSLR) can be attached to them and exported over the network. They can also act as USB print servers if need be but my printers are already network capable. They support a bunch of software plugins that can be installed to make them do other stuff too such as act as a security camera DVR..
It's probably a good idea to develop an archiving method too. Mine involves a mixture of online, nearline and offline storage with backups.
Nearline field storage includes my laptop's drive and a portable external drive. The laptop has a 320GB drive and the portable external drive is 160GB in capacity. Online and nearline home storage is a 4x1TB network attached storage array running proprietary X-RAID (like RAID-5) with a hot-spare drive. Offline backups are done with external HDs that are rotated into place for nightly incremental and weekly full backups. Current retention schema is 4 weeks of backups with a one week offsite rotation (performed monthly).
Note that if you go the NAS route, many of them have integrated multiple concurrent backup solutions. For instance, my ReadyNAS supports both local and remote nearline storage mechanisms and has a built-in multi-job backup manager. It can be set to sync to another NAS, to attached JBODs (Just a Bunch Of Disks) and/or streamed backup to an offsite backup provider such as their own cloud-based ReadyNAS Vault service which itself can support multiple devices. So when investigating NAS solutions, also pay attention to what it offers as far as integrated backup.
But as with anything, you must first develop your backup and data recovery strategy and policy. Once you've done that then you can develop a plan for implementing it. Once you've done that then you can settle upon what components you will need to purchase and set up.
One key thing to consider is your retention needs. Do you require that all data must be kept and archived or is it okay to overwrite, rotate or delete from archive? Do you need to only keep the most up-to-date copy of your data or do you need version control starting from data birth? How long can you survive with a loss of data? What is your critical recovery time threshold? How many users will be effected? How much data are you willing to lose? If you lose your last week's worth of work for an hour, is it a tragedy or can you live with the time it takes to drive to the safety deposit box at your bank to get the monthly backup drive? Will streaming back 500GB of data from an offsite network backup provider at 10Mbps simply be too long? Do you need to rebuild from bare metal or do you only need critical data backed up?
I only keep NEFs. I create JPEGs and TIFFs only for distribution and have a special Exports directory where I keep them. I then I get rid of them after a short period of time... usually within a month. I use CNX so I can manage versioning within the NEFs since only CNX can write-back NEFs. If I create a JPEG or TIFF I like, I will create a version of the CNX edit steps with the name of that JPEG/TIFF. This way I don't need to keep the JPEG/TIFF. I can always easily recreate it from the NEF version. Because NEFs support multiple versions within the same file, there is no excess space usage by having different versions of the edit steps.
I don't suggest just blindly adopting one specific setup. You have to analyse your needs, your resources, your budget, your comfort with complexity and figure out what works best for you and more importantly, you need to understand the details. Knowing where and how your data lives is very important when it comes to ensuring survivability and disaster recovery.
Picture Archival Scheme
Digital photos should be treated like any other precious data. Companies go through extensive, sophisticated and sometimes highly complex methods of protecting their data. We as individuals can learn a lot from them in terms of disaster recovery and business continuance plans. While they generally have a lot more financial resources to throw at the problem, some of their solutions can be scaled down and made feasible for individuals to implement.
For internal drives, I've had pretty good luck with Seagate, IBM and Hitachi. I have had a bunch of 18GB and 9GB SCSI Hitachi drives that had been spinning pretty much for 10 years straight in one of my old servers. For Seagate drives, I recommend the AS type drives for workstations, ES for servers and NS for storage arrays.
As for external storage, it depends on the requirements. Do you need it to be highly portable, luggable or fixed? I have all three.
[0] Portable - This drive is meant to go where I go and provides a means of extended normally acquiesced storage for in-the-field and on-the-road backups to data normally on my laptop. I'm still using an old Apricorn Aegis Bio 160GB for this solution. It has built-in security that's biometrically protected for access as well as data encryption. The biometrics is all on the drive unit itself so it's not tied to any specific laptop or operating system. It appears as a USB mass storage device to the host. It is powered through the USB connection. I need the security because I also have sensitive data for work. I'll probably snag myself a larger drive at some point but honestly, I don't need much carry-around storage since I regularly sync back to the "mothership" anyways.
[1] Luggable - This type of drive is externally powered and is usually a bit larger. It may have several ways to connect to the host including Firewire, USB and SATA. I have a couple of Western Digital MyBook drives for this. I don't carry them around. I actually use them as backup media for my network storage array. Basically they've replaced my tape-backup solution. I simply hook up the drive, start my backups and then unhook the drive and store them away. I have several of them so I can rotate them to adhere to my backup and retention strategy.
[2] Network Attached Storage (NAS) - This is a fixed location, typically multidrive array that is accessed and accessible over the network. I run a large number of computers at home and so I needed some kind of fileserver. For this, I use a QNAP TS-869 Pro. It has eight drive bays and is populated with 3TB drives configured into RAID-5 volumes. I also have a second NAS which acts as my backup NAS that mirrors my primary NAS. In the event of a total primary NAS failure, I can shift mounts to my back NAS. The backup NAS is a Netgear ReadyNAS NV+ (formerly Infrant before Netgear bought them). It holds 4 drives and is currently configured with 2TB drives. It supports several forms of RAID from simple striping to mirroring to RAID-5 with hot-standby. It also supports its own proprietary X-RAID. Both NAS appliances allow for organic growth of the array without incurring substantial rebuilding and downtime of the volumes. You simply add drives as you need and the thing grows in size... even while hot. Files are accessible via many forms of transport protocol including NFS (my main priority), CIFS, AFS, Windows sharing, HTTP/HTTPS and FTP. They provide fault-tolerant highly redundant solutions. They also include a built-in backup system which is what I use with the WD MyBooks to achieve nightly incremental and weekly full backups. They don't come with a card reader but they do act as a USB-host so a USB card reader can be attached to it. Any USB mass storage devices (including my DSLR) can be attached to them and exported over the network. They can also act as USB print servers if need be but my printers are already network capable. They support a bunch of software plugins that can be installed to make them do other stuff too such as act as a security camera DVR..
It's probably a good idea to develop an archiving method too. Mine involves a mixture of online, nearline and offline storage with backups.
Nearline field storage includes my laptop's drive and a portable external drive. The laptop has a 320GB drive and the portable external drive is 160GB in capacity. Online and nearline home storage is a 4x1TB network attached storage array running proprietary X-RAID (like RAID-5) with a hot-spare drive. Offline backups are done with external HDs that are rotated into place for nightly incremental and weekly full backups. Current retention schema is 4 weeks of backups with a one week offsite rotation (performed monthly).
Note that if you go the NAS route, many of them have integrated multiple concurrent backup solutions. For instance, my ReadyNAS supports both local and remote nearline storage mechanisms and has a built-in multi-job backup manager. It can be set to sync to another NAS, to attached JBODs (Just a Bunch Of Disks) and/or streamed backup to an offsite backup provider such as their own cloud-based ReadyNAS Vault service which itself can support multiple devices. So when investigating NAS solutions, also pay attention to what it offers as far as integrated backup.
But as with anything, you must first develop your backup and data recovery strategy and policy. Once you've done that then you can develop a plan for implementing it. Once you've done that then you can settle upon what components you will need to purchase and set up.
One key thing to consider is your retention needs. Do you require that all data must be kept and archived or is it okay to overwrite, rotate or delete from archive? Do you need to only keep the most up-to-date copy of your data or do you need version control starting from data birth? How long can you survive with a loss of data? What is your critical recovery time threshold? How many users will be effected? How much data are you willing to lose? If you lose your last week's worth of work for an hour, is it a tragedy or can you live with the time it takes to drive to the safety deposit box at your bank to get the monthly backup drive? Will streaming back 500GB of data from an offsite network backup provider at 10Mbps simply be too long? Do you need to rebuild from bare metal or do you only need critical data backed up?
I only keep NEFs. I create JPEGs and TIFFs only for distribution and have a special Exports directory where I keep them. I then I get rid of them after a short period of time... usually within a month. I use CNX so I can manage versioning within the NEFs since only CNX can write-back NEFs. If I create a JPEG or TIFF I like, I will create a version of the CNX edit steps with the name of that JPEG/TIFF. This way I don't need to keep the JPEG/TIFF. I can always easily recreate it from the NEF version. Because NEFs support multiple versions within the same file, there is no excess space usage by having different versions of the edit steps.
I don't suggest just blindly adopting one specific setup. You have to analyse your needs, your resources, your budget, your comfort with complexity and figure out what works best for you and more importantly, you need to understand the details. Knowing where and how your data lives is very important when it comes to ensuring survivability and disaster recovery.