SSDs are pretty pricey for video. I use HDDs, mirrored. For some uses I put a SSD caching layer on top to speed up frequent R/W. Using only LVM, no fancy RAID hardware or anything.
I upgraded my datahoarding server to a pair of 18TB hard drives on ZFS with mirroring a little while back. It’ll be several years before I need to upgrade again, but I expect that when I do,SSDs will be cheap enough to go that route.
Already have a 10Gbps fiber connection to that server, so the hard drives are the bottleneck.
Commercially pressed discs don’t last forever, but longer than burnable discs.IIRC, they used to say 50 years for CDs, but in practice, it was a lot less. More like 20 or 30 if you store and handle them nicely. Easily less than 10 if you don’t.
Hard drives go bad over time; I don’t like trusting spinning platters much over 7 years. They can be OK, but they can suddenly stop working whenever.
Hard drives break down from use, not from sitting around. We aren’t talking about SSDs which while they don’t break down will experience data corruption over time. It’s not really a gamble at all with mirrored drives.
You’re also telling me things I already know. I already use ZFS. I agree that you should be using something with data integrity protection. Though ZFS isn’t always what you want for archival purposes.
Magnetic platters absolutely do break down from sitting around. Bearings and other mechanics can also go bad. For those things, a professional recovery operation could still get the data if you’re willing to pay, but the drive itself should be thrown out.
Edit: keep in mind that with bit rot, the drive may superficially function just fine. Your data may even be 99% correct. That 1%, however, could cause unrecoverable problems, such as videos that glitch in the middle.
That’s why you use multiple drives with bitrot protection. Modern SSDs and HDDs have protections against bitrot built in, including internal checksums.
If you are running your hard drives once in a while, then bearing failure isn’t really a concern. You probably should be doing that anyway to refresh the data and make sure it doesn’t degrade. Regardless people have had 10 year old drives of older spin up first time. It’s not likely you are going to have a mechanical issue on multiple drives anyway.
If you refresh an SSD once every couple of years it will last decades.
You keep doing this thing where you presume I don’t know about some issue. Rather I know about these things, but they have fairly easy mitigations or are already solved.
You keep doing this thing where you presume I don’t know about some issue
Maybe because you way overestimate the reliability of old drives. Yes,10 year old drives can work. Doesn’t mean you should trust them with anything other than getting the data off of it.
How do SSDs and HDDs compare to optical disks in terms of stability in storage? SSD bits can lose charge over time until a lot of 1s read as 0s, right?
SSDs are pretty pricey for video. I use HDDs, mirrored. For some uses I put a SSD caching layer on top to speed up frequent R/W. Using only LVM, no fancy RAID hardware or anything.
ZFS FTW
I upgraded my datahoarding server to a pair of 18TB hard drives on ZFS with mirroring a little while back. It’ll be several years before I need to upgrade again, but I expect that when I do, SSDs will be cheap enough to go that route.
Already have a 10Gbps fiber connection to that server, so the hard drives are the bottleneck.
Commercially pressed discs don’t last forever, but longer than burnable discs. IIRC, they used to say 50 years for CDs, but in practice, it was a lot less. More like 20 or 30 if you store and handle them nicely. Easily less than 10 if you don’t.
Hard drives go bad over time; I don’t like trusting spinning platters much over 7 years. They can be OK, but they can suddenly stop working whenever.
SSDs are about the same as spinning platters.
I think we are talking about archival storage rather than storage in use. In which case hard drives can last decades.
I wouldn’t trust it that way, no. They might last decades. They also might not. It’s a gamble on any single drive, or even a few mirrored drives.
File system also matters. Modern ZFS has error checking that can handle some level of bit rot. Older formats generally don’t.
If it’s over 7 years or so, I want to get the data off of there.
Hard drives break down from use, not from sitting around. We aren’t talking about SSDs which while they don’t break down will experience data corruption over time. It’s not really a gamble at all with mirrored drives.
You’re also telling me things I already know. I already use ZFS. I agree that you should be using something with data integrity protection. Though ZFS isn’t always what you want for archival purposes.
Magnetic platters absolutely do break down from sitting around. Bearings and other mechanics can also go bad. For those things, a professional recovery operation could still get the data if you’re willing to pay, but the drive itself should be thrown out.
Edit: keep in mind that with bit rot, the drive may superficially function just fine. Your data may even be 99% correct. That 1%, however, could cause unrecoverable problems, such as videos that glitch in the middle.
That’s why you use multiple drives with bitrot protection. Modern SSDs and HDDs have protections against bitrot built in, including internal checksums.
If you are running your hard drives once in a while, then bearing failure isn’t really a concern. You probably should be doing that anyway to refresh the data and make sure it doesn’t degrade. Regardless people have had 10 year old drives of older spin up first time. It’s not likely you are going to have a mechanical issue on multiple drives anyway.
If you refresh an SSD once every couple of years it will last decades.
You keep doing this thing where you presume I don’t know about some issue. Rather I know about these things, but they have fairly easy mitigations or are already solved.
Maybe because you way overestimate the reliability of old drives. Yes, 10 year old drives can work. Doesn’t mean you should trust them with anything other than getting the data off of it.