What is the best way to prevent data corruption for EXT4 filesystems? Specifically while copying. I have found out that over the years, I have several TB worth of corrupted data. I don't know if there is a problem with copying, or if somehow over time the drives themselves are becoming corrupt. I'm at the point where I need to start thinking about building another NAS storage unit, so I want to do it right the first time. I know Winblows has programs like Robocopy, and when I did some research, I found some info on how to do it in the terminal, but it was beyond confusing.
I'm missing some factory service information (not cheap at all), source code to programs I have written, some digital copies of certifications I have, videos I made, and pictures. Some of this data can't be recreated.
I have 3 NAS units (one of them I think the PSU went out on), and I do daily backups, daily redundant backups, daily redundant off site backups, and super important stuff also went to Dropbox and my Email (Dropbox is no longer free, so I can't use it anymore).
Every once in a while I get a red HD light on my 16TB raid unit. I need to get the data off these 8 drives as fast as possible!
I'm thinking about just getting a single 16TB drive because I don't think I'll be any worse off than 8 drives in RAID 0. Just get a damn good one. Plus I could do SMART data with one.
When I make a move, I want to keep the drives on the old enclosure to a minimum of on time. I have these babies on order. For cost effectiveness, a 8 bay raid with USB 3.1 is $300. Cheap 2TB drives are $50X8. So I'd be about $350-$400 ahead doing it this way. Maybe in a few months the price will come down and even bigger drives will be more cost effective. I know read and write times on a HDD that big is very slow.
Thoughts? Oh, and I think I will buy the 250 GB external SSD for new data until I can muster up enough dosh until I get a replacement for the old data.
I don't know. I can't say I've ever had a corruption problem with ext4. I'm not entirely sure I'd trust USB in regards to corruption. I think ethernet with an appropriate protocol would be more reliable. I guess a big question is whether or not data is being corrupted in transit, or after it gets to the disk.
You could copy then run a checksum on original and copy, to make sure it copied correctly. I'm not sure how to do something like that automatically, but I dont' think it would be too difficult to write a bash script for it.
. . . my, my, what big DISKS you have . . .
Maybe he just needs to fsck his disks!
@bingst . . . kicking myself for not thinking of that myself.
@FearlessFly I used to chat with a guy on IRC who said "fscking" a lot. lol