▲ 770 points
back
329 comments
One key point about retention which is not often mentioned, and indeed neither does this article, is that retention is inversely proportional to program/erase cycles and decreases exponentially with increasing temperature. Hence why retention specs are usually X amount of time after Y cycles at Z temperature. Even a QLC SSD that has only been written to once, and kept in a freezer at -40, may hold data for several decades.
Manufacturers have been playing this game with DWPD/TBW numbers too --- by reducing the retention spec, they can advertise a drive as having a higher endurance with the exact same flash. But if you compare the numbers over the years, it's clear that NAND flash has gotten significantly worse; the only thing that has gone up, multiplicatively, is capacity, while endurance and rentention have both gone down by a few orders of magnitude.
For a long time, 10 years after 100K cycles was the gold standard of SLC flash.
Now we are down to several months after less than 1K cycles for QLC.
I'm sad that drives don't have a 'shutdown' command which writes a few extra bytes of ECC data per page into otherwise empty flash cells.
It turns out that a few extra bytes can turn a 1 year endurance into a 100 year endurance.
Because no one is willing to pay for SLC.
Those QLC NAND chips? Pretty much all of them have an "SLC mode", which treats each cell as 1 bit, and increases both write speeds and reliability massively. But who wants to have 4 times less capacity for the same price?
Endurance going down is hardly a surprise given that the feature size has gone down too. The same goes for logic and DRAM memory.
I suspect that 2035 years time, hardware from 2010 will work, while that from 2020 will be less reliable.
I also seem to remember reading retention is proportional to temperature at time of write. Ie, best case scenario = write data when drive is hot, and store in freezer. Would be happy if someone can confirm or deny this.
That's how it has to work. To increase capacity you have to make smaller cells where charge may easier diffuse from one cell to another. Also to make drive faster, stored charge has to be smaller, which also decrease endurance. With SLC and QLC comparison is even worse as QLC is basically clever hack to store 4 times more data in the same number physical cells - it's tradeoff.
On the other hand when capacity goes up, the cycle-count goes down for the same workload. A 4TB drive after 1K cycles has written the same amount of data as 100GB drive after 40K cycles.
> Even a QLC SSD that has only been written to once, and kept in a freezer at -40, may hold data for several decades.
So literally put your data in cold storage.
So AWS S3 Glacier might actually be cold
So on the off-chance that there's a firmware engineer in here, how does this actually work?
Like does a SSD do some sort of refresh on power-on, or every N hours, or you have to access the specific block, or...? What if you interrupt the process, eg, having a NVMe in an external case that you just plug once a month for a few minutes to just use it as a huge flash drive, is that a problem?
What about the unused space, is a 4 TB drive used to transport 1 GB of stuff going to suffer anything from the unused space decaying?
It's all very unclear about what all of this means in practice and how's an user supposed to manage it.
SSD firmware engineer here. I work on enterprise stuff, so ymmv on consumer grade internals.
Generally, the data refresh will all happen in the background when the system is powered (depending on the power state). Performance is probably throttled during those operations, so you just see a slightly slower copy while this is happening behind the scenes.
The unused space decaying is probably not an issue, since the internal filesystem data is typically stored on a more robust area of media (an SLC location) which is less susceptible to data loss over time.
As far as how a user is supposed to manage it, maybe do an fsck every month or something? Using an SSD like that is probably ok most of the time, but might not be super great as a cold storage backup.
I assume this blog is a re-hash of the JDEC retention standards[1].
The more interesting thing to note from those standards is that the required retention period differs between "Client" and "Enterprise" category.
Enterprise category only has power-off retention requirement of 3 months.
Client category has power-off retention requirement of 1 year.
Of course there are two sides to every story...
Enterprise category standard has a power-on active use of 24 hours/day, but Client category only intended for 8 hours/day.
As with many things in tech.... its up to the user to pick which side they compromise on.
[1]https://files.futurememorystorage.com/proceedings/2011/20110...
We learned this the hard way with "cold" backups stored in a literal safe.
We treated NVMe drives like digital stone tablets. A year later, we tried to restore a critical snapshot and checksums failed everywhere. We now have a policy to power-cycle our cold storage drives every 6 months just to refresh the charge traps.
It's terrifying how ephemeral "permanent" storage actually is. Tape is annoying to manage, but at least it doesn't leak electrons just sitting on a shelf.
The article implies this is not a concern for "regular" people. That is absolutely false. How many people get their family photos when they finally decide to recycle that 15 year old PC in the basement?
How many people have a device that they may only power up ever few years, like on vacation. In fact, I have a device that I've only used on rare occasions these days (an arcade machine) that now I suspect I'll have to reinstall since It's been 2 or 3 years since I've last used it.
This is a pretty big deal that they don't put on the box.
What about powered SSDs that contain files that are rarely read?
My desktop computer is generally powered except when there is a power failure, but among the million+ files on its SSD there are certainly some that I do not read or write for years.
Does the SSD controller automatically look for used blocks that need to have their charge refreshed and do so, or do I need to periodically do something like "find / -type f -print0 | xargs -0 cat > /dev/null" to make sure every file gets read occasionally?
Is there a real source that confirms this with data? I generally like xda, but the quality of their articles is uneven and they trend towards click bait headlines that try to shock/surprise you with thin content underneath. There have been a string of "Here is the one piece of software you didn't know you needed for your NAS" and it turns out to be something extremely popular like home assistant.
This article just seems to link to a series of other xda articles with no primary source. I wouldn't ever trust any single piece of hardware to store my data forever but this feels like clickbait- At one point they even state "...but you shouldn't really worry about it..."
The spinrite[0] user group has noticed some of these effects, even on in-service drives.
The theory is that operating system files, which rarely change, are written and almost never re-written. So the charges begin to decay over time and while they might not be unreadable, reads for these blocks require additional error correction, which reduces performance.
There have been a significant number of (anecdotal) reports that a full rewrite of the drive, which does put wear on the cells, greatly increases the overall performance. I haven't personally experienced this yet, but I do think a "every other year" refresh of data on SSDs makes sense.
what is the exact protocol to "recharge" an ssd which was offline for months?
do I just plug it in and let the computer on for a few minutes? does it needs to stay on for hours?
do I need to run a special command or TRIM it?
This is true, but it is generally true.
Even for UV-EPROMs the retention time can be as low as a 25 years, if kept warm, even with the window sealed correctly.
Magnetic drives are quite a lot better, around 50 years.
CD-RWs are somewhat wider in their stability, I have ~20 year old discs that are becoming unreadable because the actual foil is delaminating from the plastic disc. Meanwhile I have ~40 year old DS-DD floppies that are still fully readable even though their medium is in physical contact with the read/write heads (although here, again, storage conditions and especially the different brands/batches seem to make a difference).
How does one keep an SSD powered without actually mounting it or working with it? I have backups on SSDs and old drives I would like to keep somewhat live.
Should I pop them in an old server? Is there an appliance that just supplies power? Is there a self-hosted thing I can monitor disks which I have 0 access usage for and don't want connected to anything but want to keep "live"
Powering the SSD on isn't enough. You need to read every bit occasionally in order to recharge the cell. If you have them in a NAS, then using a monthly full volume check is probably sufficient.
Is there any type of flash-based storage (preferably accessible to end users) that focuses on long term data retention?
If not, that feels like a substantial hole in the market. Non-flash durable storage tend to be annoying or impractical for day to day use. I want to be able to find a 25 year old SD card hiding in some crevice and unearth an unintentional time capsule, much like how one can pick up 20+ year old MiniDiscs and be able to play the last thing their former owners recorded to them perfectly.
So, product idea: A powered "cold storage box" for M.2 SSDs. 2 to 8 M.2 slots. Periodically, an internal computer connects one of the slots, reads every byte, waits for some period of time, then powers off. Maybe shows a little green light next to each drive when the last read was successful. Could be battery-powered.
I've been hearing about this for years and it makes sense theoretically but has anyone ever actually seen it? What are the errors reported? Or does the drive return bad data but reports no error?
There was a guy on reddit that took about 20 cheap USB flash drives and checked 1 every 6 months. I think after 3 years nothing was bad yet.
I've copied OS ISO images to USB flash drives and I know they sat for at least 2 years unused. Then I used it to install the OS and it worked perfectly fine with no errors reported.
I still have 3 copies of all data and 1 of those copies is offsite but this scare about SSDs losing data is something that I've never actually seen.
Flash is programmed by increasing the probability that electrons will tunnel onto the floating gate and erased by increasing the probability they will tunnel back off. Those probabilities are never zero. Multiply that by time and the number of cells, and the probability you don’t end up with bit errors gets quite low.
The difference between slc and mlc is just that mlc has four different program voltages instead of two, so reading back the data you have to distinguish between charge levels that are closer together. Same basic cell design. Honestly I can’t quite believe mlc works at all, let alone qlc. I do wonder why there’s no way to operate qlc as if it were mlc, other than the manufacturer not wanting to allow it.
A strong believe of mine: there is no storage, only communication. I hold that thought since I first heard of SRAM, and I think it applies to everything, knowledge, technology, societies, our universe in general..
Hmm, so what about these modern high density hard drives which store track parameters for their servos in on-board flash (aka OptiNAND)? Do we get "spinning rust" which might loose the information where exactly it stored the data?
I note , CHEMICAL reactions proceed faster with heat.
So this why data retention is longer if device is in "...cold storage.."
(This situation is also know with old photographic film , if stored in low temperatures it can last cira 100 years, but if stored at much higher room? temperatures , the film deterioration is much faster )
In SSD drives digital bits are stored as a electrical charge amount for each "cell".
I suspect these cells have been decreased in size to provide much larger numerically increased SSD storage amounts. The decreased cell size means the charge decreases below needed amount faster and ssd becomes unreliable.
MORE IMPORTANTLY - I also contend the SSD CHIP DESIGNERS would have/SHOULD HAVE been aware of this size effect and the decreased SSD reliability was not publicised OR specified for the product data sheets.
Presently I'm looking for articales that prove my suspicions or assertions
> But, most people don't need to worry about it. [...] You should always have a backup anyway. [...] Backing up your data is the simplest strategy to counteract the limitations of storage media. Having multiple copies of your data on different types of storage ensures that any unexpected incidents protect your data from vanishing forever. This is exactly what the 3-2-1 backup rule talks about: 3 copies of data on at least 2 different storage media, with 1 copy stored off-site.
Um. Backups seem like exactly why I might have data on an unpowered SSD.
I use HDDs right now because they're cheaper, but that might not be true some day. Also, I would expect someone less technically inclined than I am to just use whatever they have lying around, which may well be an SSD.
Quick note to not store any valuable data on a single drive. And when you store it on two drives, don't use the same kind of drive. (Speaking from bitter experience using spinning drives in servers that had a firmware bug where they all died at the time number of seconds of power-on time).
For years I've been collecting a slowly growing pile of old drives from old devices as I had replaced them.
I finally decided to do something about them and got myself a USB to IDE/SATA adapter and spent a week or so duplicating them so I could check if there was anything worthwhile to keep, before destroyig and sending them to an e-waste recycler.
Some of those drives had been sitting around since early 2000.
All except two of the mechanical drives just fired up and worked fine. One won't power on at all. One powers up, but reading from some sectors just results in failures.
But the SSDs are another story. All blank/empty for any that were more than a few years old. Even though I know some were pulled from working machines.
From the above linked articale
"... Your SSD data isn't as permanent as you think Even the cheapest SSDs, say those with QLC NAND, can safely store data for about a year of being completely unpowered. More expensive TLC NAND can retain data for up to 3 years, while MLC and SLC NAND are good for 5 years and 10 years of unpowered storage, respectively. ..."
RE "...QLC NAND, can safely store data for about a year of being completely unpowered...."
QLC Nand only reliably stores data for only a year .
This would be within the warranty period for some products.
This information should be more clearly shown to product purchasers.
Also, FYI for the one person here who uses raw nand flash: run ubihealthd (https://lwn.net/Articles/663751/).
It will trigger reads in random areas in flash, and try ti correct any errors found.
Without it, the same issue as in the original article will happen (even if the device is powered on): areas in the NAND were not read for long time will have more and more errors, causing them to be non recoverable.
We may be facing a grim situation in a few years because of this. Right now most consumer-grade storage is flash memory, and all of it suffers from this problem. SSDs, pendrives, SD cards, Compact Flash... Apparently games for the Nintendo 3DS and PS Vita are already suffering from this, and people losing photos in faulty SDs is hardly news.
This is a really good case for better file systems with built in error correction and self healing. On linux they have btrfs which kinda does this, and some support for zfs. In the BSD land we have zfs and hammer2. Does NT or Mac have anything like this? I think Mac might have some unofficial zfs support but I don't know the state that's in
I've got some old SSDs just to test this myself, the old 256gb corsairs I tested previously were fine after a year and a half, but I might have misplaced them...(they only had 10% write life left, so no huge loss) the 512gb samsungs on my desk should be getting pretty ripe soon though, I'll have to check those too.
As far as I understand, this even applies to some seemingly read-only storage such as game cartridges, e.g. those for the Nintendo Switch.
Flash storage is apparently cheaper (especially for smaller production runs) and/or higher density these days, so these cartridges just use that and make it appear ROM-like via a controller.
I don't use my drive much. I still boot it up snd write some data, just not the long term one. Am I in risk?
Does this also apply to thumbdrives?
Does the same apply to USB thumb drives, i.e. do they lose their data if not plugged in?
I had to search around and feel like a dork not knowing this. I have my data backed up, but I keep the SSDs because it's nice to have the OS running like it was... I guess I need to be cloning the drives to ISOs and storing on spinning rust.
I thought that was an ancient issue with Samsung 740? I had that one and it was slowly losing speed when unpowered due to an accumulation of errors and rewriting the individual sectors once for the whole drive made it work fine for a year.
Good advice; however, past experience suggests that conventional magnetic hard drives suffer problems of stiction when left in cold storage for too long. I wouldn't trust either technology for long-term archival purposes.
shameless plug of my anti-bitrot tool, which I am actually enhancing with a --daemon mode currently
A solution I haven't yet seen in this thread is to buy multiple drives, and sacrifice the capacity of one of those drives to maintain single parity via a raidz1 configuration with zfs. (raidz2 or raidz3 are likely better, as you can guard against full drive failures as well, but you'd need to increase the number of drives' capacity that you're using for parity.)
zfs in these filesystem-specific parity-raid implementations also auto-repairs corrupted data whenever read, and the scrub utility provides an additional tool for recognizing and correcting such issues proactively.
This applies to both HDDs and SSDs. So, a good option for just about any archival use case.
I was in a team that wrote the firmware to handle that 15 years ago, with focus on automotive implementations where temperature might be high and access to the data is harder.
Making sure all your important data can be read and is checked at read is like zfs scrubs sole purpose. Seems like at least a monthly scrub is a very good idea for SSDs.
What is the best way to store data for a long time then?
This is why I would rather pay someone a couple of dollars per year to handle all this for me. If need be pay two providers to have a backup.