[PC-BSD Dev] ZFS dedup

Radio młodych bandytów radiomlodychbandytow at o2.pl
Mon Jul 22 13:37:54 PDT 2013


On 22/07/2013 22:05, Kris Moore wrote:
> On 07/22/2013 15:31, Radio młodych bandytów wrote:
>> On 22/07/2013 21:02, Kris Moore wrote:
>>> Plus everything I hear is to avoid dedup at all costs right now.
>> Why?
>>
> 
> From my understanding, dedup stores the matching blocks list entirely
> inside of memory. ZFS being rather memory hungry, its possible to exaust
> this memory pool *very* quickly. The result can be a panic, and losing
> your data on disk, since the memory tables are now lost :(
> 
> For more details you'd have to ask some of the FreeNAS ZFS guys, but
> that was the gist of it. It was a serious enough problem that they
> recommended we not even offer the option, until a solution can be found.
> 
This goes clearly against:
https://www.c0t0d0s0.org/archives/7271-ZFS-Dedup-Internals.html
and against everything I've heard so far.
Maybe they are cached somehow? Maybe FreeBSD Dedupe is different from
Solaris? Maybe it changed at some point? Maybe the blogger (claiming to
work for Oracle) got it wrong? Maybe you or your sources got something
wrong?

The only closest thing I've seen is that a loss of devices and / or data
corruption that results in loss of a part of DDT can lead to a loss of
some or all deduped data. And since DDT is metadata, there are (by
default) 2 copies of it, so the risk should be small.

This is a serious matter and would be good to have cleared up.
-- 
Twoje radio


More information about the Dev mailing list