<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
On Dec.23 I shared the following the Dru in #PCBSD. It was written
by my co-worker on Oct.10. She requested that he post it here.
Since I follow this list, and he doesn't I am going to post it here
for him.<br>
<br>
Thanks.<br>
Lars<br>
<br>
FreeBSD needs work<br>
last modified on 2010-10-10 at 19:33 - keywords: freebsd zfs iscsi
whining<br>
<br>
Warning: This is not a post whining about Adobe Flash or how hard it
is to get my playstation controller to work in Quake.<br>
<br>
This is, instead, a post whining about how hard it is to use ZFS and
iSCSI on FreeBSD. Recently I was tasked with building some
storage-related projects. The tools of choice involved FreeBSD
sitting on some hardware with a large collection of high-volume
drives. These drives were presented JBOD-style to the host OS, and a
couple of them were SSDs. The idea was to make one massive raidz2
zpool with a mirrored ZIL on the SSDs. Then we could spin off zvols
and pitch them via iSCSI to our application and database servers.<br>
<br>
Sound simple? It isn't. To get it working, we had to switch to
OpenSolaris. Which is a bit of a damn problem.<br>
<br>
There were two really bad problems in FreeBSD that prevented us from
deploying our OS of choice to these systems.<br>
<br>
Problem the first:<br>
ZFS on FreeBSD (at zpool version 13 as of this project) is only like
80% there. What I mean by that is ZFS is friggin fantastic for your
massive desktop nerdmachine, or your creepy 8TB basement "media
archive", or any other low-traffic consumer-grade project you might
want to undertake. But when you start putting some load on it...<br>
<br>
The biggest problem I had seemed to be caused by multiple read
operations during snapshot creation. If you took a snapshot while
lots of iSCSI (or NFS) (or local) access is going on, the zfs
process gets stuck with a wait state. What it boils down to is that
you have to be super careful to disable access while you're creating
(or, in bad cases, sending) snapshots off -- which completely wrecks
the point of the damn snapshot functionality.<br>
<br>
Oh, and it would crash BSD too. Sometimes it would just eat all the
RAM and the machine would fall down.<br>
<br>
Problem the second:<br>
iSCSI support in FreeBSD is abominable. I'm given to understand this
is because the main iSCSI dev in FreeBSD-land is possessed of
insufficient hardware to model high-performance workloads. If that's
the case, we need to get that man some damn hardware. Or convince
Covad to update their AoE stuff, since AoE is nicer anyway.<br>
<br>
If the iSCSI traffic crossed subnets, performance would tank. If the
iSCSI targets were accessed from a non-FreeBSD initiator,
performance would tank. If a FreeBSD initiator accesses a
non-FreeBSD target, performance tanks. Are you seeing a pattern
here? Best-practices objections aside, it's clear that the dev has a
handful of machines on a dumb switch, and that's the test platform.
As soon as you instantiate some sophistication, the whole thing
falls down. Again.<br>
How do we fix this?<br>
<br>
I suppose I could devote some time to mastering the implementation
details of iSCSI and ZFS, then fix the stuff myself. That's not
really the best use of my time, however, and I'm not in a position
to get paid to do that sort of thing. But there are a few things
anyone (myself included) can do:<br>
test.<br>
<br>
* report bugs.<br>
* provide stack traces and failure scenarios.<br>
* whine constantly.<br>
* provide testing hardware.<br>
<br>
All of these are helpful, especially the constant whining. The
problem needs to be front-and-center, or it'll get de-prioritized in
favor of other (and in my opinion) less important problems. I don't
give a crap that your video card doesn't push hard enough to run EVE
in FreeBSD. I want the ZFS functionality that now only FreeBSD
actively provides, and I want the iSCSI functionality ZFS was
designed to enable.<br>
<span class="Apple-style-span" style="border-collapse: separate;
color: rgb(0, 0, 0); font-family: sans; font-style: normal;
font-variant: normal; font-weight: normal; letter-spacing: normal;
line-height: normal; orphans: 2; text-indent: 0px; text-transform:
none; white-space: normal; widows: 2; word-spacing: 0px;
font-size: medium;"><span class="Apple-style-span" style="color:
rgb(220, 220, 204);"></span></span><br>
</body>
</html>