[PC-BSD Testing] upgrading.. slower than needs be

Kris Moore kris at pcbsd.org
Mon Dec 5 12:16:44 PST 2011


On 12/05/2011 13:18, Julian Elischer wrote:
> On 12/5/11 8:02 AM, Kris Moore wrote:
>> On 12/01/2011 21:06, Julian Elischer wrote:
>>> On 12/1/11 5:03 PM, Julian Elischer wrote:
>>>> Watching the RC1->RC2 upgrade on my nice shiny new 15Mb/sec internet
>>>> link, I now see that teh upgrade process does one file at a time.
>>>> it would be a lot faster to overlap the downloads if it were
>>>> possible. Does the upgrade process know of more than one file that it
>>>> needs to get?
>>>
>>> One could at least make the fetch of the checksum file be done in
>>> parallel, so that the tcp handshake is done during the transfer of the
>>> main file.
>>> Looking at my system upgrading however it looks as though there is
>>> something else that should be found too.
>>> It's 95 % idle, yet it is only asking for each file, one at a time
>>> with large gaps between them.(10 seconds)?
>>> Disk I doesn't seem to be  a bottleneck either so I can't work out
>>> what the holdup is.
>>
>> I'll see what we can do to speed this up :) Parallel downloads would
>> help for sure, if anything we can do the md5 at the same time.
>
> unfortunately I have now upgraded so I can't do it again to test, but
> I could se no reason as to why it was so slow.
>
> teh disk was barely being touched, and there was only 5% cpu usage,
> and teh network IO was only sporadic.. it would do aburst of high
> bandwidth where it downloaded the file, then there would be a blip
> where it did someting on cpu then a 5 second gap of nothing then the
> md 5 would be received and then another 5 seconds of nothing much, and
> then the cycle would repeat.
>
> I just realised that I took a ktrace of the upgrade and that shoudl
> still be there.  heh.. maybe that has some clues..

Ok, I've removed the slowdowns, it should zip right through them now :)

The only time it "sleeps" is when a failure downloading occurs, and it
waits a few moments before trying again.

BTW, there are some interesting bugs with "fetch" on 9, causing errors like:

fetch:  <mirror>/libXft-2.1.14.tbz.md5: Protocol error
fetch: <mirror>/libXft-2.1.14.tbz.md5: Operation not permitted

These only occur when doing a TON of downloads right after one-another,
not sure what the deal is with them. if we sleep about 30 seconds, it
works again, but if you try to fetch too soon it keeps failing.

-- 
Kris Moore
PC-BSD Software
iXsystems



More information about the Testing mailing list