[PC-BSD Testing] upgrading.. slower than needs be

Kris Moore kris at pcbsd.org
Mon Dec 5 12:16:44 PST 2011

On 12/05/2011 13:18, Julian Elischer wrote:
> On 12/5/11 8:02 AM, Kris Moore wrote:
>> On 12/01/2011 21:06, Julian Elischer wrote:
>>> On 12/1/11 5:03 PM, Julian Elischer wrote:
>>>> Watching the RC1->RC2 upgrade on my nice shiny new 15Mb/sec internet
>>>> link, I now see that teh upgrade process does one file at a time.
>>>> it would be a lot faster to overlap the downloads if it were
>>>> possible. Does the upgrade process know of more than one file that it
>>>> needs to get?
>>> One could at least make the fetch of the checksum file be done in
>>> parallel, so that the tcp handshake is done during the transfer of the
>>> main file.
>>> Looking at my system upgrading however it looks as though there is
>>> something else that should be found too.
>>> It's 95 % idle, yet it is only asking for each file, one at a time
>>> with large gaps between them.(10 seconds)?
>>> Disk I doesn't seem to be  a bottleneck either so I can't work out
>>> what the holdup is.
>> I'll see what we can do to speed this up :) Parallel downloads would
>> help for sure, if anything we can do the md5 at the same time.
> unfortunately I have now upgraded so I can't do it again to test, but
> I could se no reason as to why it was so slow.
> teh disk was barely being touched, and there was only 5% cpu usage,
> and teh network IO was only sporadic.. it would do aburst of high
> bandwidth where it downloaded the file, then there would be a blip
> where it did someting on cpu then a 5 second gap of nothing then the
> md 5 would be received and then another 5 seconds of nothing much, and
> then the cycle would repeat.
> I just realised that I took a ktrace of the upgrade and that shoudl
> still be there.  heh.. maybe that has some clues..

Ok, I've removed the slowdowns, it should zip right through them now :)

The only time it "sleeps" is when a failure downloading occurs, and it
waits a few moments before trying again.

BTW, there are some interesting bugs with "fetch" on 9, causing errors like:

fetch:  <mirror>/libXft-2.1.14.tbz.md5: Protocol error
fetch: <mirror>/libXft-2.1.14.tbz.md5: Operation not permitted

These only occur when doing a TON of downloads right after one-another,
not sure what the deal is with them. if we sleep about 30 seconds, it
works again, but if you try to fetch too soon it keeps failing.

Kris Moore
PC-BSD Software

More information about the Testing mailing list