[PC-BSD Dev] Binary patch support for updates.

Kris Moore kris at pcbsd.com
Wed Apr 22 07:10:43 PDT 2009

Dylan Cochran wrote:
> I was looking into adding support for bsdiff to the system updates and
> PBIs, however, I am missing two bits of information.
> 1): I can find no definition of the current PBI file format, other
> then it is the concantenation of the pbistart binary, a header, a png
> file, a header, and two tarballs. To do binary updating properly, you
> need to first decompress any of the input files, as compressed input
> yields patches of the same size as the output file (which defeats the
> purpose). So I have to start by writing a PBI file format unpacker to
> break it into uncompressed components, and then update those. If there
> is any information about querying for the size of each element, and
> the 'proper' way to parse the input file, it would be a great help.


I don't really have any docs on how to query the elements, but this is 
how pbistart does it:

strcpy(awkcmd, "awk '/^__PBI_ARCHIVE__/ { print NR + 1; exit 0; }' '");

We first fine the __PBI_ARCHIVE__ flag, which indicates that we've found 
the first small tarbal, which contains the PBI details / configuration.

Then to extract this data we use tar

# tail +<offset as found above> <file> | head -n <lengh of tar ball> 
tar xvf -C <tmp dir> -f -

During the PBI installation, we then do the same type process for 
__PBI_ARCHIVE2__, which is the flag that indicates we're at the tar / 
lzma compressed file.


> 2): I am unable to find any program or script that generates the .upd
> files and the update tarballs. The system updates look to be the
> simplest ones to get started on, as they are guarenteed to have an
> existing reference point, they are simple collections of files, and
> use an extensible metadata format that are simpler to add the extra
> information to. I want to get started on that, but I need the update
> generation and pushing script/programs.

Sadly, there isn't any auto-generator for .upd files yet :( I'm writing 
them by hand, when we need to issue an update, which isn't all that bad, 
since they are fairly easy to make, only a few lines.

The update tarbals are also rolled by hand, we simply put the data we 
want into a patch, and create the update.sh script, which actually 
performs the update, notifies users, etc. Then to make it, I use this 

# tar cvf - . | lzma e -si ../patch-<date>-<name>.lzma

I guess the first thing to do would be to write some sort of generation 
program, correct? I think it'll still need some manual input though, for 
cases when we are updating files which we need to notify the user about, 
or prompt for some reason?


Kris Moore
PC-BSD Software

More information about the Dev mailing list