[PC-BSD Dev] pcbsd system source idea and question

Joe Maloney jpm820 at gmail.com
Thu Dec 19 07:47:54 PST 2013


On Thu, 2013-12-19 at 09:16 -0500, Kris Moore wrote:
> On 12/19/2013 03:12, Joe Maloney wrote:
> 
> > Here is what I've been able to come up with now for a more simple
> > version.
> > 
> > 
> > http://pastebin.com/7PHASL4S
> > 
> > 
> > I've been testing and it works but it takes about 25 minutes to
> > fetch all the source.  
> > 
> > 
> > I had to use something like this to make up for not being able to
> > run to instances of prog, args in startPorts to run two git commands
> > on after another.:
> > 
> > 
> > system("git --git-dir=/usr/ports/.git fetch" ); 
> > 
> > 
> > Otherwise they would run at the same time which would be bad.  I'm
> > not sure of a better way to do it at this point but my gut tells me
> > using system() might not be a great idea.
> > 
> > 
> > I haven't commited this yet as it's really slow and I figured I
> > would see what your thoughts were first.
> > 
> > 
> > Joe Maloney
> > 
> > 
> > 
> > 
> So, to speed it up, I would add --depth=1, which will greatly speed up
> the initial checkout. 
> 
> # git clone --depth=1 https://github.com/pcbsd/freebsd-ports.git
> 
> I do that on my builders here and it took the checkout from 20 minutes
> down to about 45 seconds :)

OMG that made a huge difference.  27 mb vs 725mb lol.  Thanks for the
tip.
> 
> As for using "system" it may be better to use QProcess::execute()
> instead. 

I will test using this as well some more over the next few days.

> 
> > 
> > 
> > On Wed, Dec 18, 2013 at 11:52 AM, Kris Moore <kris at pcbsd.org> wrote:
> >         On 12/14/2013 12:56, Joe Maloney wrote:
> >         
> >         > Ok since this is going to take me a while here is what I
> >         > have so far. 
> >         > 
> >         > 
> >         > portsnapprogress.h
> >         > http://pastebin.com/yDLMJnhk
> >         > 
> >         > portsnapprogress.cpp
> >         > http://pastebin.com/S8BsmpSr
> >         > 
> >         > 
> >         > When I first started this I ran into the limitation with
> >         > git that it can’t clone into a dir like /usr/ports/ which
> >         > has an existing dir like /usr/ports/distfiles.  Which also
> >         > is a dataset.  Otherwise this would have been easy by just
> >         > using your example for fetchsource() and renaming the
> >         > buttons in portsnapprogress.ui.  
> >         > 
> >         > 
> >         > The only way to fix this in git was to change some things
> >         > in the global configuration for git so I didn’t think that
> >         > would be a good idea.  I didn’t think running git clone to
> >         > a temp dir and moving it was a really clean way of doing
> >         > it either.  So I started with the idea that I would init a
> >         > fresh repo in /usr/ports.  Then add the pcbsd ports tree
> >         > as an upstream repo.  Then fetch the ports and merge
> >         > them.  This way if /usr/ports/.git/config existed with an
> >         > upstream repo I could just fetch and merge only the
> >         > updates and not have to re-clone the entire thing.  I
> >         > think this could work for fetchsource() also if you wanted
> >         > to only grab the updates quickly each time.
> >         > 
> >         > 
> >         > What I’m missing is the code to
> >         > check /usr/ports/.git/config to check to see if the
> >         > upstream repo has been added, add it, otherwise continue
> >         > on to fetch ports.  Then continue on to fetchports()
> >         > mergeports().  Also I’ve ripped out some of the functions
> >         > the fetch done stuff at the end of the script while I was
> >         > testing.
> >         > 
> >         > 
> >         > Although I’ve learned a few things so far I’m by no means
> >         > an expert in qt.  So I figured I would pass along what
> >         > I’ve been able to figure out so far.  I see that some
> >         > similar things already with pcdm as far as parsing config
> >         > files with qt so I suspect someone will know how to finish
> >         > this.  I just wasn’t sure how to implement that type of
> >         > function for what I was trying to do here.  I will
> >         > continue researching it but if someone is able to finish
> >         > this before I do or do it another way I will just learn
> >         > from the examples.  
> >         > 
> >         > 
> >         > Joe Maloney
> >         > 
> >         > 
> >         > 
> >         > 
> >         
> >         
> >         So, I'm a big fan of "simple" solutions :)
> >         
> >         Maybe what you do is this. Have 2 buttons, "Init ports tree"
> >         and "update ports tree"
> >         
> >         The init one will do a destructive clone of our ports tree
> >         into /usr/ports, removing and replacing whatever is already
> >         there. (That is what you would run the first time of course)
> >         
> >         Then if you detect we have /usr/ports/.git, enable the
> >         "update ports tree" button which just does a "git pull
> >         origin master".  
> >         
> >         If the user is already manually checking out files
> >         into /usr/ports from elsewhere, then I would expect them to
> >         continue managing it from the CLI or other methods. This
> >         would be more for new users who want to get an initial
> >         checkout, update, or blow-away a bad ports tree they
> >         mangled. 
> >         
> >         
> >         -- 
> >         Kris Moore
> >         PC-BSD Software
> >         iXsystems
> >         
> >         _______________________________________________
> >         Dev mailing list
> >         Dev at lists.pcbsd.org
> >         http://lists.pcbsd.org/mailman/listinfo/dev
> >         
> > 
> > 
> > 
> > 
> > _______________________________________________
> > Dev mailing list
> > Dev at lists.pcbsd.org
> > http://lists.pcbsd.org/mailman/listinfo/dev
> 
> 
> -- 
> Kris Moore
> PC-BSD Software
> iXsystems
> _______________________________________________
> Dev mailing list
> Dev at lists.pcbsd.org
> http://lists.pcbsd.org/mailman/listinfo/dev




More information about the Dev mailing list