[PC-BSD Dev] pcbsd system source idea and question

Joe Maloney jpm820 at gmail.com
Fri Dec 27 15:54:59 PST 2013

Ok here is what I have been able to come up with so far.  I’m confident there is still not quite right but this is just to show you where I am at a stopping point.

I was also considering renaming all portsnapprogress* files of course once it’s functional to maybe something like gitprogress.h, gitprogress,cpp, gitprogress.ui since it’s not really portsnap anymore.  Then call the portsnapproc something else of course as well like gitProc or something like that.  

portsnapprogress.h -> http://pastebin.com/TF6bSKFz

portsnapprogress.cpp -> http://pastebin.com/aL7mfgqi

pbsystemtab.cpp -> http://pastebin.com/NtqEM35Z

pbsystemtab.h -> http://pastebin.com/Y2yYczzb

The above compiles fine if i remove the show function from pbsystemtab.cpp under the fetchSourcePressed and fetchPortsPressed sections .  You will see that I added init back into public to make it compile in portsnaprogress.h.  Otherwise pbsystemtab.cpp complained that no member named init was found.  Even though I had pointed portnsapProc to PortsnapUI in pbsystemtab.h


Also I know that I will need to find a way to get branch detection to work with the new methods.  The old method was using this for source.  I’m not sure how to fit this in with what I have for startSource now.  

void PortsnapProgress::startSource(QString pcVer)
    label->setText(tr("Downloading FreeBSD sources..."));

    QString branch;
    // Figure out which to download
    if ( pcVer.indexOf("STABLE") )
       branch = "stable/" + pcVer.section("-", 0, 0).section(".", 0, 0); 
    else if ( pcVer.indexOf(".") != -1 )
       branch = "releng/" + pcVer.section("-", 0, 0); 
       branch = "master";

    QStringList args;
    QString prog;
    prog = "git";
    args << "clone" << "https://github.com/pcbsd/freebsd.git" << "-b" << branch << "/usr/src";
    qDebug() << args;
    portsnap = new QProcess(this);
    connect(portsnap, SIGNAL(readyReadStandardOutput()), this, SLOT(parseUpdate()));
    connect(portsnap, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(fetchDone()));
    portsnap->start(prog, args);

On Dec 26, 2013, at 9:52 AM, Ken Moore <ken at pcbsd.org> wrote:

> On 12/25/13 14:29, Joe Maloney wrote:
>> Merry Christmas.  I’ve had some success after modifying a few syntaxes here and there getting some of this to compile and the GUI to interact with the commands.
>> I do have some more questions just to make sure I understand this right.  For example the portsnapproc.h file you sent could essentially just replace whats in portsnapprogress.h?  Please see my comments below your replies for more questions.
> Almost: the portsnapProc class would simply replace all the QProcess and backend stuff within portsnapprogress. What would still need to be in portsnapprogress is all the GUI stuff (since I am assuming that portsnapProgress is some kind of pop-up dialog that reports the current process output).
> Basically you would just need to do this for portsnapprogress.
> 1) Still have pbsystemtab call your portSnapProgress class (since that is basically opening up the new dialog) just the way you already had it set up.
> 2) In the portsnapprogress::init(bool, QString) function you will need to add the PortsnapProc initialization code I wrote before (changing the GUI variable names appropriately), and then instead of calling the "initPorts()" and "startSource(QString)" functions, you would call them with "longProcess->initPorts()" and "longProcess->startSource(QString)" instead. This lets you basically remove the entire initPorts(), startPorts(), startSource(), parseFetch(), parseUpdate(), updateDone() functions (these are replaced by the PortsnapProc functions).
> 3) Remember that the PortsnapProc template I sent to you was not completely filled out. You will need to finish moving the functions I mentioned above to that new class if necessary (although all the complicated stuff in it was already setup for you, it is basically just the public startPorts, startSource, etc.. functions that you need to move over). Just remember to use the special shortCMD() command for any quick system calls - just like in the one function that I demonstrated for you (that should make everything a lot simpler).
>> On Dec 23, 2013, at 6:50 AM, Ken Moore <ken at pcbsd.org> wrote:
>>> On 12/22/13 00:39, Joe Maloney wrote:
>>>> In order to make this work with pbsystemtab.ui could you give an example?  In particular I am not sure what I should do for this section.
>>>> void PBSystemTab::fetchPortsPressed()
>>>> {
>>>>     portsnapUI = new PortsnapProc();
>>>>     portsnapUI->init(true, Version);
>>>>     portsnapUI->show();
>>>> }
>>>> Joe Maloney
>>>> On Dec 21, 2013, at 6:49 PM, Ken Moore <ken at pcbsd.org> wrote:
>>>>> On 12/21/13 16:02, Joe Maloney wrote:
>>>>>> I’ve made the necessary changes as suggested and now it’s working much better.
>>>>>> http://pastebin.com/ka5U9E1i
>>>>>> The only thing I’m missing is a way to connect these processes to most importantly fetchDone.  If I were somehow able to do this I should be able to have initPorts continue on to startPorts without the need for two buttons, initPorts, startPorts. I know this was working at least before I changed initPorts to also use  QCoreApplication::processEvents() as I did initially for startPorts.
>>>>>> I believe that I could also use  QCoreApplication::processEvents(); to put everything under startPorts() and do away with initPorts.  I’m starting to understand how it works now.  The second button idea would also still be an easy fix.
>>>>>> However the blocker is I still need a way to tell the portsnapprogress GUI that the task is done running. Otherwise it just sits there with a cancel button even when it’s finished no matter how I go about it.
>>>>>> I’m not sure how to implement something like this if it’s even possible without closing the entire application when finished.
>>>>>> portsnap->setProcessChannelMode(QProcess::MergedChannels);
>>>>>> connect(portsnap, SIGNAL(readyReadStandardOutput()), this, SLOT(parseUpdate()));
>>>>>> connect(portsnap, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(fetchDone()));
>>>>>> Joe Maloney
>>>>>> On Dec 19, 2013, at 8:24 AM, Ken Moore <ken at pcbsd.org> wrote:
>>>>>>> On 12/19/13 09:16, Kris Moore wrote:
>>>>>>>> On 12/19/2013 03:12, Joe Maloney wrote:
>>>>>>>>> Here is what I've been able to come up with now for a more simple version.
>>>>>>>>> http://pastebin.com/7PHASL4S
>>>>>>>>> I've been testing and it works but it takes about 25 minutes to fetch all the source.
>>>>>>>>> I had to use something like this to make up for not being able to run to instances of prog, args in startPorts to run two git commands on after another.:
>>>>>>>>> system("git --git-dir=/usr/ports/.git fetch" );
>>>>>>>>> Otherwise they would run at the same time which would be bad.  I'm not sure of a better way to do it at this point but my gut tells me using system() might not be a great idea.
>>>>>>>>> I haven't commited this yet as it's really slow and I figured I would see what your thoughts were first.
>>>>>>>>> Joe Maloney
>>>>>>>> So, to speed it up, I would add --depth=1, which will greatly speed up the initial checkout.
>>>>>>>> # git clone --depth=1 https://github.com/pcbsd/freebsd-ports.git
>>>>>>>> I do that on my builders here and it took the checkout from 20 minutes down to about 45 seconds :)
>>>>>>>> As for using "system" it may be better to use QProcess::execute() instead.
>>>>>>> -snip-
>>>>>>> Since you are already using QProcesses, you can also have it run QCoreApplication::processEvents() in regular intervals while the command is running to prevent the GUI from hanging like this:
>>>>>>> --------------------------
>>>>>>> proc->start("command")
>>>>>>> while(!proc->waitForFinished(500)){ //0.5 sec wait
>>>>>>>   QCoreApplication::processEvents();
>>>>>>> }
>>>>>>> if( proc->exitCode() != 0){ qDebug() << "Error"; }
>>>>>>> else{
>>>>>>>   proc->start("command 2")
>>>>>>> }
>>>>>>> ---------------------------
>>>>>>> and repeat as necessary to run other commands in sequence.
>>>>>>> You might want to look at the QProcess.setWorkingDirectory() function as well, so you might not need to run special flags for running in an alternate directory if you just make the process "change" into the directory you want to run from.
>>>>>>> -- 
>>>>>>> ~~ Ken Moore ~~
>>>>>>> PC-BSD/iXsystems
>>>>>>> _______________________________________________
>>>>>>> Dev mailing list
>>>>>>> Dev at lists.pcbsd.org
>>>>>>> http://lists.pcbsd.org/mailman/listinfo/dev
>>>>> First impression:
>>>>> There is a bit of a disconnect between the InitPorts(), startPorts(), and startSource() functions in that they all use different QProcesses - so only the startSource/portsnap process will actually send out the update and finished signals. I would set them all up to use a single known QProcess (portsnap) for all the long downloads to make use of the signals without a lot of duplication (assuming you only plan on one of these running at a time).
>>>>> My recommendation:
>>>>> It seems like you need to disconnect the process from the GUI a bit more - maybe just subclass QProcess and add a couple special signals for GUI output.
>>>>> For instance, something like the quick header file I am attaching should work quite well for you. You will just need to fill in the rest of the functions (and probably put the special ones I explicitly wrote out into a companion *.cpp file), but it should give you the general idea at least. The basic idea is that you just have a single PortsnapProc in your GUI class that you can just connect the signals to GUI slots that will update the GUI appropriately - while all the "grisly bits" can be separated out of the GUI class this way.
>>>>> -- 
>>>>> ~~ Ken Moore ~~
>>>>> PC-BSD/iXsystems
>>>>> <portsnapProc.h>
>>> Sure:
>>> In your *.h file for the GUI, you just make sure you have this declared:
>>> -----------------
>>> private:
>>> PortsnapProc *longProcess;
>> So this could go in pbsystemtab.h?
> No this will go in portsnapprogress.h (I assume you already have a *.h file with a private section - just add that one line in that section).
>>> ------------------
>>> And in the GUI *.cpp file you set it up like this:
>>> ----------------
>>> <in Initilization code>
>>> longProcess = new PortsnapProc;
>>>  connect(longProcess, SIGNAL(ProcMessage(QString)), this, SLOT(UpdateProcessDisplay(QString)) );
>>>  connect(longProcess, SIGNAL(procFinished(bool)), this, SLOT(ProcessFinished(bool)) );
>>> <To use it to start a process in a later function>
>>> if( longProcess->isRunning() ){
>>> qDebug() << "Ports are already being modified, try again after it finishes";
>>> return;
>>> }
>>> if( longProcess->startInitPorts() ){ qDebug() << "Ports init started successfully"; }
>>> else{ qDebug() << "Ports init could not be started"; }
>>> (same idea to just use startPorts() or startSource() )
>> The above could go in pbsystemtab.cpp under fetchPortsPressed()?  Or would this go in portsnapprogress.cpp?
> portsnapprogress.cpp - see above steps.
>>> The main distinction is that you just need to treat it like another simple variable (like a QString), you don't need to worry about "init"ing it or showing it because it is just a background process worker. Once you have the two signals connected to the GUI slots below, you basically just need to call the appropriate "start" function on it to get it going.
>>> sample GUI connection slots
>>> --------------------------------------
>>> void UpdateProcessDisplay(QString msg){
>>>  ui->lineEdit_processDisplay->setText(msg);
>>> }
>>> void ProcessFinished(bool success){
>>>  if(success){
>>>    QMessageBox::info(this, tr("Success"), tr("Ports Tree successfully modified") );
>>>  }else{
>>>    QMessageBox::warning(this, tr("Failure"), tr("Ports Tree modification failed") );
>>>  }
>>>  //Clean up the display
>>>  ui->lineEdit_processDisplay->setVisible(false); //assuming you make it visible appropraitely
>>> }
>> Would this go in pbsystemtabui.cpp under fetchPortsPressed instead of what I asked about above?
> These would be the two new functions you have to create in portsnapprogress.[h/cpp]. You will just need to replace the "ui->lineEdit_processDisplay" variable with whatever the proper variable name is for the widget is you are using to display the process information in portsnapprogress.
> In fact, portsnapprogress should only need the following functions anymore:
> 1) one function per button on the GUI (so a "cancel" button would have an associated "slotCancel" function if you want to call it that).
> 2) The two functions for detecting PortsnapProc signals I wrote above.
> 3) The main "init" function that is used to create/start the dialog from within pbsystemtabui.cpp
> That should make things extremely simple for this GUI.  :-)
>>> If you want to get more information out of the process regarding failure messages and such, it then becomes just a trivial modification to the signal/slot to add another output variable (like a QStringList with an error log).
>>> I hope this helps!
>>> -- 
>>> ~~ Ken Moore ~~
>>> PC-BSD/iXsystems
>> If any of the above is correct what would I do with portsnapprogress.cpp?  Again thanks for all your help this far.  It was only a few weeks ago I didn’t know a thing about QT. :)
>> Joe Maloney
> No problem: just remember that any function names I used are not "needed" to make it work - you just need to use the names that you want to define and/or use yourself.
> -- 
> ~~ Ken Moore ~~
> PC-BSD/iXsystems

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pcbsd.org/pipermail/dev/attachments/20131227/3aaf3bb1/attachment.html>

More information about the Dev mailing list