[PC-BSD Dev] pcbsd system source idea and question

Ken Moore ken at pcbsd.org
Thu Dec 26 07:52:57 PST 2013

On 12/25/13 14:29, Joe Maloney wrote:
> Merry Christmas.  I’ve had some success after modifying a few syntaxes here and there getting some of this to compile and the GUI to interact with the commands.
> I do have some more questions just to make sure I understand this right.  For example the portsnapproc.h file you sent could essentially just replace whats in portsnapprogress.h?  Please see my comments below your replies for more questions.

Almost: the portsnapProc class would simply replace all the QProcess and 
backend stuff within portsnapprogress. What would still need to be in 
portsnapprogress is all the GUI stuff (since I am assuming that 
portsnapProgress is some kind of pop-up dialog that reports the current 
process output).

Basically you would just need to do this for portsnapprogress.

1) Still have pbsystemtab call your portSnapProgress class (since that 
is basically opening up the new dialog) just the way you already had it 
set up.

2) In the portsnapprogress::init(bool, QString) function you will need 
to add the PortsnapProc initialization code I wrote before (changing the 
GUI variable names appropriately), and then instead of calling the 
"initPorts()" and "startSource(QString)" functions, you would call them 
with "longProcess->initPorts()" and "longProcess->startSource(QString)" 
instead. This lets you basically remove the entire initPorts(), 
startPorts(), startSource(), parseFetch(), parseUpdate(), updateDone() 
functions (these are replaced by the PortsnapProc functions).

3) Remember that the PortsnapProc template I sent to you was not 
completely filled out. You will need to finish moving the functions I 
mentioned above to that new class if necessary (although all the 
complicated stuff in it was already setup for you, it is basically just 
the public startPorts, startSource, etc.. functions that you need to 
move over). Just remember to use the special shortCMD() command for any 
quick system calls - just like in the one function that I demonstrated 
for you (that should make everything a lot simpler).

> On Dec 23, 2013, at 6:50 AM, Ken Moore <ken at pcbsd.org> wrote:
>> On 12/22/13 00:39, Joe Maloney wrote:
>>> In order to make this work with pbsystemtab.ui could you give an example?  In particular I am not sure what I should do for this section.
>>> void PBSystemTab::fetchPortsPressed()
>>> {
>>>      portsnapUI = new PortsnapProc();
>>>      portsnapUI->init(true, Version);
>>>      portsnapUI->show();
>>> }
>>> Joe Maloney
>>> On Dec 21, 2013, at 6:49 PM, Ken Moore <ken at pcbsd.org> wrote:
>>>> On 12/21/13 16:02, Joe Maloney wrote:
>>>>> I’ve made the necessary changes as suggested and now it’s working much better.
>>>>> http://pastebin.com/ka5U9E1i
>>>>> The only thing I’m missing is a way to connect these processes to most importantly fetchDone.  If I were somehow able to do this I should be able to have initPorts continue on to startPorts without the need for two buttons, initPorts, startPorts.  I know this was working at least before I changed initPorts to also use  QCoreApplication::processEvents() as I did initially for startPorts.
>>>>> I believe that I could also use  QCoreApplication::processEvents(); to put everything under startPorts() and do away with initPorts.  I’m starting to understand how it works now.  The second button idea would also still be an easy fix.
>>>>> However the blocker is I still need a way to tell the portsnapprogress GUI that the task is done running. Otherwise it just sits there with a cancel button even when it’s finished no matter how I go about it.
>>>>> I’m not sure how to implement something like this if it’s even possible without closing the entire application when finished.
>>>>> portsnap->setProcessChannelMode(QProcess::MergedChannels);
>>>>> connect(portsnap, SIGNAL(readyReadStandardOutput()), this, SLOT(parseUpdate()));
>>>>> connect(portsnap, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(fetchDone()));
>>>>> Joe Maloney
>>>>> On Dec 19, 2013, at 8:24 AM, Ken Moore <ken at pcbsd.org> wrote:
>>>>>> On 12/19/13 09:16, Kris Moore wrote:
>>>>>>> On 12/19/2013 03:12, Joe Maloney wrote:
>>>>>>>> Here is what I've been able to come up with now for a more simple version.
>>>>>>>> http://pastebin.com/7PHASL4S
>>>>>>>> I've been testing and it works but it takes about 25 minutes to fetch all the source.
>>>>>>>> I had to use something like this to make up for not being able to run to instances of prog, args in startPorts to run two git commands on after another.:
>>>>>>>> system("git --git-dir=/usr/ports/.git fetch" );
>>>>>>>> Otherwise they would run at the same time which would be bad.  I'm not sure of a better way to do it at this point but my gut tells me using system() might not be a great idea.
>>>>>>>> I haven't commited this yet as it's really slow and I figured I would see what your thoughts were first.
>>>>>>>> Joe Maloney
>>>>>>> So, to speed it up, I would add --depth=1, which will greatly speed up the initial checkout.
>>>>>>> # git clone --depth=1 https://github.com/pcbsd/freebsd-ports.git
>>>>>>> I do that on my builders here and it took the checkout from 20 minutes down to about 45 seconds :)
>>>>>>> As for using "system" it may be better to use QProcess::execute() instead.
>>>>>> -snip-
>>>>>> Since you are already using QProcesses, you can also have it run QCoreApplication::processEvents() in regular intervals while the command is running to prevent the GUI from hanging like this:
>>>>>> --------------------------
>>>>>> proc->start("command")
>>>>>> while(!proc->waitForFinished(500)){ //0.5 sec wait
>>>>>>    QCoreApplication::processEvents();
>>>>>> }
>>>>>> if( proc->exitCode() != 0){ qDebug() << "Error"; }
>>>>>> else{
>>>>>>    proc->start("command 2")
>>>>>> }
>>>>>> ---------------------------
>>>>>> and repeat as necessary to run other commands in sequence.
>>>>>> You might want to look at the QProcess.setWorkingDirectory() function as well, so you might not need to run special flags for running in an alternate directory if you just make the process "change" into the directory you want to run from.
>>>>>> -- 
>>>>>> ~~ Ken Moore ~~
>>>>>> PC-BSD/iXsystems
>>>>>> _______________________________________________
>>>>>> Dev mailing list
>>>>>> Dev at lists.pcbsd.org
>>>>>> http://lists.pcbsd.org/mailman/listinfo/dev
>>>> First impression:
>>>> There is a bit of a disconnect between the InitPorts(), startPorts(), and startSource() functions in that they all use different QProcesses - so only the startSource/portsnap process will actually send out the update and finished signals. I would set them all up to use a single known QProcess (portsnap) for all the long downloads to make use of the signals without a lot of duplication (assuming you only plan on one of these running at a time).
>>>> My recommendation:
>>>> It seems like you need to disconnect the process from the GUI a bit more - maybe just subclass QProcess and add a couple special signals for GUI output.
>>>> For instance, something like the quick header file I am attaching should work quite well for you. You will just need to fill in the rest of the functions (and probably put the special ones I explicitly wrote out into a companion *.cpp file), but it should give you the general idea at least. The basic idea is that you just have a single PortsnapProc in your GUI class that you can just connect the signals to GUI slots that will update the GUI appropriately - while all the "grisly bits" can be separated out of the GUI class this way.
>>>> -- 
>>>> ~~ Ken Moore ~~
>>>> PC-BSD/iXsystems
>>>> <portsnapProc.h>
>> Sure:
>> In your *.h file for the GUI, you just make sure you have this declared:
>> -----------------
>> private:
>> PortsnapProc *longProcess;
> So this could go in pbsystemtab.h?

No this will go in portsnapprogress.h (I assume you already have a *.h 
file with a private section - just add that one line in that section).
>> ------------------
>> And in the GUI *.cpp file you set it up like this:
>> ----------------
>> <in Initilization code>
>> longProcess = new PortsnapProc;
>>   connect(longProcess, SIGNAL(ProcMessage(QString)), this, SLOT(UpdateProcessDisplay(QString)) );
>>   connect(longProcess, SIGNAL(procFinished(bool)), this, SLOT(ProcessFinished(bool)) );
>> <To use it to start a process in a later function>
>> if( longProcess->isRunning() ){
>> qDebug() << "Ports are already being modified, try again after it finishes";
>> return;
>> }
>> if( longProcess->startInitPorts() ){ qDebug() << "Ports init started successfully"; }
>> else{ qDebug() << "Ports init could not be started"; }
>> (same idea to just use startPorts() or startSource() )
> The above could go in pbsystemtab.cpp under fetchPortsPressed()?  Or would this go in portsnapprogress.cpp?
portsnapprogress.cpp - see above steps.
>> The main distinction is that you just need to treat it like another simple variable (like a QString), you don't need to worry about "init"ing it or showing it because it is just a background process worker. Once you have the two signals connected to the GUI slots below, you basically just need to call the appropriate "start" function on it to get it going.
>> sample GUI connection slots
>> --------------------------------------
>> void UpdateProcessDisplay(QString msg){
>>   ui->lineEdit_processDisplay->setText(msg);
>> }
>> void ProcessFinished(bool success){
>>   if(success){
>>     QMessageBox::info(this, tr("Success"), tr("Ports Tree successfully modified") );
>>   }else{
>>     QMessageBox::warning(this, tr("Failure"), tr("Ports Tree modification failed") );
>>   }
>>   //Clean up the display
>>   ui->lineEdit_processDisplay->setVisible(false); //assuming you make it visible appropraitely
>> }
> Would this go in pbsystemtabui.cpp under fetchPortsPressed instead of what I asked about above?

These would be the two new functions you have to create in 
portsnapprogress.[h/cpp]. You will just need to replace the 
"ui->lineEdit_processDisplay" variable with whatever the proper variable 
name is for the widget is you are using to display the process 
information in portsnapprogress.

In fact, portsnapprogress should only need the following functions anymore:
1) one function per button on the GUI (so a "cancel" button would have 
an associated "slotCancel" function if you want to call it that).
2) The two functions for detecting PortsnapProc signals I wrote above.
3) The main "init" function that is used to create/start the dialog from 
within pbsystemtabui.cpp

That should make things extremely simple for this GUI.  :-)

>> If you want to get more information out of the process regarding failure messages and such, it then becomes just a trivial modification to the signal/slot to add another output variable (like a QStringList with an error log).
>> I hope this helps!
>> -- 
>> ~~ Ken Moore ~~
>> PC-BSD/iXsystems
> If any of the above is correct what would I do with portsnapprogress.cpp?  Again thanks for all your help this far.  It was only a few weeks ago I didn’t know a thing about QT. :)
> Joe Maloney

No problem: just remember that any function names I used are not 
"needed" to make it work - you just need to use the names that you want 
to define and/or use yourself.

~~ Ken Moore ~~

More information about the Dev mailing list