[Scons-users] Batching behavior of the Parallel class in Jobs.py

Dirk Bächle tshortik at gmx.de
Wed Oct 5 14:46:22 EDT 2016


Hmm,

On 05.10.2016 20:42, Dirk Bächle wrote:
> Hi Andrew,
>
> On 05.10.2016 19:57, Andrew C. Morrow wrote:
>>
>> I noticed something a little odd, maybe, in the Parallel class:
>>
>> https://bitbucket.org/scons/scons/src/ea74a7f7a61561bebfbbfd5e7e70c4c346dd8bdb/src/engine/SCons/Job.py?at=default&fileviewer=file-view-default#Job.py-403:426
>>
>> The way I read this, Jobs are processed in 'batches', but this doesn't seem to be necessary. To experiment with this, I added prints
>> at lines 396 and 407 to print the number of jobs. The current behavior when I do a -j12 build looks like:
>>
>
> thanks for noticing this. I haven't been around back then, but it appears to be an intended change from revision #1281 "Reap as many
> thread responses as possible at once.  (J.T. Conklin)". It's also reflected in one of the Taskmaster tests, so I'd expect that one
> to fail with your patch.
> There's some more info about the background of this change in the threads
>
>     http://osdir.com/ml/programming.tools.scons.user/2005-05/msg00025.html
>     http://osdir.com/ml/programming.tools.scons.user/2005-05/msg00028.html
>     http://osdir.com/ml/programming.tools.scons.devel/2005-05/msg00011.html
>
> but I haven't digested these yet, such that I could say "I understand what they meant". It's definitely *not* an oversight...
>

I guess this note by Steven Knight

"This patch should help mitigate any performance bottlenecks SCons may
have determining the next task when there are outstanding tasks whose
responses have not yet been processed."

is the key... Does it help?

Dirk



More information about the Scons-users mailing list