[Scons-users] Parallel MD5

[Eric van Gyzen]

On Feb 24, 2014, at 6:33 PM, Dirk Bächle wrote:


> On 22.02.2014 14:12, Eric van Gyzen wrote:

>> SCons Users:

>> 

>> It seems that SCons uses a single thread to calculate the MD5 sums of the sources, regardless of the -jN flag.  Is this correct?  When working in large projects with thousands of files, and only modifying one file, parallelizing the MD5 calculations across the -jN CPUs could be a major performance improvement.

> 

> I've done some work on analysing and profiling SCons performance in the 

> past (see http://www.scons.org/wiki/WhySconsIsNotSlow ), and so far I 

> didn't find that MD5 computation is a real bottleneck.

> 

> If you've experienced major slowdowns due to the calculation of 

> hashsums, I'd be very interested in your results. This includes not only 

> the actual figures, but also information about the used project setup 

> and the evaluation process.


Dirk,

I am embarrassed to admit that my claim was based on a hunch, with no scientific experimentation of any kind.  It is complete conjecture.  Thank you for your performance analysis.  It is very well done.  Obviously, I need to do a similar analysis with our code base.

Kind regards,

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://four.pairlist.net/pipermail/scons-users/attachments/20140226/240ce26c/attachment.html