[Scons-users] SCons Python Builder not building in parallel?

Hua Yanghao huayanghao at gmail.com
Sat Sep 22 13:16:14 EDT 2018


Dear Scons Users,
I recently noticed that with -jN my CPU utilization is less than 100%
(4 cores) during some phases (e.g. when executing a python builder)
while reaches 400% when compiling C codes. I have a python builder
that generates quite some .h files, which all the C files depend on.
Those .h files has no relationship/dependencies among themselves, and
running with --debug=explain gives below information:

./run configs/qemu_arm_vexpress.py -j8 --debug=explain
Running : ./tools/scons -Q config=configs/qemu_arm_vexpress.py -j8
--debug=explain
USW INFO: usw_config completed in 1.60 seconds.
scons: rebuilding
`build/configs/qemu_arm_vexpress/bios_bin/config/config.h' because
AlwaysBuild() is specified
__CONFIG build/configs/qemu_arm_vexpress/bios_bin/config/config.h
scons: rebuilding
`build/configs/qemu_arm_vexpress/bios_elf/config/config.h' because
AlwaysBuild() is specified
__CONFIG build/configs/qemu_arm_vexpress/bios_elf/config/config.h
scons: rebuilding
`build/configs/qemu_arm_vexpress/core0/config/config.h' because
AlwaysBuild() is specified
__CONFIG build/configs/qemu_arm_vexpress/core0/config/config.h
scons: rebuilding
`build/configs/qemu_arm_vexpress/core1/config/config.h' because
AlwaysBuild() is specified
__CONFIG build/configs/qemu_arm_vexpress/core1/config/config.h
scons: rebuilding
`build/configs/qemu_arm_vexpress/core2/config/config.h' because
AlwaysBuild() is specified
__CONFIG build/configs/qemu_arm_vexpress/core2/config/config.h
scons: rebuilding
`build/configs/qemu_arm_vexpress/core3/config/config.h' because
AlwaysBuild() is specified
__CONFIG build/configs/qemu_arm_vexpress/core3/config/config.h

Any hints why these are not building in parallel? Is it because scons
builds python with multithreads where the GIL eats all the CPU
intensive workloads performance? Or should I re-implement the builder
in such a way that it is not using scons tools but rather running as a
dedicated executable and then wrapped into scons tools?

Thanks,
Yanghao Hua


More information about the Scons-users mailing list