[Scons-users] InstallVersionedLibrary sometimes fails when symlinking

[Andrew C. Morrow]

Hi Rob -

I found some time to experiment with this again. I've posted my
example project to:
https://github.com/acmorrow/scons-versioned-lib-demo

There are a few issues:

1) I don't see that the issue with rebuilding the symlink in the
install or build directory if it is manually removed is fixed.
2) The change to _INSTALL_TARGETS.extend needs, I think, to place its
arguments in brackets (see README and or NOTE 2)
3) Because InstallVersionedLib doesn't return all of the artifacts (I
think) there is no way to apply different tags to each artifact (e.g.
the dev link goes in a -dev package, and the hard file and soname
symlink go in the main package). See (README or NOTEs 1 and 3).

Also, there appears to be an issue with how Package works with targz,
since it always encodes the variant directory and packaging directory
into the tarfiles, instead of capturing only the $PREFIX path. FWIW, I
also think it would be useful to instruct Package to not include any
of the $PREFIX, for cases where the project is relocatable in the
filesystem due to all targets having proper $ORIGIN settings in RPATH,
etc. Just a thought. Perhaps there is already a way to do this.

Thanks,
Andrew



On Tue, Jun 18, 2013 at 11:51 AM, Managan, Rob <managan1 at llnl.gov> wrote:

> I just got a test added and made a pull request. Your email had me look at

> this again and I just updated the fork to include a potential fix.

>

> Please try it out and is there a way to make a simple test case?

>

> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

> Rob Managan                             email managan at llnl.gov

> LLNL                                             phone: 925-423-0903

> P.O. Box 808, L-095                   FAX:   925-422-3389

> Livermore, CA  94551-0808

>

>

>

>

>

> On 6/18/13 5:34 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com> wrote:

>

>>Hi Rob -

>>

>>Any update on the dependency tracking issues with InstallVersionedLibrary?

>>

>>Thanks,

>>Andrew

>>

>>

>>On Mon, Jun 3, 2013 at 11:41 AM, Managan, Rob <managan1 at llnl.gov> wrote:

>>> Hi Andrew,

>>>

>>> Good catch. They probably are not cleaned as well. Let me see what is

>>> involved to fix that.

>>>

>>> Rob

>>>

>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

>>> Rob Managan                             email managan at llnl.gov

>>> LLNL                                             phone: 925-423-0903

>>> P.O. Box 808, L-095                   FAX:   925-422-3389

>>> Livermore, CA  94551-0808

>>>

>>>

>>> On 6/2/13 10:13 AM, "Andrew C. Morrow" <andrew.c.morrow at gmail.com>

>>>wrote:

>>>

>>>

>>> This is most likely related to the above, but it also appears that the

>>> installed symlinks are not captured by FindInstalledFiles, which means

>>>that

>>> Package (if I'm reading it correctly) won't package the symlinks.

>>>

>>> Thanks,

>>> Andrew

>>>

>>>

>>>

>>> On Sat, Jun 1, 2013 at 2:09 PM, Andrew C. Morrow

>>><andrew.c.morrow at gmail.com>

>>> wrote:

>>>>

>>>>

>>>> Hi Rob -

>>>>

>>>> Sorry it took me a couple days to get back to this. I just tried out

>>>>your

>>>> branch. The good news is that the issue with the build failing due to

>>>>errors

>>>> while symlinking is fixed, so that is a big help.

>>>>

>>>> While experimenting with the fix though, I've noticed another issue.

>>>>Using

>>>> the same test case as before:

>>>>

>>>> $ python $MYSCONS/script/scons.py --prefix=$(pwd)/install install

>>>> SCons import failed. Trying to run from source directory

>>>> scons: Reading SConscript files ...

>>>> scons: done reading SConscript files.

>>>> scons: Building targets ...

>>>> Install file: "build/foo.hpp" as "install/include/foo.hpp"

>>>> g++ -o build/foo.os -c -fPIC build/foo.cpp

>>>> g++ -o build/libfoo.0.1.2.dylib -dynamiclib -current_version 0.1.2

>>>> -compatibility_version 0.1.2 -undefined dynamic_lookup build/foo.os

>>>> Install file: "build/libfoo.0.1.2.dylib" as

>>>> "install/lib/libfoo.0.1.2.dylib"

>>>> scons: done building targets.

>>>>

>>>> Everything builds OK. Lets remove the symlink in the install directory:

>>>>

>>>> $ rm install/lib/libfoo.dylib

>>>>

>>>> Rerunning the build claims install is up to date:

>>>>

>>>> $ python $MYSCONS/script/scons.py --prefix=$(pwd)/install install

>>>> SCons import failed. Trying to run from source directory

>>>> scons: Reading SConscript files ...

>>>> scons: done reading SConscript files.

>>>> scons: Building targets ...

>>>> scons: `install' is up to date.

>>>> scons: done building targets.

>>>>

>>>> And the file was not re-created:

>>>>

>>>> $  ls install/lib/libfoo.dylib

>>>> ls: install/lib/libfoo.dylib: No such file or directory

>>>>

>>>> The same effect is observed removing libfoo.dylib from the build

>>>> directory. So it appears that there are some dependency edges that are

>>>>not

>>>> being captured correctly.

>>>>

>>>> Thanks,

>>>> Andrew

>>>>

>>>>

>>>>

>>>> On Thu, May 30, 2013 at 2:08 PM, Managan, Rob <managan1 at llnl.gov>

>>>>wrote:

>>>>>

>>>>> Andrew,

>>>>>

>>>>> I have a proposed fix at

>>>>>https://bitbucket.org/managan/scons_versionedlib

>>>>> .

>>>>> Let me know if it works for you.

>>>>> I added code to delete any existing symlink before creating them

>>>>>again.

>>>>> That was how it failed for me even if the error message seemed to say

>>>>> something else. I also made sure the symlinks in the install

>>>>>directory were

>>>>> set up the same as in the build directory. Not sure how that ever was

>>>>> different.

>>>>>

>>>>> I need to add a test case before generating a pull request but this

>>>>> should be good to go.

>>>>>

>>>>> Rob

>>>>>

>>>>> *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

>>>>> Rob Managan                             email managan at llnl.gov

>>>>> LLNL                                             phone: 925-423-0903

>>>>> P.O. Box 808, L-095                   FAX:   925-422-3389

>>>>> Livermore, CA  94551-0808

>>>>>

>>>>>

>>>>> On 5/27/13 10:46 PM, "William Deegan" <bill at baddogconsulting.com>

>>>>>wrote:

>>>>>

>>>>> Andrew,

>>>>>

>>>>> On May 27, 2013, at 10:04 PM, Andrew C. Morrow

>>>>> <andrew.c.morrow at gmail.com> wrote:

>>>>>

>>>>>

>>>>> On Mon, May 27, 2013 at 11:03 PM, William Deegan

>>>>> <bill at baddogconsulting.com> wrote:

>>>>>>

>>>>>> Andrew,

>>>>>>

>>>>>> Were you able to repro this? If so, should I file a ticket?

>>>>>>

>>>>>>

>>>>>> Unfortunately, very busy IRL at the moment so haven't had a chance to

>>>>>> try it out.

>>>>>> Lots of work leading up to a conference, once that's over (6/6) I'll

>>>>>> have some time to take a look at this.

>>>>>

>>>>>

>>>>> Sounds good. I just wanted to make sure it didn't get missed. Built in

>>>>> support for versioned shared libraries is a really useful feature and

>>>>>I'd

>>>>> very much like to use it. I'm sure I'm not the only one.

>>>>>

>>>>>>

>>>>>>

>>>>>>

>>>>>> Also, is there some documentation on the SCons bugfix and release

>>>>>>cycle?

>>>>>> I've noticed that there are only infrequently 'z +1' releases of

>>>>>>SCons

>>>>>> x.y.z. If the problem with InstallVersionedLibrary is legitimate,

>>>>>>should I

>>>>>> expect to see a fix in a to-be-released-somewhat-soonish SCons 2.3.1

>>>>>>bugfix

>>>>>> release, or longer term in some future SCons 2.4 release?

>>>>>>

>>>>>>

>>>>>>

>>>>>> SCons is a volunteer run open source project.

>>>>>> If you want to ensure that a fix is made, you'll either have to

>>>>>>submit a

>>>>>> fix, hope that someone in the community picks up the issue and fixes

>>>>>>it, or

>>>>>> finance a consultant to resolve the issue for you.

>>>>>

>>>>>

>>>>> I'm aware of the nature of the project. Still, this is a feature

>>>>> documented in the SCons manual pages. Someone loved this feature

>>>>>enough to

>>>>> shepherd it through development, code review, testing, documentation,

>>>>>and

>>>>> release.

>>>>>

>>>>>

>>>>> Though all of this is true, that doesn't guarantee that the developer

>>>>>of

>>>>> any feature is immediately available to address a bug.

>>>>>

>>>>>

>>>>>>

>>>>>> In general we will release a new version either when something

>>>>>>notable

>>>>>> has been fixed or if there's been a sufficient number of small fixes.

>>>>>> If a fix is submitted for your issue (assuming it's not user error

>>>>>> (which from a cursory glance it doesn't look like it is)), then it

>>>>>>would be

>>>>>> in the next release.

>>>>>

>>>>>

>>>>> Right, but "next release" at which revision level, and on (roughly)

>>>>>what

>>>>> time scale? Without some insight into the release process, intended

>>>>> schedule, defect triage process, backport policy, etc., it is

>>>>>difficult to

>>>>> reason about whether I should abandon my attempts to use this new

>>>>>feature or

>>>>> not. Even if I were to debug this particular issue and submit a patch

>>>>>as you

>>>>> have suggested, the adoption of that fix into a published release may

>>>>>not

>>>>> mesh with my own release schedule. I'm not complaining - I merely

>>>>>want to

>>>>> better understand: If I or someone else submitted a patch tonight,

>>>>>would it

>>>>> likely land in a 2.3.1 release in one month, or in a 2.4.0 in 6

>>>>>months, or

>>>>> something else entirely?

>>>>>

>>>>>

>>>>> If this is a bug in an existing feature (and assuming the fix doesn't

>>>>> require breaking compatability), then the fix would show in the next

>>>>>minor

>>>>> release published after a fix is accepted.

>>>>> This likely means 2.3.1, however if other less trivial (assuming the

>>>>>fix

>>>>> is fairly trivial) changes have already landed in the main branch,

>>>>>then it

>>>>> might appear in the 2.4.0 branch.

>>>>> There is not direct correlation between a 2.3.1 release or a 2.4.0

>>>>> release and a timeline.

>>>>>

>>>>>

>>>>>

>>>>>>

>>>>>> Also keep in mind all the work is done in a public mercurial repo

>>>>>>hosted

>>>>>> on bitbucket, so as soon as anyone published a fix you can try it

>>>>>>against

>>>>>> your build, and use that code if it resolves your issue.

>>>>>

>>>>>

>>>>> That works well for projects that I plan to build for myself. However,

>>>>> for open source projects which use SCons as the publicly facing build

>>>>>system

>>>>> it is not a realistic solution to require downstream consumers to use

>>>>>the

>>>>> SCons mainline or manually patch their SCons installs. Such projects

>>>>>are

>>>>> constrained to use the demonstrably working subset of the official

>>>>>SCons

>>>>> releases that make their way into distribution and language package

>>>>> repositories. Potentially, a project could drag in the entire slug of

>>>>>fixed

>>>>> code and redefine the InstallVersionedLibrary, but this has different

>>>>> problems.

>>>>>

>>>>>

>>>>> Fair enough.

>>>>> Note however that SCons can be run in place with a scons-local

>>>>>package,

>>>>> so you could unpackage a build scons-local package from the latest

>>>>>code and

>>>>> check that into your project until such time as the needed fix is

>>>>>released.

>>>>>

>>>>> Hope that helps.

>>>>>

>>>>> In the meantime, it looks like Rob is hot on the trail of this

>>>>>issue.. so

>>>>> stay tuned.

>>>>>

>>>>> -Bill

>>>>>

>>>>> _______________________________________________

>>>>> Scons-users mailing list

>>>>> Scons-users at scons.org

>>>>> http://four.pairlist.net/mailman/listinfo/scons-users

>>>>>

>>>>

>>>

>>>

>>> _______________________________________________

>>> Scons-users mailing list

>>> Scons-users at scons.org

>>> http://four.pairlist.net/mailman/listinfo/scons-users

>>>

>>_______________________________________________

>>Scons-users mailing list

>>Scons-users at scons.org

>>http://four.pairlist.net/mailman/listinfo/scons-users

>

> _______________________________________________

> Scons-users mailing list

> Scons-users at scons.org

> http://four.pairlist.net/mailman/listinfo/scons-users