Category: Gentoo

Categories
Embedded Systems Engineering Gentoo Linux

Don’t build standard libraries with LTO unless you really want to suffer

I wanted to make this post more as a reminder to myself.

A few years ago I ran into an interesting issue while working with a cross-compiler for ARM on my Gentoo Linux systems. Effectively, I was getting a ton of libc undefined symbols errors. After a thread on the gcchelp mailing list, this was the summary of the outcome:

Gabriel Marcano:
Can newlib not be built with -flto?

...

Alexander Monakov:
It matters because the compiler recognizes several libc functions by name (including 'malloc'); recognizing such so-called "builtins" is necessary to optimize their uses. Unfortunately, builtins are different from common functions in LTO bytecode, and sometimes behave in unexpected ways. You can find examples searching GCC Bugzilla for stuff like "LTO built-in".

In your case the reference to built-in malloc is not visible to the linker when it invokes the plugin while scanning input files, so it does not unpack malloc.o from libc.a. This is a simple case of a more general problem where a compiler creates new references to built-in functions after optimizations (e.g. transforming 'printf("Hello\n")' to 'puts("Hello")'). Somehow it happens to work when libc is not slim-LTO (I guess it is rescanned), but rescanning for LTO code-generation is not implemented, as far as I know.

So no, apparently the tricks that GCC/LD make to optimize strlen, malloc, etc. make it next to impossible to properly enable LTO for the standard library.

For Gentoo this means remembering to disable LTO for newlib builds for cross-compilers. This is done by making a /etc/portage/env/no-lto.conf file:

CFLAGS="${CFLAGS} -fno-lto"
CXXFLAGS="${CXXFLAGS} -fno-lto"
FFLAGS="${FFLAGS} -fno-lto"

And then somewhere under /etc/portage/package.env/ making a file with:

cross-*/newlib no-lto.conf
Categories
Gentoo Linux

Gentoo’s sets.conf DateSet

I wanted to write this blog entry because this isn’t the first time I’ve rediscovered this bit of Gentoo functionality, and I suspect it will not be the last.

Recently my GCC upgrade from sys-devel/gcc:12 to sys-devel/gcc:13, and because I build most of my system with LTO, “bad things” started happening (mostly when building newer packages, LTO bytecode mismatch). I found myself wanting to rebuild all of the packages built prior to gcc:13 being installed… and remembered that at some point I had figured out this exact problem on an older system of mine through the use of Gentoo portage sets.

Gentoo’s portage’s set support is quite extensive, including a bunch of set classes that can control how packages are selected (like AgeSet and CategorySet). My goal was to make a set, like @older_gcc, that contained all of the packages built before the current gcc. I briefly considered using AgeSet, which lets you specify a number of days ago, and then you specify whether to select packages newer than that day, or older. Something like:

[older_gcc]
class = portage.sets.dbapi.AgetSet
age = 1
mode = older

But, that configuration doesn’t cover any packages that might have installed today before gcc:13. Also, I’ll probably forget to run this next time gcc does a major update, and I’ll have to update the age parameter.

It was here than I remembered doing something else many years ago, but I could not find anything relevant in the portage documentation. After digging through the portage source code, I found DateSet. This class allows one to specify a package as a point of reference! This is exactly what I wanted. As such, I’ve now added the following to my /etc/portage/sets.conf :

[older_gcc]
class = portage.sets.dbapi.DateSet
package = sys-devel/gcc

As far as I can tell from checking with emerge -pv @older_gcc, everything looks to be selected properly.

Hopefully this is the last time I rediscover this bit of functionality…