I hit a pair of head-scratchers while cross-compiling SystemTap on Ubuntu 18.04 for aarch64:
During ./configure
configure: error: missing elfutils development headers/libraries
During make
(link stage)
/usr/bin/ld: libstrfloctime.a(...): Relocations in generic ELF (EM: 183)
... error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
Below is exactly how I decoded the problem, reproduced it with a tiny “broken on purpose” project, and then fixed it. I’ll also share some practice helpers I now keep around to avoid this class of issues.
TL;DR What Was Actually Wrong
- Mixed compilers.
configure
found a cross C compiler (aarch64-linux-gnu-gcc
) but not a cross C++ compiler. It fell back to nativeg++
. Later, the x86_64 linker tried to link aarch64 objects → “wrong format / EM: 183”. - pkg-config & headers. My
pkg-config
/CPPFLAGS
/LDFLAGS
were pointing at host elfutils rather than target (aarch64) elfutils, soconfigure
insisted elfutils was missing. - Over-eager
-static
. Full static cross-linking only works if every dependency has a target static.a
. I made life harder than necessary.
The Early Red Flag in My configure
Log
The log literally said:
checking whether we are cross compiling... no
...
checking for aarch64-linux-gnu-g++... no
checking for g++... g++
If I’m cross-compiling, that first line should be “yes.” The test binaries ran on my host, which means they were built for x86_64, not aarch64. Also, with no aarch64-linux-gnu-g++
, C++ parts silently used native g++
.
Later at link time:
Relocations in generic ELF (EM: 183)
EM: 183
means AArch64. My native linker (/usr/bin/ld
, invoked by native g++
) tried to link AArch64 objects. Boom.
A Tiny Project That Reproduces the Error
I like to catch mistakes with a minimal repro. Here’s a one-file C++ program and a bad Makefile that uses a cross C compiler but a native C++ compiler exactly the pitfall that bit me.
First (Broken) Code: hello.cpp
#include <iostream>
int main() {
std::cout << "hello from aarch64 C++\n";
return 0;
}
Bad Makefile
# Makefile (broken on purpose)
HOST ?= aarch64-linux-gnu
CC := $(HOST)-gcc
CXX := g++ # <-- WRONG: falls back to native C++ linker
LD := ld
CXXFLAGS := -O2
LDFLAGS :=
all: hello
hello: hello.cpp
$(CXX) $(CXXFLAGS) hello.cpp -o $@
clean:
rm -f hello
What Happen
CXX=g++
invokes the host linker.- The file is C++ only, so you might not see the arch mismatch immediately. To force the mismatch, compile an AArch64 object first and then try to link it with native
g++
:
# Add this (even more wrong) rule to force arch mismatch
hello.o: hello.cpp
$(HOST)-g++ -c hello.cpp -o hello.o # aarch64 object
hello: hello.o
$(CXX) hello.o -o $@ # native g++ tries to link aarch64
Expected Error:
/usr/bin/ld: hello.o: Relocations in generic ELF (EM: 183)
hello.o: error adding symbols: File in wrong format
collect2: error: ld returned 1 exit status
The Error Define & Explain
- “Relocations in generic ELF (EM: 183)” = the object file is for Machine 183 (AArch64).
- Native
g++
(x86_64) invokes the x86_64 linker, which cannot link AArch64 objects → “File in wrong format.”
Fixing The Tiny Project
Correct Makefile:
# Makefile (fixed)
HOST ?= aarch64-linux-gnu
CC := $(HOST)-gcc
CXX := $(HOST)-g++
AR := $(HOST)-ar
LD := $(HOST)-ld
CXXFLAGS := -O2
LDFLAGS :=
all: hello
hello: hello.cpp
$(CXX) $(CXXFLAGS) hello.cpp -o $@
clean:
rm -f hello
Now:
make
file hello
# ELF 64-bit LSB executable, ARM aarch64 ...
Turning Back to SystemTap the Clean fix I Used
Install the cross toolchain (both C and C++)
sudo apt-get update
sudo apt-get install -y \
gcc-aarch64-linux-gnu g++-aarch64-linux-gnu \
binutils-aarch64-linux-gnu pkg-config file
Without aarch64-linux-gnu-g++
, configure
falls back to native g++
and you’ll see EM:183 again.
Export all Cross Tools Explicitly
export HOST=aarch64-linux-gnu
export BUILD=$(gcc -dumpmachine) # likely x86_64-linux-gnu
export SYSROOT=/usr/$HOST # or your SDK sysroot
export CC=$HOST-gcc
export CXX=$HOST-g++
export LD=$HOST-ld
export AR=$HOST-ar
export RANLIB=$HOST-ranlib
export STRIP=$HOST-strip
export PKG_CONFIG=$HOST-pkg-config # if available
No $HOST-pkg-config
? Use regular pkg-config
, but force it toward target .pc
files:
export PKG_CONFIG=pkg-config
export PKG_CONFIG_SYSROOT_DIR=$SYSROOT
# your cross-built install prefixes:
export PKG_CONFIG_LIBDIR=/home/zhongyi/tools/elfutils-0.188/_install/lib/pkgconfig:\
/home/zhongyi/tools/zlib-1.2.13/_install/lib/pkgconfig
Point to the Cross Built Dependencies
export Z=/home/zhongyi/tools/zlib-1.2.13/_install
export EU=/home/zhongyi/tools/elfutils-0.188/_install
export CPPFLAGS="-I$EU/include -I$Z/include"
export LDFLAGS="-L$EU/lib -L$Z/lib"
export LIBS="-ldw -lelf -lz"
I drop -static
initially; I add it back only after the dynamic build works.
Configure SystemTap Clearly
./configure \
--build="$BUILD" \
--host="$HOST" \
--prefix="$PWD/_install" \
--with-elfutils="$EU" \
--without-rpm \
--without-nss \
--disable-server \
--disable-java \
--disable-docs
--build
= my machine triple (x86_64-linux-gnu).--host
= the target (aarch64-linux-gnu).- I disable extras that drag in more deps on older distros.
Build & Install
make -j"$(nproc)"
make install
I then copy _install/
into the target rootfs (or onto the device).
Reminder: the SystemTap compiler (stap
) runs on the host, and the runtime (staprun
) runs on the target.
Understanding the Elfutils configure
Failure
configure
printed:
checking for ebl_strtabinit in -lebl... no
checking for dwfl_module_getsym in -ldw... no
configure: error: missing elfutils development headers/libraries
In my case this meant:
CPPFLAGS/LDFLAGS
pointed at host headers/libs, not target ones.- Or
pkg-config
was pulling host.pc
files.
How I verified/fixed it:
pkg-config --modversion libdw
pkg-config --cflags --libs libdw
These must resolve under my target prefix ($EU
), not /usr/lib/x86_64-linux-gnu
.
If not, I fix PKG_CONFIG_SYSROOT_DIR
and PKG_CONFIG_LIBDIR
.
More Practice Functionality Sanity Checks I Now Run First
Toolchain Guard Fail Fast if Cross C++ is Missing
cat > toolchain-check.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
HOST="${HOST:-aarch64-linux-gnu}"
need() { command -v "$1" >/dev/null || { echo "Missing $1" >&2; exit 1; }; }
need "$HOST-gcc"
need "$HOST-g++"
need "$HOST-ar"
need "$HOST-ld"
echo "[ok] cross C and C++ toolchain found for $HOST"
EOF
chmod +x toolchain-check.sh
pkg-config
Guard Ensure I’m Not Linking Host Libs
cat > pkgconfig-check.sh <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
for pc in libdw libelf zlib; do
if pkg-config --exists "$pc"; then
echo "[$pc] cflags: $(pkg-config --cflags "$pc")"
echo "[$pc] libs : $(pkg-config --libs "$pc")"
else
echo "[$pc] MISSING"
fi
done
EOF
chmod +x pkgconfig-check.sh
Minimal “first code” to Prove Both C and C++ are aarch64
hello.c
#include <stdio.h>
int main(void) { puts("hello from C (aarch64)"); return 0; }
hello.cpp
#include <iostream>
int main() { std::cout << "hello from C++ (aarch64)\n"; return 0; }
Build & verify:
$HOST-gcc -o hello_c hello.c
$HOST-g++ -o hello_cpp hello.cpp
file hello_c hello_cpp
# Expect: ELF 64-bit LSB executable, ARM aarch64 ...
readelf -h hello_cpp | grep 'Machine'
# Expect: Machine: AArch64
If file
shows x86_64 for either binary, I know I’ve accidentally used the native toolchain somewhere.
Optional Tightening
libs:
export CFLAGS="-O2 -static"
export CXXFLAGS="-O2 -static"
If it breaks, you’re missing a target .a
(libdw/libelf/libz/libstdc++/libgcc…).
Strip final binaries:
$HOST-strip _install/bin/staprun
For SDK/sysroot builds:
export SYSROOT=/path/to/sdk/sysroot
export CC="$HOST-gcc --sysroot=$SYSROOT"
export CXX="$HOST-g++ --sysroot=$SYSROOT"
export PKG_CONFIG_SYSROOT_DIR="$SYSROOT"
export PKG_CONFIG_LIBDIR="$SYSROOT/usr/lib/pkgconfig:$SYSROOT/usr/share/pkgconfig"
export CPPFLAGS="-I$SYSROOT/usr/include $CPPFLAGS"
export LDFLAGS="-L$SYSROOT/usr/lib $LDFLAGS"
Final Thought
I used to treat cross-compiling failures as mysterious linker voodoo. They aren’t. Every time I see EM: 183 now, I immediately ask: “Did I mix compilers or leak host libs?” A 60-second check of CXX
, LD
, and PKG_CONFIG_*
has saved me hours of digging. Start with the tiny repro project above, wire your environment cleanly, and SystemTap (and friends) will fall into place.