Linking errors: accidental compiler update

Linking a compiled program generally consists of three parts: a function call, a function declaration, and a function definition as shown in the example below.

int my_function(double); // definition
int main()
{
    // ...
    int result = my_function(2.3); // function call
    // ...
}
int my_function(double d) // declaration
{
    return d * 2.0;
}

If these were one program this would just be compiled. However, definition is generally in a header file for a library; my_function in the compiled library (.so) file; and our program would use it. In normal C; the functions would be loaded using dlopen(library_name, mode)¹ and dlsym(library, function_name)²; which would provide a basic function pointer.

In C++ however, hundred of functions can share a name in different classes and namespaces; and this means that a function name is ambiguous. To solve this, C++ compilers decorate the function names with information about class, namespace, return type and arguments. my_function might be transformed into _Z1my_functiond (in GCC), which means global symbol that accepts a double argument. Had it been a class, there would be another part after _Z1 related to the class. This string becomes the function_name given to dlsym.

The problem I ran into was that I switched my compiler from GCC version 4.x, to GCC 5.3 and updating some of my libraries. GCC 5.3 uses C++11 as a standard instead of C++0x; and C++11 has a different std::string class. This is known as an ABI switch³, and I had even read about this one before but forgot about it again. When the class changed, it means that all functions/classes that use std::string suddenly had one mangled name in my program (which was compiled using C++0x) and one mangled name in the compiler and new libraries (which were compiled using C++11).

Normally, this type of error would be detected during compilation because it couldn’t find the symbol, but I had only compiled libraries that didn’t depend on other libraries using std::string. However, these libraries were used by my programs (with strings), and now those programs couldn’t start probably. In order to update the programs that were no longer working all the libraries that they depend on had to be recompiled to use the new std::string. In order to solve this; I had to recursively trace dependencies until I found a library that didn’t depend on std::string and start compiling from there, until I reached the program that had stopped working.

The most helpful discovery however was that some important libraries had switched major version number. When this happens it causes a new .so.# file to be created; and often breaks older dependencies. Once I started getting libraries recompiled; I wrote a script that would give me a list of all programs with broken dependencies in a folder. Using this list I reduced the problem to copy-pasting and resolving the dependency order. This probably saved me hours of debugging to find out which libraries were outdated. The very short script is shown below.

#!/bin/bash
FILES=./*
STRING=""
for f in $FILES
do
    if `ldd "$f" 2>/dev/null | grep -q "not found"`; then
        echo "IN: $f"
        STRING=$STRING"\t${f:2}\n"
        ldd $f | grep "not found"
    fi
done
printf "Reinstall the following libraries:"
printf $STRING

After working on this since Wednesday I have gotten most of the programs back into working order for daily use; but obviously I haven’t attempted to run a fraction of all installed programs yet. It will be interesting to see for how long I will continue to receive random linking errors.