5

I'm creating an Android library with NDK (r19b) for arm, aarch64, x86, and x86_64. All is OK, except that when I build an app for aarch64 architecture, I get the following error message.

ld.lld: error: found local symbol '_edata' in global part of symbol table in file libmystuff.so
ld.lld: error: found local symbol '_end' in global part of symbol table in file libmystuff.so
ld.lld: error: found local symbol '__bss_start' in global part of symbol table in file libmystuff.so

When I checked each build variant with readelf -s libmystuff.so, I noticed that only aarch64 is different.

[arm]
    4021: 007a30f0     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    4022: 007c6b10     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    4023: 007a30f0     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[x86]
    3848: 00c82c88     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    3849: 00ca4b28     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    3850: 00c82c88     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[x86_64]
    3874: 0000000000c9b890     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    3875: 0000000000ce5f68     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    3876: 0000000000c9b890     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[aarch64]
       3: 0000000000b4f168     0 NOTYPE  LOCAL  DEFAULT  ABS _edata
       4: 0000000000b990e8     0 NOTYPE  LOCAL  DEFAULT  ABS _end
       5: 0000000000b4f168     0 NOTYPE  LOCAL  DEFAULT  ABS __bss_start

     865: 0000000000b9e3e8     0 NOTYPE  GLOBAL DEFAULT  ABS __end__
    2468: 0000000000b54168     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start__

I can surely see _edata, _end, and __bss_start are in LOCAL instead of GLOBAL, but I don't (or at least I think I don't) do anything special for aarch64; they all use the same build configuration.

jni/Application.mk
    NDK_TOOLCHAIN_VERSION := clang
    APP_STL := c++_static
    APP_CFLAGS := -fstack-protector-all -fvisibility=hidden -ffunction-sections -fdata-sections
    APP_CPPFLAGS := -fstack-protector-all -std=c++11 -fvisibility=hidden -ffunction-sections -fdata-sections -frtti
    APP_LDFLAGS := -Wl,--gc-sections,-fvisibility=hidden,--strip-debug

So, why is aarch64 different? Better yet, how do I move those to GLOBAL?

[UPDATE] Thanks to the gentle folks at https://github.com/android-ndk/ndk/issues/927, I found the solution works best; notice "-fuse-ld=lld" at the end.

APP_LDFLAGS := -Wl,--gc-sections,--strip-debug -fvisibility=hidden -fuse-ld=lld

This way, I still get to keep --gc-sections, and --no-fatal-warnings is not necessary either.

[MORE UPDATE] The example shown in this question is using ndk-build, and if you are using Android Studio with Gradle, you'll most likely have CMake. In that case, add your compiler flags as the following.

[app/build.gradle]
android {
    ...
    defaultConfig {
        externalNativeBuild {
            cmake {
                cppFlags '-fuse-ld=lld'
            }
2

2 Answers 2

3

APP_LDFLAGS := -Wl,-fvisibility=hidden

Does this do anything? This option doesn't appear in the help page for bfd, but those symbols are emitted by the linker so I'm wondering if that's what's causing this.

Failing that, try adding -fuse-ld=gold (or -fuse-ld=lld if you're feeling adventurous, but there are a number of Windows KIs with that if that's something you need to support) to your APP_LDFLAGS. The biggest difference between arm64 and the other architectures in the NDK is that we still use bfd for arm64.

Since these are symbols that are emitted by the linker, this seems like the most likely culprit.

8
  • Once I removed --gc-sections, nothing showed up in LOCAL. The library size increased slightly, but I can live with it.
    – solamour
    Commented Mar 6, 2019 at 20:52
  • That sounds like a bug to me. You can probably keep that flag if you switch to using gold (though be warned that it can generate bad debug info for lldb for aarch64) or lld (which is in beta for the NDK). It's very unfortunate that there isn't one good choice for a linker in the NDK right now, but we're working on that.
    – Dan Albert
    Commented Mar 6, 2019 at 21:09
  • Neither -fuse-ld=gold nor -fuse-ld=lld made any difference; _edata, _end, and __bss_start were all in LOCAL. I might need to chalk it as the current NDK's limitation.
    – solamour
    Commented Mar 6, 2019 at 22:14
  • Yeah, I just noticed that lld is what you were using in the first place too. That's definitely weird, and the fix is weird too. I filed github.com/android-ndk/ndk/issues/927 for this. If you could provide a test case that would be awesome (haven't tried anything yet, so maybe this happens even in a simple hello world?)
    – Dan Albert
    Commented Mar 8, 2019 at 8:10
  • 1
    ^ this is an excellent way to turn build failures into runtime failures.
    – Dan Albert
    Commented Jan 11, 2021 at 21:13
0

I faced this problem when using an external libExternal.so file.

Since I did not have the source code, I couldn't rebuild it with LLD.

I eventually was able to use the libExternal.so file by disabling the linker warnings in my project. This link was very helpful https://github.com/android/ndk/issues/927

I have made a JNI android app in AOSP (Android 10) that uses a sample.cpp file that uses libExternal.so. My app was able to be built with mm command and worked properly on device.

Here is the Android.mk file that I used in my AOSP build (Android 10).

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)
LOCAL_MODULE := External
LOCAL_SRC_FILES := libExternal.so
LOCAL_MODULE_TAGS := samples
#LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_SUFFIX := .so
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
#LOCAL_LDFLAGS += -fuse-ld=gold
#LOCAL_EXPORT_LDFLAGS += -fuse-ld=gold
#LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
include $(BUILD_PREBUILT)

include $(CLEAR_VARS)
LOCAL_MODULE := c++
LOCAL_SRC_FILES := libc++.so
LOCAL_MODULE_SUFFIX := .so
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
#LOCAL_LDFLAGS += -fuse-ld=gold
#LOCAL_EXPORT_LDFLAGS += -fuse-ld=gold
#LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
include $(BUILD_PREBUILT)

include $(CLEAR_VARS)
LOCAL_MODULE := libsample
LOCAL_C_INCLUDES := $(LOCAL_PATH)/include
#LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
#LOCAL_CFLAGS += -Iinclude
LOCAL_SRC_FILES := sample.cpp
LOCAL_REQUIRED_MODULES := External
#LOCAL_SHARED_LIBRARIES := External cdsprpc cutils c++ base log
LOCAL_SHARED_LIBRARIES := External
LOCAL_MODULE_TARGET_ARCH := arm64
#LOCAL_MODULE_TARGET_ARCH := arm64-v8a
#LOCAL_LDLIBS += $(LOCAL_PATH)/libExternal.so
#https://stackoverflow.com/questions/65597460/how-to-restrict-android-mk-to-compile-a-module-to-generate-only-64bit-library
LOCAL_CFLAGS += -Wall -std=c99 -g
LOCAL_CPPFLAGS += -Wall -std=c++03 -D_GLIBCXX_USE_CXX11_ABI=0 -g
# Does not build even after commenting below line
#LOCAL_LDLIBS := -lm -lstdc++ -ldl
#LOCAL_LDFLAGS += -fuse-ld=gold
#LOCAL_EXPORT_LDFLAGS += -fuse-ld=gold
#include $(BUILD_EXECUTABLE)
LOCAL_DISABLE_FATAL_LINKER_WARNINGS := true
LOCAL_LDFLAGS := -Wl,--no-fatal-warnings
include $(BUILD_SHARED_LIBRARY)

include $(CLEAR_VARS)
res_dir := res $(LOCAL_PATH)/res
#LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_TAGS := samples
LOCAL_SRC_FILES := $(call all-java-files-under, java)
LOCAL_RESOURCE_DIR := $(addprefix $(LOCAL_PATH)/, $(res_dir))
LOCAL_USE_AAPT2 := true
LOCAL_JAVA_LIBRARIES := com.google.android.material_material
LOCAL_STATIC_ANDROID_LIBRARIES := \
        androidx.appcompat_appcompat \
        androidx-constraintlayout_constraintlayout \
        androidx.preference_preference \
        androidx.fragment_fragment \
        androidx.core_core
#LOCAL_REQUIRED_MODULES := sample External
LOCAL_REQUIRED_MODULES := External \
                           libsample \
                           libc++
LOCAL_SHARED_LIBRARIES := External \
                           libsample \
                           libc++
#LOCAL_LDLIBS += -lsample
LOCAL_JNI_SHARED_LIBRARIES := libsample libc++
#LOCAL_JNI_SHARED_LIBRARIES := libsample
#LOCAL_PREBUILT_JNI_LIBS := $(LOCAL_PATH)/libsample.so
LOCAL_PREBUILT_JNI_LIBS := libExternal.so
LOCAL_CERTIFICATE := platform
LOCAL_PRIVILEGED_MODULE := true
LOCAL_PACKAGE_NAME := NativeCPP
LOCAL_PRIVATE_PLATFORM_APIS := true
include $(BUILD_PACKAGE)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.