2

Error message:

Undefined symbols for architecture arm64:
  "std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> >::shrink_to_fit()", referenced from:
      base::UTF8ToUTF16(char const*, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> >*) in libbase.a(utf_string_conversions.o)
      base::WideToUTF16(wchar_t const*, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator<unsigned short> >*) in libbase.a(utf_string_conversions.o)

ld: symbol(s) not found for architecture arm64

subprocess.CalledProcessError: Command '['clang++', '-B', '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/', '-shared', '-Xlinker', '-install_name', '-Xlinker', '@rpath/Cronet.framework/Cronet', '-Xlinker', '-objc_abi_version', '-Xlinker', '2', '-arch', 'arm64', '-Werror', '-isysroot', '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.0.sdk', '-stdlib=libc++', '-miphoneos-version-min=8.0', '-fembed-bitcode', '-Wl,-ObjC', '-o', 'obj/components/cronet/ios/arm64/Cronet', '-Wl,-filelist,obj/components/cronet/ios/arm64/Cronet.rsp', '-framework', 'UIKit', '-framework', 'CoreFoundation', '-framework', 'CoreGraphics', '-framework', 'CoreText', '-framework', 'Foundation', '-framework', 'JavaScriptCore', '-framework', 'CFNetwork', '-framework', 'MobileCoreServices', '-framework', 'Security', '-framework', 'SystemConfiguration', '-lresolv']' returned non-zero exit status 1
8
  • 2
    I’m aware of one report of nearly identical errors (undefined symbols used by these same two functions) when building Chromium from QtWebEngine 5.15.7 on an Apple Silicon Mac running macOS 12.0 Monterey. Commented Nov 3, 2021 at 23:51
  • Is there a workround?
    – yfainaer
    Commented Nov 5, 2021 at 8:45
  • Not that I am aware of. Consider subscribing to trac.macports.org/ticket/63725 for updates. Commented Nov 5, 2021 at 22:43
  • Did you 0) update libbase, 1) include <string> before libbase? This comment seems to indicate those functions are only needed on Windows
    – viraltaco_
    Commented Nov 26, 2021 at 12:13
  • Can you provide a minimal working example that causes this issue? Commented Nov 28, 2021 at 11:48

3 Answers 3

3

This was pretty tricky to track down, but I think I found the issue. First of all, string16.ii can be simplified to:

template <class T>
struct basic_string {
    __attribute__((internal_linkage))
    void shrink_to_fit();
};

template <class T>
void basic_string<T>::shrink_to_fit() { }

template class basic_string<char>;

utf_string_conversions.ii can be simplified to:

template <class T>
struct basic_string {
    __attribute__((internal_linkage))
    void shrink_to_fit();
};

template <class T>
void basic_string<T>::shrink_to_fit() { }

extern template class basic_string<char>;

int main() {
    basic_string<char> s;
    s.shrink_to_fit();
}

Since shrink_to_fit has internal linkage, the compiler doesn't even bother emitting it inside string16.o, because it is not used in that TU. If it did emit it in string16.o, it would be dead code since no other TU can refer to it anyway.

Then, from utf_string_conversions.o, we see an extern template instantiation declaration, which basically promises that we will be able to find shrink_to_fit() in some other TU (presumably string16.o). Of course, that can't be the case, since another TU can't "export" shrink_to_fit, which has internal linkage as explained above.

As a result, we end up in a situation where utf_string_conversions.o expects to see shrink_to_fit() in some other TU, but the other TU that should provide it doesn't. Furthermore, if we compile the above, we can actually see that the compiler is warning us from exactly that:

<stdin>:4:10: warning: function 'basic_string<char>::shrink_to_fit' has internal linkage but is not defined [-Wundefined-internal]
    void shrink_to_fit();
         ^
<stdin>:14:7: note: used here
    s.shrink_to_fit();
      ^
1 warning generated.
Undefined symbols for architecture x86_64:
  "basic_string<char>::shrink_to_fit()", referenced from:
      _main in main.o
ld: symbol(s) not found for architecture x86_64

This warning does not show up in the original code because warnings inside system headers are suppressed. This really tripped me up initially, and I think it would make sense for the compiler to warn if the explicit template instantiation declaration appears outside of a system header, regardless of whether the class itself is declared inside a system header. I actually removed the system header directives from the original reproducer and I was able to get the same warning, see this.

Now, you might be wondering why that doesn't happen with other methods of basic_string? Well, apparently, most if not all other methods of basic_string are either

  1. defined inside the class (and are hence implicitly inline), or
  2. defined outside the class but marked with inline explicitly, or
  3. not marked with __attribute__((internal_linkage))

I believe that's the reason why this issue is only showing up with shrink_to_fit(): it is a non-inline function because it is defined outside its class definition, despite the class being a template.

Concretely:

  1. Chromium should consider removing their explicit instantiation -- those are brittle, as explained here.
  2. I will fix libc++'s shrink_to_fit() so it is inline. (edit: here)
  3. I will file a Clang bug to discuss the possibility of emitting that warning when the explicit template instantiation declaration appears outside of a system header. (edit: here)

Thanks for reporting this tricky issue.

1

I don’t know why the output object file for base/strings/string16.cc doesn’t contain base::string16::shrink_to_fit() (my current guess is that it has something to do with the include/c++/v1/string header file from libc++ in the macOS 12 and iOS 15 SDKs), but it does appear to contain base::string16::reserve(unsigned long). For std::basic_string in C++17 and earlier (not C++20 or later), shrink_to_fit() is equivalent to reserve(0). Since the affected version of string16.cc is presumably being compiled with -std=c++14, a workaround is to replace (in UTFConversion() in base/strings/utf_string_conversions.cc):

  bool res = DoUTFConversion(src_str.data(), src_len32, dest, &dest_len32);

  dest_str->resize(dest_len32);
  dest_str->shrink_to_fit();

  return res;
}

with:

  bool res = DoUTFConversion(src_str.data(), src_len32, dest, &dest_len32);

  dest_str->resize(dest_len32);
  dest_str->reserve(0);

  return res;
}
10
  • 1
    This suggestion isn't great, instead we should figure out why string16.cc doesn't contain shrink_to_fit. Can you please share a self-contained reproducer for creating that object file? Commented Dec 7, 2021 at 22:13
  • 1.checkout chromium source code 2.setup Cronet iOS build environment 3.compile with Xcode 13.1 and iOS 15.0, MacOS 12.0.1
    – yfainaer
    Commented Dec 8, 2021 at 8:56
  • This is a lot of work just to get my hands just on one object file. If you have that setup locally, can you share a Gist with the preprocessed content of the file and the command-line invocation used to compile it? That should be enough. Commented Dec 8, 2021 at 14:19
  • @LouisDionne Here is the preprocessed string16.cc (from qtwebengine-chromium with macOS 12 SDK rather than cronet with iOS 15 SDK, but exhibits the same issue): gist.github.com/chrstphrchvz/f9d486d060bf4d1726dfc82d4f47bb17 Commented Dec 9, 2021 at 2:20
  • Thanks @chrstphrchvz. Can you show me a minimal translation unit (preprocessed) that, when linked against the object file, fails with the missing symbol error? And also the compiler invocation you're using. Commented Dec 9, 2021 at 21:26
0

Here is my feedback to Apple and their reply:

Basic information

Please provide a descriptive title for your feedback:

undefined symbols of shrink_to_fit() when compiling Cronet on iOS 15 with Xcode 13

Which area are you seeing an issue with?

Xcode

What type of feedback are you reporting?

Incorrect/unexpected behavior

Details

What version of Xcode are you using?

13.1

Description

Please describe the issue:

When compiling Cronet on the iOS platform with Xcode 13.1 and iOS 15.0, an error happens:

Undefined symbols for architecture arm64:

"std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator >::shrink_to_fit()", referenced from: base::UTF8ToUTF16(char const*, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator >) in libbase.a(utf_string_conversions.o) base::WideToUTF16(wchar_t const, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator >*) in libbase.a(utf_string_conversions.o)

ld: symbol(s) not found for architecture arm64

subprocess.CalledProcessError: Command '['clang++', '-B', '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/', '-shared', '-Xlinker', '-install_name', '-Xlinker', '@rpath/Cronet.framework/Cronet', '-Xlinker', '-objc_abi_version', '-Xlinker', '2', '-arch', 'arm64', '-Werror', '-isysroot', '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.0.sdk', '-stdlib=libc++', '-miphoneos-version-min=8.0', '-fembed-bitcode', '-Wl,-ObjC', '-o', 'obj/components/cronet/ios/arm64/Cronet', '-Wl,-filelist,obj/components/cronet/ios/arm64/Cronet.rsp', '-framework', 'UIKit', '-framework', 'CoreFoundation', '-framework', 'CoreGraphics', '-framework', 'CoreText', '-framework', 'Foundation', '-framework', 'JavaScriptCore', '-framework', 'CFNetwork', '-framework', 'MobileCoreServices', '-framework', 'Security', '-framework', 'SystemConfiguration', '-lresolv']' returned non-zero exit status 1

Please list the steps you took to reproduce the issue

  1. checkout Chromium source code
  2. set up Cronet build environment
  3. compile with Xcode 13.1 and iOS 15.0, macOS v12.0.1 (Sierra)

What did you expect to happen?

Compile successfully like Xcode 12 does

What actually happened?

Undefined symbols for architecture arm64: "std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator >::shrink_to_fit()", referenced from: base::UTF8ToUTF16(char const*, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator >) in libbase.a(utf_string_conversions.o) base::WideToUTF16(wchar_t const, unsigned long, std::__1::basic_string<unsigned short, base::string16_internals::string16_char_traits, std::__1::allocator >*) in libbase.a(utf_string_conversions.o)

Apple's reply:

It looks like you are using std::basic_string<char16, base::string16_char_traits>, which comes from Chromium. Since there is an explicit instantiation declaration for that type 2, the instantiation of std::basic_string<char16, base::string16_char_traits> appears to be provided in a library that probably ships with Chromium. You need to make sure you link against it, otherwise such link errors are to be expected.

We think this link may also be relevant: https://stackoverflow.com/a/17484003/627587. It doesn't pertain to libc++, however the issue is most likely the same.

Also, in case you are wondering why this started happening with Xcode 13.1, we suspect this is simply because std::basic_string<char16, base::string16_char_traits>::shrink_to_fit was previously being inlined in your code, which means that the fact that you're not linking to the required Chromium library was not triggering any error. We made some changes to shrink_to_fit and the compiler is probably deciding not to inline it anymore because it is more efficient to use the instantiation announced by the string16.h header, which uncovers the fact that you haven't been linking against that library.

7
  • I disagree that the cause is “not linking the required Chromium library”. But presumably Apple was provided too little information to suggest a more likely alternative… Commented Dec 9, 2021 at 11:31
  • 1
    …The objects for string16.cc and utf_string_conversions.cc are linked simultaneously into libbase.a (this is when the linking error has appeared). I can replace shrink_to_fit() usage with reserve(0) and not get a similar undefined symbols error for reserve(unsigned long), which I think suggests that the required “library” for string16 is being linked in. And, there was no similar linking error for previous macOS versions’ SDKs (assuming chromium build toolchain behaving differently on each macOS version isn’t a factor). Commented Dec 9, 2021 at 11:31
  • The first part (without the literal questions (but some or all could be transformed into sub section titles)) belongs in the question. Stack Overflow is not a forum. You can edit (change) both: change your answer and change your question. Or in other words, this ought to be changed into the Q&A format. Thanks in advance. Commented Dec 9, 2021 at 15:25
  • This answer is being discussed on meta.
    – cigien
    Commented Dec 9, 2021 at 17:14
  • 1
    I wrote that Apple reply. I was indeed taking a guess at what might be happening, and the most likely from looking at the Chromium sources was that you were not linking against the required .a. I'll take a look at the preprocessed output above. Commented Dec 9, 2021 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.