7

I am running Python 3.7.2 and using Miniconda3 to create a new environment named test-env. I have installed the pyarrow package from the default channel into this environment; however, when I try and import pyarrow, the Python interpreter gives me an undefined symbol error:

Import Error: /test-env/lib/python3.7/site-packages/pyarrow/lib.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZNK5arrow6Status8ToStringB5cxx11Ev

From other threads on SO, I gather that this is due to a compiler incompatibility. I'm not a developer so I'm not sure how to fix this.

2
  • Nester, which operating system are you running on? I am seeing a similar issue on Ubuntu Linux, following updates (late last week?) to Arrow/Parquet. My hunch is that if you're also on Linux, this problem with the shared object is in the system libraries, at a lower level than pyarrow
    – Paco
    Nov 8, 2021 at 5:19
  • This looks like what happens when pyarrow is compiled with an earlier/out-of-date release of Apache Arrow C++ ?
    – Paco
    Nov 8, 2021 at 7:14

2 Answers 2

3

I had the same error after I installed by conda install, after I conda uninstall, I did pip install, the issue went away.

>>> import pyarrow
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dv6/anaconda3/envs/spark/lib/python3.6/site- 
  packages/pyarrow/__init__.py", line 54, in <module>
    from pyarrow.lib import cpu_count, set_cpu_count
  ImportError: /home/dv6/anaconda3/envs/spark/lib/python3.6/site- 
  packages/pyarrow/lib.cpython-36m-x86_64-linux-gnu.so: undefined symbol: 
  _ZNK5arrow5Field8B5cxx11Ev                                                                         
   >>> quit()

Then I conda uninstall

(spark) dv6@dv6:~$ conda uninstall pyarrow

Then I pip install

(spark) dv6@dv6:~$ pip install pyarrow --user
Collecting pyarrow
  Downloading pyarrow-0.16.0-cp36-cp36m-manylinux2014_x86_64.whl (63.1 MB)
     |████████████████████████████████| 63.1 MB 25 kB/s
Requirement already satisfied: numpy>=1.14 in 
./anaconda3/envs/spark/lib/python3.6/site-packages (from pyarrow) (1.18.1)
Requirement already satisfied: six>=1.0.0 in 
./anaconda3/envs/spark/lib/python3.6/site-packages (from pyarrow) (1.14.0)
Installing collected packages: pyarrow
Successfully installed pyarrow-0.16.0
(spark) dv6@dv6:~$ python -c "import pyarrow"
(spark) dv6@dv6:~$

Here is my Python version

(spark) dv6@dv6:~$ python
Python 3.6.10 |Anaconda, Inc.| (default, Jan  7 2020, 21:14:29)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
0

I would recommend opening a ticket at https://issues.apache.org/jira/projects/ARROW/issues/ARROW-4774?filter=allopenissues

Provide how you installed pyarrow (pip or conda), which distribution of linux. If you can reproduce this in a clean environment, even better.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.