4

I have created an AWS Lambda Layer using following command:

aws lambda publish-layer-version --layer-name TensorflowLambdaLayer --compatible-runtimes go1.x --zip-file fileb://tensorflowLayer.zip

Here is ARN that was generated: `arn:aws:lambda:us-east-1:757767972066:layer:TensorflowLambdaLayer:1

When I try to run my Lambda function which uses Tesnroflow library via AWS SAM, it gets stuck on "mounting" step:

2019-07-18 15:51:29 Mounting /tmp/tmpgz8cb80s as /var/task:ro,delegated inside runtime container

Once I terminate it with Ctrl + C I am getting following message:

^C/var/task/bin/inference: error while loading shared libraries: libtensorflow.so: cannot open shared object file: No such file or directory
Makefile:82: recipe for target 'run-inference' failed

Here is crucial part of my template.yml:

Parameters:
  LambdaTensorflowLayerArn:
    Type: String
    Default: 'arn:aws:lambda:us-east-1:757767972066:layer:TensorflowLambdaLayer:1'
  LambdaFFMPEGLayerArn:
    Type: String
    Default: 'arn:aws:lambda:us-east-1:757767972066:layer:ffmpeg:1'

Inference:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: bin/inference
      Runtime: go1.x
      Timeout: 300
      CodeUri: ./bin/inference.zip
      Layers: 
         - Ref: LambdaFFMPEGLayerArn
         - Ref: LambdaTensorflowLayerArn

I am not sure what might be causing this

2

1 Answer 1

8

I was having the same issue with the message

error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory

In my case, it was trying to use TensorFlow with go. The problem is that the official installation for libtensorflow.so (and libtensorflow_framework.so) doesn't seem to be working if the package is left in /usr/local (recommended) or in other path. Using ldconfig as suggested for /usr/local doesn't help either. Also, the base example for gcc doesn't work (even with the -L. This has surprised me) until the LD_LIBRARY_PATH is set:

 $ gcc -I/usr/local/include -L/usr/local/lib hello_tf.c -ltensorflow -o hello_tf
 $ ./hello_tf 
 ./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory
 $ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
 $ ./hello_tf 
 Hello from TensorFlow C library version 1.14.0
 $ unset LD_LIBRARY_PATH 
 $ ./hello_tf 
 ./hello_tf: error while loading shared libraries: libtensorflow.so.1: cannot open shared object file: No such file or directory

This is the content of /usr/local in my case

 $ ll /usr/local/lib
 total 245424
 lrwxrwxrwx. 1 root root        28 dic 31  1999 libtensorflow_framework.so -> libtensorflow_framework.so.1
 lrwxrwxrwx. 1 root root        33 dic 31  1999 libtensorflow_framework.so.1 -> libtensorflow_framework.so.1.14.0
 -r-xr-xr-x. 1 root root  34748520 dic 31  1999 libtensorflow_framework.so.1.14.0
 lrwxrwxrwx. 1 root root        18 dic 31  1999 libtensorflow.so -> libtensorflow.so.1
 lrwxrwxrwx. 1 root root        23 dic 31  1999 libtensorflow.so.1 -> libtensorflow.so.1.14.0
 -r-xr-xr-x. 1 root root 216546752 dic 31  1999 libtensorflow.so.1.14.0

The other solution was to manually create the symlinks to /usr/lib. I don't know how to do that in a serverless config. Was going to write this in a comment but still dont have enough rep.

2
  • I see. Thank you for your answer! I will try this out and if it works, accept your answer. Could you please show me how does your serverless config look like? Did you use EVN VARIABLE's to set ld_library_path or? By the way, I am using Go as well. If you have some spare time would you mind adding me on skype:stefan.radonjic995 so we can chat for 5 min about this? It is really important to me. Commented Jul 23, 2019 at 8:02
  • Also, I should note that I am using AWS SAM for testing purposes. And when I run this no logs are being showed. I have a log that should be printed as soon as Handler is invoked. Does that meant that error occurs before Handler is invoked? If that is so, then I cannot set LD_LIBRARY_PATH from my program, only by configuration? Commented Jul 23, 2019 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.