When deploying a Python spaCy app to AWS Lambda, I get the following error in the deploy (see below). Why deploy using zappa? The zip file is 125MB compressed, so a direct upload from the aws-cli fails on space, and a transfer to S3 also fails because the uncompressed is more than 250MB.
My program, itself, is not doing any multithreading nor multiprocessing, and it is only using spaCy 2.0. I built and deployed on an EC2 AWS Linux t2.medium. What are the exact steps that get a round-trip answer from a spaCy AWS Lambda function?
Failure trace below:
[1520570028387] Failed to find library...right filename?
[1520570029826] [Errno 38] Function not implemented: OSError
Traceback (most recent call last):
File "/var/task/handler.py", line 509, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 237, in lambda_handler
handler = cls()
File "/var/task/handler.py", line 129, in __init__
self.app_module = importlib.import_module(self.settings.APP_MODULE)
File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/tmp/spaciness/front.py", line 1, in <module>
import spacy
File "/tmp/spaciness/spacy/__init__.py", line 4, in <module>
from .cli.info import info as cli_info
File "/tmp/spaciness/spacy/cli/__init__.py", line 1, in <module>
from .download import download
File "/tmp/spaciness/spacy/cli/download.py", line 10, in <module>
from .link import link
File "/tmp/spaciness/spacy/cli/link.py", line 7, in <module>
from ..compat import symlink_to, path2str
File "/tmp/spaciness/spacy/compat.py", line 11, in <module>
from thinc.neural.util import copy_array
File "/tmp/spaciness/thinc/neural/__init__.py", line 1, in <module>
from ._classes.model import Model
File "/tmp/spaciness/thinc/neural/_classes/model.py", line 12, in <module>
from ..train import Trainer
File "/tmp/spaciness/thinc/neural/train.py", line 7, in <module>
from tqdm import tqdm
File "/tmp/spaciness/tqdm/__init__.py", line 1, in <module>
from ._tqdm import tqdm
File "/tmp/spaciness/tqdm/_tqdm.py", line 53, in <module>
mp_lock = mp.Lock() # multiprocessing lock
File "/var/lang/lib/python3.6/multiprocessing/context.py", line 67, in Lock
return Lock(ctx=self.get_context())
File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 163, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/var/lang/lib/python3.6/multiprocessing/synchronize.py", line 60, in __init__
unlink_now)
OSError: [Errno 38] Function not implemented