2

I have tried the similar problems' solutions on here but none seem to work. It seems that I get a memory error when installing tensorflow from requirements.txt. Does anyone know of a workaround? I believe that installing with --no-cache-dir would fix it but I can't figure out how to get EB to do that. Thank you.

Logs:

----------------------------------------
Collecting tensorflow==2.8.0
  Downloading tensorflow-2.8.0-cp38-cp38-manylinux2010_x86_64.whl (497.6 MB)

2022/02/05 22:08:17.264961 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with requirements.txt file with error Command /bin/sh -c /var/app/venv/staging-LQM1lest/bin/pip install -r requirements.txt failed with error exit status 2. Stderr:ERROR: Exception:
Traceback (most recent call last):
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
    status = run_func(*args)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 338, in run
    requirement_set = resolver.resolve(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 482, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 349, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 201, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 281, in __init__
    super().__init__(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
    dist = self._prepare_distribution()
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 527, in _prepare_linked_requirement
    local_file = unpack_url(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 213, in unpack_url
    file = get_http_url(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 94, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/network/download.py", line 145, in __call__
    for chunk in chunks:
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/cli/progress_bars.py", line 144, in iter
    for x in it:
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_internal/network/utils.py", line 63, in response_chunks
    for chunk in response.raw.stream(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/urllib3/response.py", line 576, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/urllib3/response.py", line 519, in read
    data = self._fp.read(amt) if not fp_closed else b""
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 65, in read
    self._close()
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 52, in _close
    self.__callback(self.__buf.getvalue())
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/cachecontrol/controller.py", line 309, in cache_response
    cache_url, self.serializer.dumps(request, response, body=body)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/cachecontrol/serialize.py", line 72, in dumps
    return b",".join([b"cc=4", msgpack.dumps(data, use_bin_type=True)])
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/__init__.py", line 35, in packb
    return Packer(**kwargs).pack(o)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 960, in pack
    self._pack(obj)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 943, in _pack
    return self._pack_map_pairs(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 1045, in _pack_map_pairs
    self._pack(v, nest_limit - 1)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 943, in _pack
    return self._pack_map_pairs(
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 1045, in _pack_map_pairs
    self._pack(v, nest_limit - 1)
  File "/var/app/venv/staging-LQM1lest/lib/python3.8/site-packages/pip/_vendor/msgpack/fallback.py", line 889, in _pack
    return self._buffer.write(obj)
MemoryError
 

2022/02/05 22:08:17.264976 [INFO] Executing cleanup logic
2022/02/05 22:08:17.265065 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1644098897,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1644098897,"severity":"ERROR"}]}]}

Requirements.txt:

absl-py==1.0.0
asgiref==3.5.0
astunparse==1.6.3
awsebcli==3.20.3
backports.zoneinfo==0.2.1
botocore==1.23.49
cachetools==5.0.0
cement==2.8.2
certifi==2021.10.8
charset-normalizer==2.0.11
colorama==0.4.3
cycler==0.11.0
Django==4.0.2
django-crispy-forms==1.14.0
django-environ==0.8.1
flatbuffers==2.0
fonttools==4.29.1
future==0.16.0
gast==0.5.3
google-auth==2.6.0
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.43.0
h5py==3.6.0
idna==3.3
importlib-metadata==4.10.1
imutils==0.5.4
jmespath==0.10.0
keras==2.8.0
Keras-Preprocessing==1.1.2
kiwisolver==1.3.2
libclang==13.0.0
Markdown==3.3.6
matplotlib==3.5.1
numpy==1.22.2
oauthlib==3.2.0
opencv-python==4.5.5.62
opt-einsum==3.3.0
packaging==21.3
pathspec==0.9.0
Pillow==9.0.1
protobuf==3.19.4
psycopg2-binary==2.9.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==3.0.7
python-dateutil==2.8.2
PyYAML==5.4.1
requests==2.26.0
requests-oauthlib==1.3.1
rsa==4.8
semantic-version==2.8.5
six==1.14.0
sqlparse==0.4.2
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.8.0
tensorflow-io-gcs-filesystem==0.24.0
termcolor==1.1.0
tf-estimator-nightly==2.8.0.dev2021122109
typing_extensions==4.0.1
tzdata==2021.5
urllib3==1.26.8
wcwidth==0.1.9
Werkzeug==2.0.2
wrapt==1.13.3
zipp==3.7.0
1
  • 1
    @Marcin I ended up going with Opalstack as they were cheaper but yeah, I would’ve gone with a t2.small should I have stayed with AWS. thank you!
    – bdutton12
    Feb 9, 2022 at 13:44

1 Answer 1

2

The error says MemoryError. You must upgrade your ec2 instance to something with more memory. tensorflow is very memory hungry application.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.