2

Am deploying my ML model on Heroku using Django, I need en_core_web_lg for my application but couldn't install it

My requirements.txt is like:

..
..
djangorestframework==3.12.2
en-core-web-lg==2.3.1
en-core-web-sm==2.3.1
gunicorn==20.0.4
heroku==0.1.4
..

The Error is:

 ERROR: Could not find a version that satisfies the requirement en-core-web-lg==2.3.1 (from -r /tmp/build_c3075f3c_/requirements.txt (line 14)) (from versions: none)
       ERROR: No matching distribution found for en-core-web-lg==2.3.1 (from -r /tmp/build_c3075f3c_/requirements.txt (line 14))
 !     Push rejected, failed to compile Python app.
 !     Push failed

9
  • Did you check the logs? Commented Jan 3, 2021 at 11:19
  • It is possible that you do not have enough space left in your dyno. Check this out - heroku is not for compute, it is primarily for simple websites. I know the model itself is small (around 350MB), but see if your other deps already took up too much space - a free/hobby/standard has only 512mb, so you would only have 162 MB left - will your other deps fit here? Commented Jan 3, 2021 at 11:20
  • Yes! This link I got from one SO post but it's also not working. I just need the way to install this so that I can load it. Commented Jan 3, 2021 at 11:21
  • Does it install on your local machine? Commented Jan 3, 2021 at 11:22
  • There's no memory overlfow message. Commented Jan 3, 2021 at 11:23

2 Answers 2

9

Your error is because there is no pip library named en-core-web-lg. Keep in mind that you cannot put en-core-web-lg in the requirements.txt since it is not a library that you can install using pip. They are models that spacy uses.

The accepted answer is fine. However, if you want a complete automatic deployment, you can also add something like this into your code:

    try:
        nlp = spacy.load("en_core_web_md")
    except: # If not present, we download
        spacy.cli.download("en_core_web_md")
        nlp = spacy.load("en_core_web_md")

This will download the models automatically if they are not present. And, it will not require you to use the Heroku terminal. Only spacy on the requirements.txt file will be necessary.

Finally, keep in mind that spacy models are loaded in memory (RAM). The lg model (about 700MB) is too big to be fitted in a Free, Hobby or Standard 1X dynos (512MB of RAM). So it will fail to load and the dyno will be killed with R15 (https://devcenter.heroku.com/articles/error-codes#r15-memory-quota-vastly-exceeded).

More info about dynos: https://www.heroku.com/dynos

1
  • Yes, it works for other models too. I did use for en_core_web_lg, worked fine. Commented Jul 16, 2023 at 9:20
2

I don't know why it is not installing direct from requirements.txt file. There is another way it is not right way but the work will go on. First you have to remove the package from requirements.txt file for which the error is coming. Then push the app on Heroku, once your app have come to Heroku then write this code either from the terminal or from the Heroku dashboard.

If you are using terminal then:

heroku run bash

Then run:

pip install spacy

and then install which requiremts you want from spacy

python -m spacy download en_core_web_lg

if you are using Heroku dashboard then : first go to your Heroku dashboard click on your app and then at top right click on More and select the Run Console example image:

enter image description here than this interface will come up enter image description here in this you have to click on bash or type bash and run after this you can put same cammand and install.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.