2

I am struggling to find a way to pass arguments to a python script within a docker container based on ubuntu. I am working with docker-compose.yml.

Please find the example below!

docker-compose.yml

version: "3"
services:
   bcp:
      image: ubuntu:18.04
      restart: always
      tty: true
      entrypoint: ["/bin/bash", "/ingestion/bcp-entrypoint.sh"]
      volumes:
          - ./services/bcp:/ingestion/services/bcp
          - ./bcp-entrypoint.sh:/ingestion/bcp-entrypoint.sh

bcp-entrypoint.sh

apt-get update
apt-get upgrade -y
apt-get clean -y
apt-get install -y python3-pip
...

python script

required_args.add_argument("--database", metavar="str", type=str, help="database from where to extract", required=True)

The way I call the script - in container and on host machine - is python3 -m services.bcp --database foo and it works just perfect. The question is, how can I achieve the same from host machine on docker container?

Basically, I am looking for something like docker-compose exec services.bcp --database foo.

I do not want to use dockerfile! Ideally everything is based on docker-compose.

4
  • You can add the extra options to your entrypoint: array. (Though Compose may not be the tool you're looking for here.)
    – David Maze
    Aug 11, 2020 at 10:22
  • Thanks for the reply. Something like: entrypoint: ["/bin/bash", "/ingestion/bcp-entrypoint.sh", "python3 m", "boooo.py"]?
    – abe
    Aug 11, 2020 at 10:23
  • The container only runs one command, so if entrypoint.sh knows how to handle a sub-command passed this way (maybe it ends with exec "$@") and you correctly break the command into words, that might work.
    – David Maze
    Aug 11, 2020 at 10:26
  • Do you mean, I need to put at the end of bcp-entrypoint.sh something like python3 m --database $arg1? And then I call docker-compose exec bcp arg1=foo? Then the question is how do I pass args to the entrypoint?
    – abe
    Aug 11, 2020 at 10:38

2 Answers 2

5

Here is a solution for both docker and docker compose of how to parse arguments within the docker container:

python script

#!/usr/bin/env python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("name", help="Name of person to greet")

args = parser.parse_args()

print(f"Hello there, {args.name}!")

With a custom dockerfile (seems like the most straight-forward for this situation):

FROM python:3.8

COPY main.py .

ENTRYPOINT ["./main.py"]

Then build and run the containerized cli:

# Should make the script executable to invoke directly
$ chmod +x main.py

$ docker build -t dockerized-cli .
$ docker run -it --rm dockerized-cli -h
usage: main.py [-h] name

positional arguments:
  name        Name of person to greet

optional arguments:
  -h, --help  show this help message and exit
$ docker run -it --rm dockerized-cli Jeff
Hello there, Jeff!

With just docker-compose (Original question)

version: "3"
services:
  app:
    image: python:3.8
    volumes:
      - .:/opt/app
    entrypoint: ["/opt/app/main.py"]

Then to run,

$ docker-compose run app Jeff
Hello there, Jeff!

If you need to use the ubuntu:18.04 image and still can't build your own image off of it (to install python outside of the entrypoint), then you need to do what you already have which is create an entrypoint script wherein you first install python and then invoke your script. Something of note is that now your entrypoint is the entrypoint script itself and not your python CLI so you need to propogate any arguments from the shell script to your python script. Below is a simple example of how to do so - notice the bash variable "$@":

entrypoint.sh

#!/bin/bash

# install python here...

python print_args.py $@

print_args.py

import sys
print(sys.argv)
2
  • Hi, thanks for your reply! I do need to go through ubuntu since I have to install some packages, namely bcp. I did find the way to parse the arguments docker-compose run bcp --database=foo. Then, in the while loop I take the named argument like --database=*) database="${1#*=}". However, I have another quick question. When I am inside of container I can execute bcp command. But if I run the script - meaning, at the end of bcp-entrypoint.sh - I have the python command python3 -m services.bcp --database $database, it says bcp command does not exist. But it does ...
    – abe
    Aug 11, 2020 at 15:25
  • 1
    Ok, I found the solution to the question above - in comment. You need to export and then source ~/.bash_profile
    – abe
    Aug 11, 2020 at 16:16
0

The only solution I found is the following: you can pass on startup(passing arguments on building is much easier) arguments from docker-compose to Dockerfile via ARG keyword. The issue here is that ARG has a very bizarre scope, so ENTRYPOINT cannot see defined ARGs, but does see ENVs. So you have to copy ARG to ENV.

ingest_data.py

#!/usr/bin/env python
import argparse

if __name__ == '__main__':    
    
    parser = argparse.ArgumentParser(description='Ingest CSV data to Postgres')

    parser.add_argument('--host', required=True, help='host for postgres')
    parser.add_argument('--port', required=True, help='port for postgres')
    parser.add_argument('--db', required=True, help='database name for postgres')
    parser.add_argument('--table_name', required=True, help='name of the table where we will write the results to')

    args = parser.parse_args()

In a Dockerfile. Note: To pass arguments to ENTRYPOINT you should run it in a shell form, not in a exec form!:

FROM python:3.9.1

RUN pip install pandas sqlalchemy psycopg2 pyarrow

WORKDIR /app
COPY ingest_data.py ingest_data.py

ARG host
ARG port
ARG db
ARG table_name

ENV host=${host} port=${port} db=${db} table_name=${table_name}
RUN echo "Host: ${host}, Port: ${port}, DB: $db"

ENTRYPOINT python ingest_data.py -host=$host --port=$port --db=$db --table_name=$table_name

In a docker-compose.yaml

version: "3"
services:
  app:
    build: 
      context: .
      args: 
        - host=pgdatabase
        - port=5432
        - db=ny_taxi
        - table_name=yellow_taxi_trips_n

Finally to run it,

$ docker-compose up -d 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.