The code included here is helpful for converting your docker-based lambda to a zip-based lambda.
In our case, we needed an entrypoint lambda to handle incoming requests from API gateway,
and these requests needed to be answered as quickly as possible. The entrypoint is just
responsible for validating the payload and then pushing the work to SQS for async resolution.
We need the entrypoint Lambda to be a zip deployment so that it responds asap, but then
we use the docker deployments for the async handlers that pull work from SQS.
This way, we build the framework via a single Dockerfile, and then we simply dump the
framework from the built docker image to a .zip file for the entrypoint Lambda.
(We previously experimented with deploying the entrypoint Lambda as a docker container
but found that it was too slow. The details of this experiment are in this comment.)
The directory structure looks like:
project_root_dir
├── framework_lib_main.Dockerfile
├── py-lib
│ └── framework_lib_main
│ ├── *.py
│ └── framework_lib_a
│ ├── *.py
│ └── framework_lib_b
│ ├── *.py
The framework_lib_main.Dockerfile
looks like:
# https://docs.aws.amazon.com/lambda/latest/dg/python-image.html#python-alt-create
# Use amazonlinux image to support extracting to zip for lambda:
# https://docs.aws.amazon.com/lambda/latest/dg/python-package.html#python-package-source-dist
##
# This will build `framework_lib_main`
#
# The requirements.txt of `framework_lib_main` includes local packages:
# - framework_lib_a
# - framework_lib_b
# Each local package has its own setup.py
#
# The requirements.txt of `framework_lib_main` also includes non-local packages, for example:
# - boto3
# - awslambdaric
##
ARG PY_VERSION="3.11"
ARG BUILD_DIR="/py-lib"
FROM public.ecr.aws/amazonlinux/amazonlinux:latest AS build-image
ARG PY_VERSION
ARG BUILD_DIR
RUN mkdir -p ${BUILD_DIR}
WORKDIR /py-lib-src
COPY ./framework_lib_a ./framework_lib_a/
COPY ./framework_lib_b ./framework_lib_b/
RUN dnf update && dnf install -y python${PY_VERSION} python${PY_VERSION}-pip
WORKDIR /py-lib-src/framework_lib_main
COPY ./framework_lib_main/requirements.txt .
RUN \
/usr/bin/python${PY_VERSION} -m pip install --upgrade --no-cache-dir pip && \
/usr/bin/python${PY_VERSION} -m pip install --upgrade --no-cache-dir pip-tools && \
/usr/bin/python${PY_VERSION} -m pip install --no-cache-dir --target "${BUILD_DIR}" -r requirements.txt
COPY ./framework_lib_main .
RUN /usr/bin/python${PY_VERSION} -m pip install --no-cache-dir --target "${BUILD_DIR}" .
FROM public.ecr.aws/amazonlinux/amazonlinux:latest
ARG PY_VERSION
ARG BUILD_DIR
RUN dnf update && dnf install -y python${PY_VERSION}
WORKDIR ${BUILD_DIR}
COPY --from=build-image ${BUILD_DIR} ${BUILD_DIR}
RUN ln -s /usr/bin/python${PY_VERSION} /usr/local/bin/python_entrypoint
ENTRYPOINT ["/usr/local/bin/python_entrypoint"]
Below is the script responsible for (1) creating the .zip, (2) uploading the .zip to S3, and (3) updating the Lambda to use the new .zip:
if [[ -z "$ECR_REPO_URI" ]]; then
echo "no value for \$ECR_REPO_URI" >&2
exit 1
fi
if [[ -z "$S3_BUCKET" ]]; then
echo "no value for \$S3_BUCKET" >&2
exit 1
fi
if [[ -z "$S3_KEY" ]]; then
echo "no value for \$S3_KEY" >&2
exit 1
fi
if [[ -z "$FUNCTION_NAME" ]]; then
echo "no value for \$FUNCTION_NAME" >&2
exit 1
fi
push_to_s3() {
ZIP_DIR='/tmp/docker_to_lambda'
ZIP_PATH="$ZIP_DIR/lambda.zip"
SRC_DIR='/py-lib'
mkdir -p "$ZIP_DIR"
docker run \
--rm \
-v "$ZIP_DIR:$ZIP_DIR" \
--entrypoint /bin/sh \
"$ECR_REPO_URI:latest" \
-c "$(cat <<-EOF
dnf update && \
dnf install -y findutils zip && \
cd "$SRC_DIR" && \
chmod -R 644 . && \
find . -type d -exec chmod 755 '{}' \; && \
find . -perm /111 -type f -exec chmod 755 '{}' \; && \
zip -9 -r "$ZIP_PATH" *
EOF
)"
aws s3api put-object \
--bucket "$S3_BUCKET" \
--key "$S3_KEY" \
--body "$ZIP_PATH"
rm "$ZIP_PATH"
}
update_lambda() {
(aws lambda update-function-code \
--function-name "$FUNCTION_NAME" \
--s3-bucket "$S3_BUCKET" \
--s3-key "$S3_KEY" \
1>/dev/null) || return 1
aws lambda wait function-updated --function-name "$FUNCTION_NAME" || return 1
}
push_to_s3 && update_lambda