14

I have an Elastic Beanstalk application deployed with a Docker container. The application itself is a Java Application.

My goal is to get the logs to Cloudwatch. In particular I would like to get the stdouterr.log file to Cloudwatch. The file can be found under /var/log/eb-docker/containers/eb-current-app/*

I followed the official AWS documentation here. Based on the example configuration files I managed to get the nginx Webrequest to Cloudwatch.

For the EB docker stdouterr log I adapted the cwl-log-setup.config file to the following:

Mappings:
  CWLogs:
    ApplicationLogGroup:
      LogFile: "/var/log/eb-docker/containers/eb-current-app/*"
      TimestampFormat: "%d/%b/%Y:%H:%M:%S %z"

Outputs:
  ApplicationLogGroup:
    Description: "The name of the Cloudwatch Logs Log Group created for this environments web server access logs. You can specify this by setting the value for the environment variable: WebRequestCWLogGroup. Please note: if you update this value, then you will need to go and clear out the old cloudwatch logs group and delete it through Cloudwatch Logs."
    Value: { "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup"}


Resources :
  AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup:    ## Must have prefix:  AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0
    Type: "AWS::Logs::LogGroup"
    DependsOn: AWSEBBeanstalkMetadata
    DeletionPolicy: Retain     ## this is required
    Properties:
      LogGroupName:
        "Fn::GetOptionSetting":
          Namespace: "aws:elasticbeanstalk:application:environment"
          OptionName: ApplicationLogGroup
          DefaultValue: {"Fn::Join":["-", [{ "Ref":"AWSEBEnvironmentName" }, "webrequests"]]}
      RetentionInDays: 14

The cloudwatch log group is created but no logs arrive. What steps am I missing or what is wrong in my configuration file?

3 Answers 3

5

I've just encountered the same issue - I managed to get the log files by changing the LogFile configuration to

Mappings:
  CWLogs:
    WebRequestLogGroup:
      LogFile: "/var/log/eb-docker/containers/eb-current-app/*.log"
      TimestampFormat: "%d/%b/%Y:%H:%M:%S %z"

Note this only works if there is a single log file, if you re-deploy a container or apply a configuration change which results in multiple logs in this directory then only the events from the log file with the most current modified time will be processed by the awslogs agent

Also by default the agent will compare the first line of the log file to determine if it is a different file, if the first line is the same it will ignore it. You can specify the lines the agent uses to fingerprint the file by adding file_fingerprint_lines configuration,

for example to use lines 1 - 20 to identify the file:

AWSEBAutoScalingGroup:
    Metadata:
      "AWS::CloudFormation::Init":
        CWLogsAgentConfigSetup:
          files:
            ## any .conf file put into /tmp/cwlogs/conf.d will be added to the cwlogs config (see cwl-agent.config)
            "/tmp/cwlogs/conf.d/apache-access.conf":
              content : |
            [    apache-access_log]
                file = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "LogFile"]}`
                log_group_name = `{ "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup" }`
                file_fingerprint_lines = 1-20
                log_stream_name = {instance_id}
                datetime_format = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "TimestampFormat"]}`
              mode  : "000400"
              owner : root
              group : root
10
  • Do you happen know if there's a workaround for this?
    – Ian
    Sep 22, 2016 at 14:47
  • I've since found that this is incorrect, the awslogs agent which pushes the log file events to CloudWatch will use the file with the most current last modified time, I'll edit the answer
    – Bacon
    Oct 20, 2016 at 14:27
  • Do you mean that logging will now continue after a deployment?
    – Ian
    Nov 15, 2016 at 19:14
  • 1
    Yes, it will continue
    – Bacon
    Nov 15, 2016 at 22:00
  • 2
    I'm finding it's not the case for me. I get logs from my initial deployment, but not from subsequent updates. Any advice? Dec 6, 2016 at 21:06
3

I create Dockerrun file for each ElasticBeanstalk application. https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html

There you can specify a logging configuration:

           "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "${you-log-group-name}",
                    "awslogs-region": "${region}",
                    "awslogs-stream-prefix": "${serviceName}"
                }
            }

This sets up Docker's logging driver configuration more details: https://docs.docker.com/config/containers/logging/configure/

2
  • 1
    This configuration is only available for multiple docker container at the moment
    – Baboo
    Jun 26, 2019 at 8:50
  • This is now possible in Amazon Linux 2's docker platform using docker-compose.yaml: ``` version: '3.8' services: your-service: logging: driver: awslogs options: awslogs-group: "${your-log-group-name}" awslogs-region: "us-west-2" ``` Sep 7, 2021 at 17:57
0

This is now possible in Amazon Linux 2's docker platform using docker-compose.yaml:

version: '3.8'
services:
  your-service:
    logging:
      driver: awslogs
      options:
        awslogs-group: "${your-log-group-name}"
        awslogs-region: "us-west-2"

See here for more options: https://docs.docker.com/config/containers/logging/awslogs/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.