2

I have a basic docker-compose file file for wurstmeister/kafka

I'm trying to configure it to use SASL_PLAIN with SSL However I keep getting this error no matter how many ways I try to specify my jaas file

This is the error I get

[2018-04-11 10:34:34,545] FATAL [KafkaServer id=1001] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
java.lang.IllegalArgumentException: Could not find a 'KafkaServer' or 'sasl_ssl.KafkaServer' entry in the JAAS configuration. System property 'java.security.auth.login.config' is not set

These are the vars I have. Last one is where I specify my jaas file

environment:
  KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
  KAFKA_HOST_NAME: 10.10.10.1
  KAFKA_PORT: 9092
  KAFKA_ADVERTISED_PORT: 9093
  KAFKA_ADVERTISED_HOST_NAME: 10.10.10.1
  KAFKA_LISTENERS: PLAINTEXT://:9092,SASL_SSL://:9093
  KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://10.10.10.1:9092,SASL_SSL://10.10.10.1:9093
  KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_SSL
  KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
  SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
  KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
  KAFKA_SSL_TRUSTSTORE_LOCATION: /kafka.server.truststore.jks
  KAFKA_SSL_TRUSTSTORE_PASSWORD: password
  KAFKA_SSL_KEYSTORE_LOCATION: /kafka.server.keystore.jks
  KAFKA_SSL_KEYSTORE_PASSWORD: password
  KAFKA_SSL_KEY_PASSWORD: password
  KAFKA_OPTS: '-Djava.security.auth.login.config=/path/kafka_server_jaas.conf'

Also when I try to check the docker logs I see

/usr/bin/start-kafka.sh: line 96: KAFKA_OPTS=-Djava.security.auth.login.config: bad substitution

Any help is greatly appreciated!

2 Answers 2

1

equals '=' inside the last value is causing this issue.
KAFKA_OPTS: '-Djava.security.auth.login.config=/path/kafka_server_jaas.conf'
This is what I have got after debugging.

+ for VAR in $(env)
+ [[ KAFKA_OPTS=- 
Djava.security.auth.login.config=/path/kafka_server_jaas.conf =~ ^KAFKA_ ]]
+ [[ ! KAFKA_OPTS=- 
Djava.security.auth.login.config=/path/kafka_server_jaas.conf =~ 
^KAFKA_HOME ]]
++ echo KAFKA_OPTS=- 
Djava.security.auth.login.config=/path/kafka_server_jaas.conf
++ sed -r 's/KAFKA_(.*)=.*/\1/g'
++ tr '[:upper:]' '[:lower:]'
++ tr _ .
+ kafka_name=opts=-djava.security.auth.login.config
++ echo KAFKA_OPTS=- 
Djava.security.auth.login.config=/path/kafka_server_jaas.conf
++ sed -r 's/(.*)=.*/\1/g'
+ env_var=KAFKA_OPTS=-Djava.security.auth.login.config
+ grep -E -q '(^|^#)opts=-djava.security.auth.login.config=' 
/opt/kafka/config/server.properties
start-kafka.sh: line 96: KAFKA_OPTS=-Djava.security.auth.login.config: bad 
substitution

and this is the piece of code that is performing this operation.

88  for VAR in $(env)
89  do
90    if [[ $VAR =~ ^KAFKA_ && ! $VAR =~ ^KAFKA_HOME ]]; then
91      kafka_name=$(echo "$VAR" | sed -r 's/KAFKA_(.*)=.*/\1/g' | tr '[:upper:]' '[:lower:]' | tr _ .)
92      env_var=$(echo "$VAR" | sed -r 's/(.*)=.*/\1/g')
93      if grep -E -q '(^|^#)'"$kafka_name=" "$KAFKA_HOME/config/server.properties"; then
94          sed -r -i 's@(^|^#)('"$kafka_name"')=(.*)@\2='"${!env_var}"'@g' "$KAFKA_HOME/config/server.properties" #note that no config values may contain an '@' char
95      else
96          echo "$kafka_name=${!env_var}" >> "$KAFKA_HOME/config/server.properties"
97      fi
98    fi
99
100    if [[ $VAR =~ ^LOG4J_ ]]; then
101      log4j_name=$(echo "$VAR" | sed -r 's/(LOG4J_.*)=.*/\1/g' | tr '[:upper:]' '[:lower:]' | tr _ .)
102      log4j_env=$(echo "$VAR" | sed -r 's/(.*)=.*/\1/g')
103      if grep -E -q '(^|^#)'"$log4j_name=" "$KAFKA_HOME/config/log4j.properties"; then
104          sed -r -i 's@(^|^#)('"$log4j_name"')=(.*)@\2='"${!log4j_env}"'@g' "$KAFKA_HOME/config/log4j.properties" #note that no config values may contain an'@' char
105      else
106          echo "$log4j_name=${!log4j_env}" >> "$KAFKA_HOME/config/log4j.properties"
107      fi
108    fi
109  done
3
  • 1
    Thanks but doesn't really answer how do I configure this value currently.
    – jimgug
    Apr 11, 2018 at 13:34
  • You can do following steps to overcome this situation: 1: Create a Dockerfile
    – fly2matrix
    Apr 12, 2018 at 5:22
  • You can do following steps to overcome this situation: 1: Create a Dockerfile <br/> 2: Extend the original docker image <br/>3. Create a new script where you append required option in "$KAFKA_HOME/config/server.properties" <br/> 4: Copy new script and change docker entrypoint <br/> 5: Do not set environment variable for KAFKA_OPTS in docker-compose.yml
    – fly2matrix
    Apr 12, 2018 at 5:29
0

Update: They have fixed it and it is merged now!

https://github.com/wurstmeister/kafka-docker/pull/321

There's a bug open now with wurstmeister/kafka but they have gotten back to me with a workaround as follows

I believe his is part of a larger namespace collision problem that affects multiple elements such as Kubernetes deployments etc (as well as other KAFKA_ service settings).

Given you are referencing an external file /kafka_server_jaas.conf, i'm assuming you're OK adding/mounting extra files through; a work-around is to specify a CUSTOM_INIT_SCRIPT environment var, which should be a script similar to:

#!/bin/bash
export KAFKA_OPTS="-Djava.security.auth.login.config=/kafka_server_jaas.conf"
This is executed after the substitution part that is failing.

This could have been done inline, however there is currently a bug in how we process the environment, where we need to specify the input separator to make this work correctly.

Hopefully this works!

1
  • Actually the above doesn't work still in talks about the issue on github.
    – jimgug
    Apr 11, 2018 at 16:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.