0

A Java program connects to the MQ Server using SSL and puts a message to MQ. It was working last week, and now it is not working. Following is the error stack trace on the client side:

com.ibm.mq.MQException: MQJE001: Completion Code '2', Reason '2397'.
        at com.ibm.mq.MQManagedConnectionJ11.<init>(MQManagedConnectionJ11.java:251)
        at com.ibm.mq.MQClientManagedConnectionFactoryJ11._createManagedConnection(MQClientManagedConnectionFactoryJ11.java:449)
        at com.ibm.mq.MQClientManagedConnectionFactoryJ11.createManagedConnection(MQClientManagedConnectionFactoryJ11.java:486)
        at com.ibm.mq.StoredManagedConnection.<init>(StoredManagedConnection.java:97)
        at com.ibm.mq.MQSimpleConnectionManager.allocateConnection(MQSimpleConnectionManager.java:194)
        at com.ibm.mq.MQQueueManagerFactory.obtainBaseMQQueueManager(MQQueueManagerFactory.java:870)
        at com.ibm.mq.MQQueueManagerFactory.procure(MQQueueManagerFactory.java:818)
        at com.ibm.mq.MQQueueManagerFactory.constructQueueManager(MQQueueManagerFactory.java:760)
        at com.ibm.mq.MQQueueManagerFactory.createQueueManager(MQQueueManagerFactory.java:200)
        at com.ibm.mq.MQQueueManager.<init>(MQQueueManager.java:893)
        at MQUtility.main(MQUtility.java:405)
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9204: Connection to host 'server.ip.address.number(1919)' rejected. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLHandshakeException[Remote host terminated the handshake],3=MQServer_Name/server.ip.address.number:1919 (MQServer_Name),4=SSLSocket.startHandshake,5=default]],3=server.ip.address.number(1919),4=,5=RemoteTCPConnection.protocolConnect]
        at com.ibm.mq.jmqi.remote.api.RemoteFAP$Connector.jmqiConnect(RemoteFAP.java:13635)
        at com.ibm.mq.jmqi.remote.api.RemoteFAP$Connector.access$100(RemoteFAP.java:13175)
        at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1449)
        at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1390)
        at com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.jmqiConnect(InterceptedJmqiImpl.java:377)
        at com.ibm.mq.ese.jmqi.ESEJMQI.jmqiConnect(ESEJMQI.java:562)
        at com.ibm.mq.MQSESSION.MQCONNX_j(MQSESSION.java:916)
        at com.ibm.mq.MQManagedConnectionJ11.<init>(MQManagedConnectionJ11.java:236)
        ... 10 more
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLHandshakeException[Remote host terminated the handshake],3=MQServer_Name/server.ip.address.number:1919 (MQServer_Name),4=SSLSocket.startHandshake,5=default]
        at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1493)
        at com.ibm.mq.jmqi.remote.impl.RemoteConnection.connect(RemoteConnection.java:1011)
        at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getNewConnection(RemoteConnectionSpecification.java:688)
        at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSessionFromNewConnection(RemoteConnectionSpecification.java:282)
        at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSession(RemoteConnectionSpecification.java:181)
        at com.ibm.mq.jmqi.remote.impl.RemoteConnectionPool.getSession(RemoteConnectionPool.java:127)
        at com.ibm.mq.jmqi.remote.api.RemoteFAP$Connector.jmqiConnect(RemoteFAP.java:13375)
        ... 17 more
Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
        at sun.security.ssl.SSLSocketImpl.handleEOF(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.decode(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1460)
        at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1452)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1452)
        ... 23 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
        at sun.security.ssl.SSLSocketInputRecord.read(Unknown Source)
        at sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source)
        at sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source)
        at sun.security.ssl.SSLTransport.decode(Unknown Source)
        ... 31 more

Following is the error log extracted from this path C:\ProgramData\IBM\MQ\qmgrs\<mq-manager-name>\errors:

----- amqrmrsa.c : 938 --------------------------------------------------------
9/22/2023 16:56:09 - Process(1532.229) User(SYSTEM) Program(amqrmppa.exe)
                      Host(MQSERVER_NAME) Installation(Installation1)
                      VRMF(9.0.3.0) QMgr(MQManager_Name)
                      Time(2023-09-22T21:56:09.933Z)
                     
AMQ9620: Internal error on call to SSL function on channel '????' to host
'client_host_name (server.ip.number)'.

EXPLANATION:
An error indicating a software problem was returned from a function which is
used to provide SSL or TLS support. The error code returned was '14'. The
function call was 'gsk_secure_soc_init'. 

The channel is '????'; in some cases its name cannot be determined and so is
shown as '????'. The channel did not start. 

The remote host name is 'client_host_name (server.ip.number)'.
ACTION:
Collect the items listed in the 'Problem determination' section of the System
Administration manual and use either the MQ Support site:
http://www.ibm.com/software/integration/wmq/support/, or IBM Support Assistant
(ISA): http://www.ibm.com/software/support/isa/, to see whether a solution is
already available.  If you are unable to find a match, contact your IBM support
center. 
----- amqccisa.c : 7846 -------------------------------------------------------
9/22/2023 16:56:09 - Process(1532.229) User(SYSTEM) Program(amqrmppa.exe)
                      Host(MQSERVER_NAME) Installation(Installation1)
                      VRMF(9.0.3.0) QMgr(MQManager_Name)
                      Time(2023-09-22T21:56:09.933Z)
                     
AMQ9999: Channel '????' to host 'client_host_name (server.ip.number)' ended abnormally.

EXPLANATION:
The channel program running under process ID 1532(1188) for channel '????'
ended abnormally. The host name is 'client_host_name (server.ip.number)'; in some cases
the host name cannot be determined and so is shown as '????'.
ACTION:
Look at previous error messages for the channel program in the error logs to
determine the cause of the failure. Note that this message can be excluded
completely or suppressed by tuning the "ExcludeMessage" or "SuppressMessage"
attributes under the "QMErrorLog" stanza in qm.ini. Further information can be
found in the System Administration Guide. 
----- amqrmrsa.c : 938 --------------------------------------------------------

Please help me troubleshoot this error. If the certificate is expired, then how I can verify this? I have a key file used on the client which is loaded by the Java program. I am not familiar with the MQ Server setting on the server, but I have access, and if you tell me what to check, I can give it a try.

2 Answers 2

3

This could be a potential case of https://www.ibm.com/support/pages/apar/IT15806 given the version reported in the logs above is MQ 9.0.3 CD. It should be noted that 9.0.3 has been out of support for a very long time, and you are strongly advised to either upgrade to a current CD release, or move to a current LTS release if the CD release cycle is too fast for your organisation to keep up with.

To check whether or not a certificate is expired is fairly simple, load the JKS file that's used in the java client application into IBM Key Management, or use runmqckm / keytool to access the keystore. From there, print out the certificate details and verify the expiry date.

Note that to do the above, you'll need the password to the JKS file, otherwise you won't be able to access it.

If you use a CLI, the specific command you'll need is this: runmqckm -cert -list -db [JKS file name] -pw [password]

That should print out all the certificates currently in the keystore. From there you can then do runmqckm -cert -details -db [JKS file] -pw [password] -label [cert_label] which will show the details of the specific certificate you want. Those details will contain the issue date and the expiry date.

2
  • Thanks for the response. Is this command runmqakm provided part of the binaries of the MQ Server? I see this is a bin folder with a lot of commands. I didn't check this command yet.
    – tarekahf
    Sep 25 at 14:30
  • Thank you ... the certificate was expired. The commands you provided worked successfully. I must start the journey to renew or get a new certificate. I am not sure where to start. I'll contact the general email I have to certificate management.
    – tarekahf
    Sep 25 at 21:32
0

In addition to the answer provided by @root, I wanted to provide this complete guide to check if the certificate is expired, create a new one, and deploy it to the client and the MQ server.

The scenario here is that the Java program is running on a client machine, and the MQ server is installed on another Windows server machine.

The steps:

  • Sart on the client machine and use the Java (JRE/JDK) keytool command. Open the command line.

  • Make sure the relevant Java JDK/JRE version is on the environment PATH variable

> SET PATH=%PATH%;C:\Program Files (x86)\java\jre1.8.0_xxx\bin
  • Check if a certificate is expired, run the following Java keytool command line and check the output:
> keytool -list -v -keystore path-to-file\keystore-file-name.jks
Enter keystore password: <keystore-password>

Check the output to find out if the certificate is expired.

  • Create a new jks key file:
>keytool -genkey -keyalg RSA -alias ibmwebspheremqmytestssl -keystore key-store-file-name.jks -storepass <key-store-password> -validity 400 -keysize 2048
What is your first and last name?
  [Unknown]:  MQ-server-name
What is the name of your organizational unit?
  [Unknown]:  Org Unit Name
What is the name of your organization?
  [Unknown]:  Org Name
What is the name of your City or Locality?
  [Unknown]:  City Name
What is the name of your State or Province?
  [Unknown]:  Province Name
What is the two-letter country code for this unit?
  [Unknown]:  Country
Is CN=xxx, OU=xxx, O=xxx, L=xxx, ST=xxx, C=xxx correct?
  [no]:  yes

Enter key password for <ibmwebspheremqmytestssl>
        (RETURN if same as keystore password): <press ENTER to accept the password specified above>

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore mykeystore.jks -destkeystore keystore-file-name.jks -deststoretype pkcs12".
  • Verify the new keystore file:
> keytool -list -v -keystore path-to-file\keystore-file-name.jks
Enter keystore password: <key-store-password>
  • RDP to the machine where MQ is installed
  • Start CMD and go to the folder where SSL certificate repository files are placed for the MQ Manager in question:
C:>cd C:\ProgramData\IBM\MQ\qmgrs\mq-manager-name\ssl
C:\ProgramData\IBM\MQ\qmgrs\mq-manager-name\ssl>
  • Open MQ Explorer, and get the configured keystore for the Queue Manager. Right-click on the Queue Manager, and select properties/SSL. Check SSL Key repository and Certificate label.

  • Backup and delete the old files related to the SSL configured for the Queue Manager. For example, if the value of SSL Key repository is C:\ProgramData\IBM\MQ\qmgrs\mq-manager-name\ssl\mykeystore, then back up and delete the following files:

mykeystore.jks
mykeystore.kdb
mykeystore.rdb
mykeystore.sth   (the password stash file)
  • Copy the keystore file created above from the client machine to the above-mentioned folder on MQ Server.

  • Create the new .kdb and .rdb files using the runmqckm.exe command from IBM MQ (usually found in C:\Program Files\IBM\MQ\bin):

C:\ProgramData\IBM\MQ\qmgrs\mq-manager-name\ssl> runmqckm -keydb -convert -db mykeystore.jks -new_format kdb
5724-H72 (C) Copyright IBM Corp. 1994, 2017.
A password is required to access the source key database.
Please enter a password: <keystore-password>
  • Create the new stash .sth password file
C:\ProgramData\IBM\MQ\qmgrs\mq-manager-name\ssl> runmqckm -keydb -stashpw -db mykeystore.kdb
5724-H72 (C) Copyright IBM Corp. 1994, 2017.
A password is required to access the source key database.
Please enter a password: <keystore-password>
  • From MQ Explorer, right-click on the Queue Manager, and select Properties/SSL. Verify the keystore configuration and the label name. Click Accept/OK.

  • Right-click again on the Queue Manager, and select Security/Refresh SSL.

  • Now it should be ready, and you can test the Java program from the client machine to connect to MQ using SSL.

  • If you encounter errors, check the logs from the following folder on the MQ Server for the Queue Manager in question for example in C:\ProgramData\IBM\MQ\qmgrs\queue-manager-name\errors.

1
  • 1
    It should be noted for any future viewers that the above configuration uses a self signed certificate for communication between the Java application and the IBM MQ queue manager. If a CA signed certificate is being used, the steps are slightly different. The documentation related to TLS cert management for both MQ clients and queue managers can be found here: ibm.com/docs/en/ibm-mq/…
    – root
    Oct 2 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.