6

I have a JCE test that works fine with all Sun JDKs I have tried, but fails with various IBM J9 JDKs (e.g. 1.6.0 build pwi3260sr8-20100409_01(SR8)). The exception below happens when the cipher is initialized in encrypt mode. Why can the IBM JCE not use its own private key? Am I missing something in my code?

  public void testBasicKeyGeneration() throws NoSuchAlgorithmException, 
      NoSuchPaddingException, InvalidKeyException, IllegalBlockSizeException, 
      BadPaddingException, NoSuchProviderException, SignatureException {
      KeyPairGenerator generator = KeyPairGenerator.getInstance( "RSA" );
      generator.initialize( 2048 );
      KeyPair pair = generator.generateKeyPair();

      String data1 = "123456789012345678901234567890123456789012345678901234567890";
      Cipher cipher = Cipher.getInstance( "RSA" );
      cipher.init( Cipher.ENCRYPT_MODE, pair.getPrivate() );
      byte[] encrypted = cipher.doFinal( data1.getBytes() );

      cipher.init( Cipher.DECRYPT_MODE, pair.getPublic() );
      byte[] decrypted = cipher.doFinal( encrypted );
      String data2 = new String( decrypted );
      assertEquals( "en/decryption failed", data1, data2 );
  }

Here is the stack trace:

java.security.InvalidKeyException: Private key cannot be used to encrypt.
at com.ibm.crypto.provider.RSA.engineInit(Unknown Source)
at javax.crypto.Cipher.a(Unknown Source)
at javax.crypto.Cipher.a(Unknown Source)
at javax.crypto.Cipher.init(Unknown Source)
at javax.crypto.Cipher.init(Unknown Source)
at test.Test.testBasicKeyGeneration(LicenseHelperTest.java:56)
3
  • 1
    Just curious - what happens if you switch around and do the encrypt with the public key and the decrypt with the private key? Normally you don't encrypt with your private key because anyone with the public key could decrypt it. It would not surprise me if IBM assumed a policy of encrypting only with a public key, signing with the private key and then embedding that in their code. Long shot but curious what you find.
    – T.Rob
    Feb 2, 2011 at 4:18
  • 1
    Works like a charm when I do what you suggested. I guess my logic got switched because I encrypt licenses and didn't want to put the private key into the shipped software. If you put that in an answer, I can accept it.
    – FelixM
    Feb 2, 2011 at 4:33
  • @Stephen C: Wouldn't that be just as effectively proven by decrypting a message that was encrypted with the public key? (Granted, there may still be pre-defined protocols that do as you're describing...) Feb 2, 2011 at 5:27

6 Answers 6

10

There is a solution, see http://www-01.ibm.com/support/docview.wss?uid=swg1IV18625

with the property

-Dcom.ibm.crypto.provider.DoRSATypeChecking=false

you can use private keys to encrypt data.

5

I don't know this for sure but I believe that the JCE has an embedded policy limiting encryption to the public key and decryption to the private key.

In the example code the encryption was done with the private key. This would require the public key to decrypt, meaning that anyone with the public key could access the encoded data. Although this has it's uses it is not the accepted pattern and the IBM implementation may be "protecting" you from accidentally creating encrypted data that was publicly readable.

The fact that it tested properly when these were reversed tends to confirm my suspicions but I haven't yet found an official document stating as much.

3

IBM insists private keys cannot be used for encryption and public keys cannot be used for decryption, so they either see this artificial restriction as a feature, or someone is seriously confused here.

Here is how I worked around this problem:

RSAPrivateCrtKey privateKey = (RSAPrivateCrtKey) ks.getKey(keyAlias, ksPassword.trim().toCharArray());
RSAPublicKeySpec spec = new RSAPublicKeySpec(privateKey.getModulus(),privateKey.getPrivateExponent());
Key fakePublicKey = KeyFactory.getInstance("RSA").generatePublic(spec);
encryptCipher.init(Cipher.ENCRYPT_MODE, fakePublicKey);

Essentially, I created a public key object with private key's crypto material. You will need to do the reverse, create a private key object with public key's crypto material, to decrypt with public key if you want to avoid the "Public key cannot be used to decrypt" exception.

3
  • 1
    Interesting idea, but the reverse doesn't work for me: ` X509EncodedKeySpec spec=new X509EncodedKeySpec(bout.toByteArray()); KeyFactory factory = KeyFactory.getInstance("RSA"); RSAPublicKey pukey= (RSAPublicKey)factory.generatePublic(spec); RSAPrivateKeySpec rspec = new RSAPrivateKeySpec(pukey.getModulus(),pukey.getPublicExponent()); key=factory.generatePrivate(rspec);` works fine but I get a padding error when decrypting... Dec 15, 2011 at 11:00
  • The above issue by JP is of course resolved by using "RSA/ECB/NoPadding" is that's available. You'll need to handle the padding yourself in that case though, specified in PKCS#1 v1.5 and higher. Use the padding for signature generation, not for encryption. Apr 15, 2020 at 10:49
  • Above answer casts to RSAPrivateCrtKey. Just casting to RSAPrivateKey should be preferred. Above code would needlessly fail if the RSA private key doesn't contain the CRT parameters. Apr 15, 2020 at 10:52
3

I recently ran in to the same problem. This was eventually solved by using the bouncy castle implementation and adding this line to the java.security file

security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider

2

@T.Rob commented that you may have made a mistake in encrypting with the private key. If "everyone" knows the public key, then anyone can decrypt your file. IBM's JCE behaviour is thus protecting people against this mistake.

I can see the logic of that.

However, there may be cases where you really do need to encrypt with the private key; e.g. as part of a protocol that needs to prove that you know the private key corresponding to a published public key.

If this is really what you want to do, you probably need to use a recent Sun JCE implementation (older Sun JCEs didn't implement RSA), or Bouncy Castle.

7
  • As mentioned above: Wouldn't that be just as effectively proven by decrypting a message that was encrypted with the public key? (Granted, there may still be pre-defined protocols that do as you're describing...) Feb 2, 2011 at 5:28
  • 1
    @FelixM - I'd say it is a deliberate feature.
    – Stephen C
    Feb 2, 2011 at 14:15
  • 3
    @Stephen C - I agree that it's deliberate, but it could still be a bug ;)
    – FelixM
    Feb 2, 2011 at 16:33
  • 2
    Given that both IBM's API doc is just a copy of Sun's and both specify that Cipher.init() takes a value of type Key (as opposed to PrivateKey or PublicKey) I'm leaning toward bug as well. If it ain't documented and it doesn't work as specified...
    – T.Rob
    Feb 2, 2011 at 17:42
  • 1
    @FelixM, @T.Rob - send in a bug report then. If IBM agrees it is a bug and fixes it, then you've achieved something. But I doubt that they will.
    – Stephen C
    Feb 2, 2011 at 23:07
2

@Stephen C / @FelixM: IBM seems to be completely clueless about how RSA cryptography works and how it is intended to be used. Basically both operations (encrypt / decrypt) must be available for the public AND private key.

Encrypt with public key is needed to transmit the client-side part of the pre master secret in SSL/TLS handshakes. The server needs to decrypt with its private key. But if they negotiate something like ECDHE_RSA the server needs to SIGN parts of the handshake with the private key - thats encrypt with PrivateKey. Vice versa the client needs to decrypt with the public key from the certificate of the server to verify the hash value of the signature. (proving authenticity of the message)

So if I try to run ECDHE_RSA (server-side) on latest IBM JDK 7 the following happens:

java.security.InvalidKeyException: Private key cannot be used to encrypt.
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
   at java.lang.Thread.run(Thread.java:777)
   at com.ibm.crypto.provider.RSASSL.engineInit(Unknown Source)
   at javax.crypto.Cipher.init(Unknown Source)
   at javax.crypto.Cipher.init(Unknown Source)
   at java.security.Signature$CipherAdapter.engineInitSign(Signature.java:1239)
   at java.security.Signature$Delegate.init(Signature.java:1116)
   at java.security.Signature$Delegate.chooseProvider(Signature.java:1076)
   at java.security.Signature$Delegate.engineInitSign(Signature.java:1140)
   at java.security.Signature.initSign(Signature.java:522)
   at net.vx4.lib.tls.core.TLSSignature.createSignature(TLSSignature.java:120)

As you can see we're using "Signature" and call "initSign", which requires indeed a PrivateKey. This proves IBM being clueless about this fact and obviously they don't even have valid regression tests!

Use another crypto provider and don't believe IBM until they change their mind.

Best regards, Christian

1
  • I think that Cipher should certainly not be used by any implementation of Signature at all. Private keys need extra protection, so the internal implementation should be very different, even if both just perform modular exponentiation. Signature generation is not encryption with a private key.. As IBM seems to use this rather bad practice themselves does however point the blame arrow right back at them; Signature$CipherAdapter should not exist; there should be a separate function for the modular exponentiation instead. Apr 15, 2020 at 10:44

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.