2

RSA_size(rsa) returns modulus which equals to 256 in my application. and I am encrypting the data with RSA_PKCS1_OAEP_PADDING option, so the max length of the input buffer sent to RSA_public_encrypt() is 256 - 41 = 215

In some case, the length of my input buffer may exceeds the 215 limitation a bit, and I need call RSA_public_encrypt() multitimes.

My question is about the return value of RSA_public_encrypt().

From my test the return value is 256 (equals to RSA_size(rsa) ), and the doc also says:

RSA_public_encrypt() returns the size of the encrypted data (i.e., RSA_size(rsa)).

I just want to make sure that there will only be two possibilities for the return value of RSA_public_encrypt().

-1 (error) or modulus(success), and there is no other possiblity, yes? I am curious because I need dividing the encrypted buffer and call RSA_private_decrypt() for each of the block. If the encrypted buffer of each RSA_public_encrypt is the same, then I don't need store the size for each of them.

2 Answers 2

11

The (positive) return of a RSA_public_encrypt() will always be the same as the RSA_size(rsa) for all current known modes of PKCS#1 encryption.

So in short: your current assumptions are correct.

Practically though: If you have data that is larger than the RSA_size(rsa) and you are splitting it into blocks, you should probably think about encrypting the data with a random symmetric key and encrypting that key with your RSA_public_key(). RSA public key encryption is not meant to be used over larger blocks of data.

The best way to encrypt things that are larger than RSA_size(rsa) - XX (Where XX is dependent on the PKCS#1 mode used):

  • Generate a random IV of 16 bytes (Should be unique)
  • Generate a random key K of 32 bytes (256 bits)
  • Encrypt the data with K and IV using either AES-CBC (think about padding) or AES-CTR into E-DATA
  • Hash the encrypted data E-DATA with SHA-256 (or any suitable hash algorithm for your situation) into hash H
  • Encrypt with RSA Public key the IV, key K and the hash H (IV can be public as well, but this is often easier) into E-RSA
  • Send encrypted data E-DATA and encrypted key-data E-RSA to other side

On the other side:

  • Decrypt E-RSA into IV, K and H (Bail out if it fails)
  • Hash E-DATA and check with H (Bail out if it fails)
  • Decrypt E-DATA with IV and K
  • Done..
7
  • In such case much better is to use proven-to-work and already established scheme, like CMS, which is also supported by OpenSSL. This is reinvention of the wheel. Commented Dec 17, 2012 at 20:37
  • @NickolayO. this is a much better answer than yours, and I got the feeling you downvoted it. If you think the questioner is better suited to CMS, then post it as an answer. Commented Dec 17, 2012 at 21:36
  • I have another opinion on quality of this answer, however I'm also don't think that my answer is ideal. Commented Dec 18, 2012 at 10:22
  • @NickolayO I agree that it is good to use established schemes, such as CMS in most situations. Not everybody can use OpenSSL on both sides. And ipmlementing CMS outside of OpenSSL is quiet a hellish experience, because of the ASN.1 encoding. So yes, I agree. Use CMS in case you can..
    – Paul
    Commented Dec 18, 2012 at 10:38
  • You don't need to implement CMS/OpenPGP yourself, there are a lot of libraries available for almost any language. And, OpenSSL is available for all platforms (as well as GnuPG). Commented Dec 18, 2012 at 11:57
0

It is possible that output of RSA_encrypt() will contain some leading zero bits, and I'm not sure if openssl preserves them. Cutting them will result in output, which is one byte shorter. This can happen rarely.

If your input only abit larger, why not to increase the size of the RSA key? You can have 2536-bit RSA key, or 3072bit, whatever else.

2
  • Likely they use BN_bn2bin which is only defined for positive "big numbers". Increasing the size is an option, but you have to know the maximum size in advance. It's much less flexible and much slower than generating a symmetric key (well, depending on the speed of the secure random generator, of course) - and with slower I mean exponential slow down when related to key size. Commented Dec 17, 2012 at 20:01
  • For sure it is much better to use already-known mixed symmetric/assymetric scheme like OpenPGP or CMS. Commented Dec 17, 2012 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.