4

I want to generate a MD5 hash of a text file in ABAP. I have not found any standard solution for generating it for a very big file. Function module CALCULATE_HASH_FOR_CHAR does not meet my requirements because it takes a string as an input parameter. Although it works for smaller files, in case of a for example 4 GB file one cannot construct such a big string.

Does anybody know whether there is a standard piece of coding for doing that (my google efforts did not bring me anything) or maybe someone has an MD5 algorithm in ABAP that calculates the hash of a file?

2
  • You really want to create a hash for a 4 GB text file? In this case you'd probably better call some external tool....
    – vwegert
    Jul 5, 2012 at 20:05
  • @vwegert: Yes, I am sure. Calling for example a system command is out of the question as first, I want it to be platform independent, second, the security team will not allow any system command calls once in production, third, the hash is to be compared with the information from the header and it has to be done in ABAP.
    – Jagger
    Jul 6, 2012 at 0:13

3 Answers 3

6
+100

It looks like the implementation of this algorithm is impossible in ABAP because of the fact that the language does not allow arithmetic overflows during the calculations. This should also answer the question why it has not been implemented so far in SAP system. Either way looks that there is no other way as to call an external tool which of course is, regrettably, hardly platform independent.

EDIT: Ok! So with a great help of René and the code of Fast MD5 Implementation in Java I created the implementation of MD5 algorithm in ABAP . This implementation allows to update the calculated hash with more bytes, which of course might be coming from different sources.

There is no method which takes a file so far but anyways most of the work has been done.

Some simple ABAP Unit tests are included in the code, which also document how to use it.

9
  • Thanks for sharing your work :) I've done a couple of tests and so far it seems to be working correctly. It's really slow though; it takes the system I'm testing with about 6 minutes to hash a 1MB file.
    – René
    Jul 19, 2012 at 21:05
  • Runtime analysis tells me the bitwise left shift method takes up about 65% of the time, and the bitwise add method about 20%. Looks like I'll have to find a way to optimize those methods.
    – René
    Jul 19, 2012 at 21:33
  • I tried with a 2MB file and it took a little bit over 13 minutes. In my case the runtime analysis has shown 75% of the time used by the left shift method.
    – Jagger
    Jul 20, 2012 at 1:35
  • Another test done on some other machine and with a different operating system. File size 1,2 MB, processing time 3:30. With a little tweak in the left_shift_i at the beginning CATCH SYSTEM-EXCEPTIONS arithmetic_errors = 4 OTHERS = 8. returning = value * 2 ** positions. ENDCATCH. IF sy-subrc <> 0. *your code* ENDIF. it ran in 2 minutes and 51 seconds.
    – Jagger
    Jul 20, 2012 at 16:52
  • I simplified the left_shift_i (the final loop for zeroing is redundant) and included it in a new version available on github:gist
    – Jagger
    Jul 20, 2012 at 22:30
2

Perhaps you could read the file in data blocks of a couple megabytes and create a hash list of those using the suggested function. And then create a single top hash using the generated hash list.

7
  • I am afraid this is not possible as the MD5 hash function does not meet such condition f(x1 + x2 + ... +xn) = f(x1) + f(x2) + ... + f(xn).
    – Jagger
    Jul 13, 2012 at 12:28
  • Are you sure about that? Other languages seem to have implementations which allow you to process large files this way. For example in PHP.
    – René
    Jul 13, 2012 at 15:19
  • Yes, I am quite sure. The function from PHP is hash_update. It seems that there is no such function in ABAP. CALCULATE_HASH_FOR_RAW and CALCULATE_HASH_FOR_CHAR do not work this way. They just calculate the hash for given string or a sequence of bytes and you cannot continue by feeding them with more bytes.
    – Jagger
    Jul 13, 2012 at 18:59
  • Well let's see if a bounty will get us a nice answer then :)
    – René
    Jul 16, 2012 at 7:13
  • I am so desperate that I started to write an implementation of the MD5 algorithm with update function. Unfortunatelly I am getting CX_SY_ARITHMETIC_OVERFLOW exception. Does anybody know whether there is an option to switch off the checking if there was an arithmetic overflow in ABAP?
    – Jagger
    Jul 18, 2012 at 15:41
1

The SDN is usually a very good starting point for finding ABAP-related solutions. I was able to find this post: http://scn.sap.com/thread/1483479

The author suggests:

  • Upload the .txt file BUT as BIN.
  • Calculate the hash code using function MD5_CALCULATE_HASH_FOR_RAW

Are you able to get your file in binary format and use MD5_CALCULATE_HASH_FOR_RAW?

Edit: This post even has a more detailed answer using CALCULATE_HASH_FOR_RAW: http://scn.sap.com/thread/1298723

Quote of Shivanand Kalagi's answer:

STR_LEN = XSTRLEN( DATA ).

CALL FUNCTION 'CALCULATE_HASH_FOR_RAW'
  EXPORTING
    ALG            = 'MD5'
    DATA           = DATA
    LENGTH         = STR_LEN
  IMPORTING
    HASH           = L_MD5_HASH.
1
  • I am afraid this will not work as this function module does exactly the same as mentioned CALCULATE_HASH_FOR_CHAR, all data would have to be loaded into an internal table DATA, which does not make any sense for example for 8 GB file.
    – Jagger
    Jul 11, 2012 at 13:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.