3

Read this on apache documentation:

InputSplit represents the data to be processed by an individual Mapper.

Typically, it presents a byte-oriented view on the input and is the responsibility of RecordReader of the job to process this and present a record-oriented view.

Link - https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/mapred/InputSplit.html

Can somebody explain the difference between byte-oriented view and record-oriented view?

1 Answer 1

4

HDFS splits its blocks (byte-oriented view) so that each block is less than or equal to the block size configured. So it is considered to be not following a logical split. Means a part of last record may reside in one block and rest of it is in another block. This seems correct for storage. But At processing time, the partial records in a block cannot be processed as it is. So the record-oriented view comes into place. This will ensure to get the remaining part of the last record in the other block to make it a block of complete records. This is called input-split (record oriented view).

1
  • I really love this answer! <3 I've struggled to understand what byte-oriented view and record oriented view were. Thanks for the clear explanation! Jul 24, 2021 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.