30

I have a CSV file with the following columns: id, fname, telephone, lname, address.

I have a Person class with id, fname and lname data members. I want to map only these columns to Person object from a CSV file and discard telephone and address columns. How can I do this? The solution must scale as more columns are added in future. And should work regardless of column position.

In an ideal solution user will only specify columns to read and it should just work.

10 Answers 10

27

You can use HeaderColumnNameTranslateMappingStrategy. Lets assume your CSV has the following columns: Id, Fname, Telephone, Lname, Address for the sake of simplicity.

CsvToBean<Person> csvToBean = new CsvToBean<Person>();

Map<String, String> columnMapping = new HashMap<String, String>();
columnMapping.put("Id", "id");
columnMapping.put("Fname", "fname");
columnMapping.put("Lname", "lname");

HeaderColumnNameTranslateMappingStrategy<Person> strategy = new HeaderColumnNameTranslateMappingStrategy<Person>();
strategy.setType(Person.class);
strategy.setColumnMapping(columnMapping);

List<Person> list = null;
CSVReader reader = new CSVReader(new InputStreamReader(ClassLoader.getSystemResourceAsStream("test.csv")));
list = csvToBean.parse(strategy, reader);

The columnMapping will map the columns with your Person object.

2
  • 3
    Would it be possible to map a String in the csv "2015-01-02 23:59:50" to a joda DateTime object within a bean?
    – root
    Oct 23, 2015 at 13:47
  • 3
    This method is outdated so I wrote new answer which shows how to use BeanBuilder stackoverflow.com/a/48227474/1238944
    – agilob
    Jan 12, 2018 at 13:42
9

Recent versions of OpenCSV deprecate the method parse(X, Y) and it's recommenced to use BeanBuilder instead, so the top answer is out of date.

try {
    CsvToBeanBuilder<PersonCSV> beanBuilder = new CsvToBeanBuilder<>(new InputStreamReader(new FileInputStream("your.csv")));

    beanBuilder.withType(PersonCSV.class);
    // build methods returns a list of Beans
    beanBuilder.build().parse().forEach(e -> log.error(e.toString()));

} catch (FileNotFoundException e) {
    log.error(e.getMessage(), e);
}

This methods allows you to clean up the code and remove MappingStrategy (you can still use it if you like spaghetti), so you can annotate your CSV class as follows:

@CsvDate("dd/MM/yyyy hh:mm:ss")
@CsvBindByName(column = "Time Born", required = true)
private Date birthDate;
2
  • Getting a java.lang.NoSuchMethodError: org.apache.commons.beanutils.ConvertUtilsBean.register(ZZI)V at the beanBuilder.build() line. Where am i going wrong?
    – Shubham
    May 31, 2021 at 14:26
  • 3
    How to ignore unmapped CSV columns? I mean if in CSV there are 10 columns and I want to read only 5 (these 5 are there in bean) of them then? Currently it is throwing exception Number of data fields does not match number of headers. Jul 2, 2021 at 12:28
4

Use uniVocity-parsers and be done with it. It doesn't matter how the columns are organized in the input CSV, only the ones you need will be parsed.

If writing, the columns you have in class will be written to the correct columns, while the others will be empty.

Here's a class with some examples:

class TestBean {

    // if the value parsed in the quantity column is "?" or "-", it will be replaced by null.
    @NullString(nulls = { "?", "-" })
    // if a value resolves to null, it will be converted to the String "0".
    @Parsed(defaultNullRead = "0")
    private Integer quantity;   // The attribute type defines which conversion will be executed when processing the value.

    @Trim
    @LowerCase
    // the value for the comments attribute is in the column at index 4 (0 is the first column, so this means fifth column in the file)
    @Parsed(index = 4)
    private String comments;

    // you can also explicitly give the name of a column in the file.
    @Parsed(field = "amount")
    private BigDecimal amount;

    @Trim
    @LowerCase
    // values "no", "n" and "null" will be converted to false; values "yes" and "y" will be converted to true
    @BooleanString(falseStrings = { "no", "n", "null" }, trueStrings = { "yes", "y" })
    @Parsed
    private Boolean pending;
}

Here's how to get a list of TestBean

BeanListProcessor<TestBean> rowProcessor = new BeanListProcessor<TestBean>(TestBean.class);

CsvParserSettings parserSettings = new CsvParserSettings();
parserSettings.setRowProcessor(rowProcessor);
parserSettings.setHeaderExtractionEnabled(true);

CsvParser parser = new CsvParser(parserSettings);
parser.parse(getReader("/examples/bean_test.csv"));

List<TestBean> beans = rowProcessor.getBeans();

Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).

2
  • What if we even don't need empty column. We just want to remove unwanted columns? Is it possible with univocity?
    – Piyush N
    Feb 19, 2020 at 16:23
  • use settings.selectFields or settings.selectIndexes to choose what columns to parse. Feb 20, 2020 at 6:26
4

I can't speak for opencsv, but this is easily achievable using Super CSV, which has two different readers that support partial reading (ignoring columns), as well as reading into a Javabean. CsvDozerBeanReader is even capable of deep and index-based mapping, so you can map to nested fields.

We (the Super CSV team) have just released version 2.0.0, which is available from Maven central or SourceForge.

Update

Here's an example (based on the test in the GitHub project you've created), that uses Super CSV instead of opencsv. Note the CSV preferences needed the surroundingSpacesNeedQuotes flag enabled as your example CSV file isn't valid (it has spaces between the fields - spaces are considered part of the data in CSV).

ICsvBeanReader beanReader = null;
try {
    beanReader = new CsvBeanReader(
            new InputStreamReader(
                    ClassLoader.getSystemResourceAsStream("test.csv")),
            new CsvPreference.Builder(CsvPreference.STANDARD_PREFERENCE)
                    .surroundingSpacesNeedQuotes(true).build());

    List<String> columnsToMap = Arrays.asList("fname", "telephone", "id");

    // read the CSV header (and set any unwanted columns to null)
    String[] header = beanReader.getHeader(true);
    for (int i = 0; i < header.length; i++) {
        if (!columnsToMap.contains(header[i])) {
            header[i] = null;
        }
    }

    Person person;
    while ((person = beanReader.read(Person.class, header)) != null) {
        System.out.println(person);
    }

} finally {
    beanReader.close();
}
1

Here is a nice way to do use OpenCSV to do the mapping to POJO generically:

protected <T> List<T> mapToCSV(String csvContent, Class<T> mapToClass) {
    CsvToBean<T> csvToBean = new CsvToBean<T>();

    Map<String, String> columnMapping = new HashMap<>();
    Arrays.stream(mapToClass.getDeclaredFields()).forEach(field -> {
        columnMapping.put(field.getName(), field.getName()); 
    });

    HeaderColumnNameTranslateMappingStrategy<T> strategy = new HeaderColumnNameTranslateMappingStrategy<T>();
    strategy.setType(mapToClass);
    strategy.setColumnMapping(columnMapping);

    CSVReader reader = new CSVReader(new StringReader(csvContent));
    return csvToBean.parse(strategy, reader);
}


public static class MyPojo {
    private String foo, bar;

    public void setFoo(String foo) {
        this.foo = foo;
    }

    public void setBar(String bar) {
        this.bar = bar;
    }
}

Then from your test you can use:

List<MyPojo> list = mapToCSV(csvContent, MyPojo.class);
1

Use @CsvIgnore annotation above the field which need not to be written to csv file.

1

With opencsv, you can create a generic function like :

public static <T> void csvWriterUtil(Class<T> beanClass, List<T> data, String outputFile, String[] columnMapping) {
    try {
        Writer writer = new BufferedWriter(new FileWriter(outputFile));
        ColumnPositionMappingStrategy<T> strategy = new ColumnPositionMappingStrategy<>();
        strategy.setType(beanClass);
        strategy.setColumnMapping(columnMapping);
        StatefulBeanToCsv<T> statefulBeanToCsv =new StatefulBeanToCsvBuilder<T>(writer)
                .withMappingStrategy(strategy)
                .build();
        writer.write(String.join(",",columnMapping)+"\n");
        statefulBeanToCsv.write(data);
        writer.close();
    } catch (IOException e) {
        e.printStackTrace();
    } catch (CsvRequiredFieldEmptyException e) {
        e.printStackTrace();
    } catch (CsvDataTypeMismatchException e) {
        e.printStackTrace();
    }
}

Here you can pass only required columns through columnMapping parameter.

Code example is available in https://github.com/soumya-kole/JavaUtils/tree/master/CsvUtil

0

Have a look at jcsvdao, https://github.com/eric-mckinley/jcsvdao/ , uses hibernate style mapping files and can handle 1to1 and 1toMany relations. Good if you dont own the csv files as has flexible matching strategies.

1
  • Welcome to SO. This would be a much better answer if you included some code here, instead of behind a link.
    – Teepeemm
    Feb 25, 2016 at 22:49
0

jcvsdao example usage

Sample User CSV file

Username, Email, Registration Date, Age, Premium User
Jimmy, [email protected], 04-05-2016, 15, Yes, M
Bob, [email protected], 15-01-2012, 32, No, M
Alice, [email protected], 22-09-2011, 24, No, F
Mike, [email protected], 11-03-2012, 18, Yes, M
Helen, [email protected], 02-12-2013, 22, Yes, F
Tom, [email protected], 08-11-2015, 45, No, M

Create A CsvDao

CSVDaoFactory factory = new CSVDaoFactory("/csv-config.xml");
CSVDao dao = new CSVDao(factory);
List<UserDetail> users = dao.find(UserDetail.class);

csv-config.xml

<CSVConfig>
    <mappingFiles fileType="resource">
        <mappingFile>/example01/mapping/UserDetail.csv.xml</mappingFile>
    </mappingFiles>
</CSVConfig>

UserDetail.csv.xml

<CSVMapping className="org.jcsvdao.examples.example01.model.UserDetail" csvFile="csv-examples/example01/users.txt" delimiter="," ignoreFirstLine="true">
    <matchAll/>
    <properties>
        <property index="0" property="username" primaryKey="true"/>
        <property index="1" property="email"/>
        <property index="2" property="registrationDate" converter="myDateConverter"/>
        <property index="3" property="age"/>
        <property index="4" property="premiumUser" converter="yesNoConverter"/>
        <property index="5" property="gender" converter="myGenderConverter"/>
    </properties>
    <converters>
        <dateConverter converterName="myDateConverter" format="dd-MM-yyyy"/>
        <booleanConverter converterName="yesNoConverter" positive="Yes" negative="No"/>
        <customConverter converterName="myGenderConverter" converterClass="org.jcsvdao.examples.example01.converter.GenderCustomerConverter"/>
    </converters>
</CSVMapping>
1
  • Can do the same with Apache MetaModel
    – djangofan
    Mar 4, 2019 at 1:22
0

SimpleFlatMapper can do that easily see Getting Started csv using the headers of the csv or by manually specifying which columns map to what property.

CsvParser
    .mapTo(MyObject.class)
    .forEach(file, System.out::println);

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.