64

I have a user class that has 16 attributes, things such as firstname, lastname, dob, username, password etc... These are all stored in a MySQL database and when I want to retrieve users I use a ResultSet. I want to map each of the columns back to the user attributes but the way I am doing it seems terribly inefficient. For example I am doing:

//ResultSet rs;
while(rs.next()) {
   String uid = rs.getString("UserId");
   String fname = rs.getString("FirstName");
   ...
   ...
   ...
   User u = new User(uid,fname,...);
   //ArrayList<User> users 
   users.add(u);
} 

i.e I retrieve all the columns and then create user objects by inserting all the column values into the User constructor.

Does anyone know of a faster, neater, way of doing this?

4
  • what you mean. in efficent ? is it taking too much time
    – Mani
    Commented Feb 22, 2014 at 15:11
  • Check out Spring JDBC template and its bean mappers
    – user330315
    Commented Feb 22, 2014 at 15:17
  • There are a lot of tools that makes this kind of task a lot easier. I think the best ones are sql2o, JDBI and jOOQ
    – aaberg
    Commented Feb 23, 2014 at 10:50
  • Link
    – D3X
    Commented Feb 25, 2014 at 6:11

11 Answers 11

79

If you don't want to use any JPA provider such as OpenJPA or Hibernate, you can just give Apache DbUtils a try.

http://commons.apache.org/proper/commons-dbutils/examples.html

Then your code will look like this:

QueryRunner run = new QueryRunner(dataSource);

// Use the BeanListHandler implementation to convert all
// ResultSet rows into a List of Person JavaBeans.
ResultSetHandler<List<Person>> h = new BeanListHandler<Person>(Person.class);

// Execute the SQL statement and return the results in a List of
// Person objects generated by the BeanListHandler.
List<Person> persons = run.query("SELECT * FROM Person", h);
2
  • 2
    Seems perfect solution for pure jdbc logic without any frameworks. Cool ! +1
    – Vissu
    Commented Sep 24, 2018 at 5:15
  • 1
    DbUtils seems very nice in many ways, but it has two disadvantages: It depends on the java.desktop module, and it doesn't help with the task to store a bean state into a statement for INSERT statements.
    – Lii
    Commented Oct 21, 2020 at 11:50
37

No need of storing resultSet values into String and again setting into POJO class. Instead set at the time you are retrieving.

Or best way switch to ORM tools like hibernate instead of JDBC which maps your POJO object direct to database.

But as of now use this:

List<User> users=new ArrayList<User>();

while(rs.next()) {
   User user = new User();      
   user.setUserId(rs.getString("UserId"));
   user.setFName(rs.getString("FirstName"));
  ...
  ...
  ...


  users.add(user);
} 
3
  • Yes, that would be slightly neater and better in the sense that I would not have to unnecessarily create all the strings, integers etc. (nor use the constructor). Thanks.
    – Quanqai
    Commented Feb 22, 2014 at 15:10
  • 3
    what if result set contains 300 fields? Commented Dec 15, 2021 at 5:24
  • 3
    What if some columns are added in future? Then, changes will be required here as well. Commented Dec 21, 2021 at 18:59
27

Let's assume you want to use core Java, w/o any strategic frameworks. If you can guarantee, that field name of an entity will be equal to the column in database, you can use Reflection API (otherwise create annotation and define mapping name there)

By FieldName

/**

Class<T> clazz - a list of object types you want to be fetched
ResultSet resultSet - pointer to your retrieved results 

*/

    List<Field> fields = Arrays.asList(clazz.getDeclaredFields());
    for(Field field: fields) {
        field.setAccessible(true);
    }

    List<T> list = new ArrayList<>(); 
    while(resultSet.next()) {

        T dto = clazz.getConstructor().newInstance();

        for(Field field: fields) {
            String name = field.getName();

            try{
                String value = resultSet.getString(name);
                field.set(dto, field.getType().getConstructor(String.class).newInstance(value));
            } catch (Exception e) {
                e.printStackTrace();
            }

        }

        list.add(dto);

    }

By annotation

@Retention(RetentionPolicy.RUNTIME)
public @interface Col {

    String name();
}

DTO:

class SomeClass {

   @Col(name = "column_in_db_name")
   private String columnInDbName;

   public SomeClass() {}

   // ..

}

Same, but

    while(resultSet.next()) {

        T dto = clazz.getConstructor().newInstance();

        for(Field field: fields) {
            Col col = field.getAnnotation(Col.class);
            if(col!=null) {
                String name = col.name();
                try{
                    String value = resultSet.getString(name);
                    field.set(dto, field.getType().getConstructor(String.class).newInstance(value));
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }

        list.add(dto);

    }

Thoughts

In fact, iterating over all Fields might seem ineffective, so I would store mapping somewhere, rather than iterating each time. However, if our T is a DTO with only purpose of transferring data and won't contain loads of unnecessary fields, that's ok. In the end it's much better than using boilerplate methods all the way.

Hope this helps someone.

4
  • The pattern is very similar to a case of my own. Just my cents on this pattern. a) You need to check if you can access the field (I override that by settings always the field readable b) You can start checking first your class which might 95% have less fields than those you might get from db. for many reasons a class does not need everything c) Iterating fast to check equality is only feasible by null and equals() d) I recently placed a CacheManager build reading a tutorial and increased the performance of reflection significantly e) You need to cache repeatable query results.
    – hephestos
    Commented Jan 30, 2019 at 15:56
  • I was following this answer. But in my case, all the db fields are not similar to class fields(POJO fields). can you please help how to do in this case Commented Mar 3 at 12:02
  • @PulkitAgrawal you're not required to map directly to your business object with various field types, you can map to String-only DTO, where all fields are String, and then call a factory that will convert it to business object. Or just have String constructor for every type in your class (it is shown). But you should clarify what exactly is "not similar"
    – TEH EMPRAH
    Commented Mar 4 at 9:22
  • @TEHEMPRAH I am trying in the below repo, can you please help me?github.com/Pulkit773/RestAssured/blob/main/src/test/java/com/… Also , I raised a question in stack overflow but no success. stackoverflow.com/questions/78092897/… Commented Mar 4 at 9:44
4

Complete solution using @TEH-EMPRAH ideas and Generic casting from Cast Object to Generic Type for returning

import annotations.Column;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.sql.SQLException;
import java.util.*;

public class ObjectMapper<T> {

    private Class clazz;
    private Map<String, Field> fields = new HashMap<>();
    Map<String, String> errors = new HashMap<>();

    public ObjectMapper(Class clazz) {
        this.clazz = clazz;

        List<Field> fieldList = Arrays.asList(clazz.getDeclaredFields());
        for (Field field : fieldList) {
            Column col = field.getAnnotation(Column.class);
            if (col != null) {
                field.setAccessible(true);
                fields.put(col.name(), field);
            }
        }
    }

    public T map(Map<String, Object> row) throws SQLException {
        try {
            T dto = (T) clazz.getConstructor().newInstance();
            for (Map.Entry<String, Object> entity : row.entrySet()) {
                if (entity.getValue() == null) {
                    continue;  // Don't set DBNULL
                }
                String column = entity.getKey();
                Field field = fields.get(column);
                if (field != null) {
                    field.set(dto, convertInstanceOfObject(entity.getValue()));
                }
            }
            return dto;
        } catch (IllegalAccessException | InstantiationException | NoSuchMethodException | InvocationTargetException e) {
            e.printStackTrace();
            throw new SQLException("Problem with data Mapping. See logs.");
        }
    }

    public List<T> map(List<Map<String, Object>> rows) throws SQLException {
        List<T> list = new LinkedList<>();

        for (Map<String, Object> row : rows) {
            list.add(map(row));
        }

        return list;
    }

    private T convertInstanceOfObject(Object o) {
        try {
            return (T) o;
        } catch (ClassCastException e) {
            return null;
        }
    }
}

and then in terms of how it ties in with the database, I have the following:

// connect to database (autocloses)
try (DataConnection conn = ds1.getConnection()) {

    // fetch rows
    List<Map<String, Object>> rows = conn.nativeSelect("SELECT * FROM products");

    // map rows to class
    ObjectMapper<Product> objectMapper = new ObjectMapper<>(Product.class);
    List<Product> products = objectMapper.map(rows);

    // display the rows
    System.out.println(rows);

    // display it as products
    for (Product prod : products) {
        System.out.println(prod);
    }

} catch (Exception e) {
    e.printStackTrace();
}
3
  • The code convertInstanceOfObject will never throw a ClassCastException so there is no point of trying to catch it. The cast is an unchecked cast. Instead, if the types don't match, a ClassCastException will be thrown when the return value is assigned to a variable by the calling code.
    – Lii
    Commented Oct 21, 2020 at 7:44
  • i've tested your code.. for the column, i dont import from package annotatoin, instead just create an interface class
    – Jasonw
    Commented Jan 30 at 6:24
  • and the type must not be primitive type becuase you are checking on the null... if it is primitive type, for numeric (int,long,double,etc), it will set to 0
    – Jasonw
    Commented Jan 30 at 6:58
1

I would like to hint on q2o. It is a JPA based Java object mapper which helps with many of the tedious SQL and JDBC ResultSet related tasks, but without all the complexity an ORM framework comes with. With its help mapping a ResultSet to an object is as easy as this:

while(rs.next()) {
    users.add(Q2Obj.fromResultSet(rs, User.class));
}

More about q2o can be found here.

1

Thank you @@TEH-EMPRAH. His solution "By FieldName" will be final as:

/**
     * Method help to convert SQL request data to your custom DTO Java class object.   
     * Requirements: fields of your Java class should have Type: String and have the same name as in sql table
     *
     * @param resultSet     - sql-request result
     * @param clazz - Your DTO Class for mapping
     * @return <T> List <T> - List of converted DTO java class objects
     */

    public static <T> List <T> convertSQLResultSetToObject(ResultSet resultSet, Class<T> clazz) throws SQLException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException {

        List<Field> fields = Arrays.asList(clazz.getDeclaredFields());
        for(Field field: fields) {
            field.setAccessible(true);
        }

        List<T> list = new ArrayList<>();
        while(resultSet.next()) {

            T dto = clazz.getConstructor().newInstance();

            for(Field field: fields) {
                String name = field.getName();

                try{
                    String value = resultSet.getString(name);
                    field.set(dto, field.getType().getConstructor(String.class).newInstance(value));
                } catch (Exception e) {
                    e.printStackTrace();
                }

            }

            list.add(dto);

        }
        return list;
    }
0
0

There are answers recommending to use https://commons.apache.org/proper/commons-dbutils/. The default implementation of row processor i.e org.apache.commons.dbutils.BasicRowProcessor in db-utils 1.7 is not thread safe. So, if you are using org.apache.commons.dbutils.QueryRunner::query method in a multi-threaded environment, you should write your custom row processor. It can be done either by implementing org.apache.commons.dbutils.RowProcessor interface or by extending org.apache.commons.dbutils.BasicRowProcessor class. Sample code given below by extending BasicRowProcessor:

class PersonResultSetHandler extends BasicRowProcessor {
    @Override
    public <T> List<T> toBeanList(ResultSet rs, Class<? extends T> type) 
    throws SQLException 
   {
     //Handle the ResultSet and return a List of Person 
     List<Person> personList = ..... 
     return (List<T>) personList;
   }
        
}

Pass the custom row processor to the appropriate org.apache.commons.dbutils.ResultSetHandler implementation. A BeanListHandler has been used in the below code:

QueryRunner qr = new QueryRunner();
List<Person> personList = qr.query(conn, sqlQuery, new BeanListHandler<Person>(Person.class, new PersonResultSetHandler()));                                                                                                                                                 

However, https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-jdbc is another alternative with a cleaner API. Although, I am not sure about the thread safety aspects of it.

-1

using DbUtils...

The only problem I had with that lib was that sometimes you have relationships in your bean classes, DBUtils does not map that. It only maps the properties in the class of the bean, if you have other complex properties (refering other beans due to DB relationship) you'd have to create "indirect setters" as I call, which are setters that put values into those complex properties's properties.

-1
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.json.simple.JSONObject;
import com.google.gson.Gson;

public class ObjectMapper {

//generic method to convert JDBC resultSet into respective DTo class
@SuppressWarnings("unchecked")
public static Object mapValue(List<Map<String, Object>> rows,Class<?> className) throws Exception
{

        List<Object> response=new ArrayList<>(); 
        Gson gson=new Gson();

        for(Map<String, Object> row:rows){
        org.json.simple.JSONObject jsonObject = new JSONObject();
        jsonObject.putAll(row);
        String json=jsonObject.toJSONString();
        Object actualObject=gson.fromJson(json, className);
        response.add(actualObject);
        }
        return response;

    }

    public static void main(String args[]) throws Exception{

        List<Map<String, Object>> rows=new ArrayList<Map<String, Object>>(); 

        //Hardcoded data for testing
        Map<String, Object> row1=new HashMap<String, Object>();
        row1.put("name", "Raja");
        row1.put("age", 22);
        row1.put("location", "India");


        Map<String, Object> row2=new HashMap<String, Object>();
        row2.put("name", "Rani");
        row2.put("age", 20);
        row2.put("location", "India");

        rows.add(row1);
        rows.add(row2);


        @SuppressWarnings("unchecked")
        List<Dto> res=(List<Dto>) mapValue(rows, Dto.class);


    }

    }

    public class Dto {

    private String name;
    private Integer age;
    private String location;

    //getters and setters

    }

Try the above code .This can be used as a generic method to map JDBC result to respective DTO class.

1
  • Mapping to json string, then parsing json string back to object is inefficient.
    – cdalxndr
    Commented Oct 15, 2022 at 12:25
-1

after getting resultset as a map, you can use jackson object mapper to cnvert map to object easly.

1
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Commented Mar 16, 2023 at 7:33
-2

Use Statement Fetch Size , if you are retrieving more number of records. like this.

Statement statement = connection.createStatement();
statement.setFetchSize(1000); 

Apart from that i dont see an issue with the way you are doing in terms of performance

In terms of Neat. Always use seperate method delegate to map the resultset to POJO object. which can be reused later in the same class

like

private User mapResultSet(ResultSet rs){
     User user = new User();
     // Map Results
     return user;
}

If you have the same name for both columnName and object's fieldName , you could also write reflection utility to load the records back to POJO. and use MetaData to read the columnNames . but for small scale projects using reflection is not an problem. but as i said before there is nothing wrong with the way you are doing.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.