-4

Our data is confidential thus I am creating dummy data here. The number of records is 100 000.

Table1 Form:

+------------------+
| formID  formName |
+------------------+
|   1     student  |
|   2     teacher  |
+------------------+

Table2 Field:

+--------------------+
| fieldID  fieldName |
+--------------------+
|   1      Name      |
|   2      Location  |
+--------------------+

Table3 FormField:

+----------------------------+
|   formID fieldID Value     |
+----------------------------+
|   1      1     studentName |
|   1      2     ahmedabad   |
|   2      1     teacherName |
|   2      2     mumbai      |
+----------------------------+

My query:

select frm.formName, 
(
    select frmfld.Value from FormField frmfld 
    WHERE frmfld.fieldID = 1 AND frmfld.formID = frm.formID
) AS Name,
(
    select frmfld.Value from FormField frmfld 
    WHERE frmfld.fieldID = 2 AND frmfld.formID = frm.formID
) AS Location 
From Form frm

Here, fields may increase dynamically and for that field we are appending new subquery portion every time while creating query using codebase. As this fields grows to 100, execution time for this query grows to 7 to 8mins because of the subqueries and which is not acceptable.

7
  • Provide output of EXPLAIN, provide your MySQL configuration variables, provide your server hardware information. You are using an EAV structure, and judging by what you do - it most likely could have been designed better. Also, what is 1lac? I have never seen that, any chance you can convert that to metric units?
    – Mjh
    Sep 11, 2015 at 12:37
  • You don't need those sub-queries. Simple JOINs will do.
    – JimmyB
    Sep 11, 2015 at 12:38
  • I can't share output of Explain as it is confidential and what I have shared above is dummy. If You have better design for this kind of structure then your suggestion is welcomed. @Mjh: 1 lac means 100k. Sep 11, 2015 at 12:44
  • @HannoBinder : Simple joins are not useful here as I need to convert data from rows to columns. Sep 11, 2015 at 12:48
  • The query you posted can be exactly replicated with JOINs only. Are there any other requirements you didn't mention?
    – JimmyB
    Sep 11, 2015 at 12:56

3 Answers 3

3

My requirement is to support unlimited dynamic fields

This won't work. A fixed number of fields is quite doable, although performance will degrade with each added field, but an unknown, dynamic number is hard to handle.

SELECT frm.formName, 
       frmfld1.Value AS Name,
       frmfld2.Value AS Location 
FROM Form frm
LEFT OUTER JOIN FormField frmfld1 ON frmfld1.formID = frm.formID AND frmfld1.fieldID = 1
LEFT OUTER JOIN FormField frmfld2 ON frmfld2.formID = frm.formID AND frmfld2.fieldID = 2

Make sure you have indexes on Form(formID) and FormField(formID, fieldID).

3
  • I have tested this approach with 30 dynamic fields and 100k records, performance has been improved little bit. Previously it was taking 37sec and now its taking 32sec. What other ways should I use to improve further? Sep 14, 2015 at 11:26
  • If you already have the appropriate indexes I don't think that much more can be done. If you can restrict the result candidates, that should help. Like for instance WHERE frm.formID BETWEEN 123 AND 234.
    – JimmyB
    Sep 14, 2015 at 11:52
  • Usually, I'd say, you let the GUI/client handle the reformatting of (dynamic) rows to columns, because rel DBs and SQL are just not made for that.
    – JimmyB
    Sep 14, 2015 at 11:54
0

The more you normalize your database, the more will be the fetching cost. Meaning that You need to join more tables. So here you have the same problem.

I'd suggest to use the view instead which will help you to some level. But this is not the long term solution. Another options is If you can provide 'where' conditions to you indexed columns. That will really helpful.

0

http://sqlfiddle.com/#!9/013eb2/3

SELECT frm.formName, 
  GROUP_CONCAT(IF(frmfld.fieldID=1,frmfld.Value,NULL)) AS Name,
  GROUP_CONCAT(IF(frmfld.fieldID=2,frmfld.Value,NULL)) AS Location
FROM Form frm
LEFT JOIN FormField frmfld
ON frmfld.formID = frm.formID
GROUP BY frm.formID

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.