0

I have a query to return various figures from a combination of tables. I am using jooq to run this query.

final SiteSalesFigures siteSalesFigures =
dsl.select(
      countDistinct(LINE.TRANSACTION_ID).as("transactionCount"),
      sum(LINE.PROFIT).as("totalProfit"),
      sum(LINE.TOTAL).as("totalSalesAmount"),
      sum(LINE.QUANTITY).as("totalItemsSold"),
      sum(LINE.PROFIT).divide(sum(LINE.TOTAL)).multiply(100).as("profitMarginPercentage"),
      avg(TRANSACTIONS.NO_OF_ITEMS).as("averageItemsPerTransaction"),
      sum(LINE.TOTAL).divide(countDistinct(LINE.TRANSACTION_ID)).as("averageSalesTotalPerTransaction"),
      sum(LINE.PROFIT).divide(countDistinct(LINE.TRANSACTION_ID)).as("averageProfitTotalPerTransaction"))
    .from(TRANSACTIONS)
    .join(LINE).on(TRANSACTIONS.TRANSACTION_ID.equal(LINE.TRANSACTION_ID))
    .leftJoin(ITEM).on(LINE.ITEM_ID.equal(ITEM.ITEM_CODE))
    .where(TRANSACTIONS.SITE_ID.equal(siteId))
    .and(TRANSACTIONS.NO_OF_LINES.greaterThan(0))
    .and(TRANSACTIONS.START_TIME
      .between(new Timestamp(reportStartDate.toInstant().toEpochMilli()))
      .and(new Timestamp(reportEndDate.toInstant().toEpochMilli())))
    .and(TRANSACTIONS.TRANSACTION_TYPE_ID.notEqual(cancelledSaleID))
    .fetchOneInto(SiteSalesFigures.class);

The averageItemsPerTransaction is proving to be the issue. I completely understand why it doesn't work, but I'm not sure how I can make it work. Unfortunately the join is necessary due to exclusions which make use of the Line table.

If a transaction has 3 lines, then the transaction details (which includes no_of_items) get replicated three times, which is resulting in incorrect values.

I know the correct values as I have ran the average(no_of_items) query on just the transactions table.

Here is what the table looks like for just two transactions (hiding columns not needed for this example):

          **transaction_id**       **no_of_lines    no_of_items**
8abf1720-51f6-a1bf-4714-004b644cb99f --- 2         --- 2
8abf1720-51f6-a1bf-4714-004b644cb99f --- 2         --- 2
d239feab-38ea-7c8a-4814-7d5a38f74949 --- 3         --- 4
d239feab-38ea-7c8a-4814-7d5a38f74949 --- 3         --- 4
d239feab-38ea-7c8a-4814-7d5a38f74949 --- 3         --- 4

You'll notice the number of lines doesn't always equal the number of items (a line can have one item scanned twice for example)

Does anybody have a solution?

2
  • What MySQL version are you using? 8 or less than 8?
    – Lukas Eder
    Mar 15, 2019 at 8:11
  • @LukasEder I am using MySQL version 8.0.13! Mar 15, 2019 at 8:23

2 Answers 2

1

Solution using 2 queries

One obvious solution would be to run two queries to get these results. The first query would be the one you already have (but without the averages) and the second query would only calculate the averages:

final SiteSalesFigures siteSalesFigures =
dsl.select(
      avg(TRANSACTIONS.NO_OF_ITEMS).as("averageSalesTotalPerTransaction"),
      avg(TRANSACTIONS.PRICE).as("averageSalesTotalPerTransaction"),
      avg(TRANSACTIONS.PROFIT).as("averageProfitTotalPerTransaction"))
    .from(TRANSACTIONS)
    .where(TRANSACTIONS.SITE_ID.equal(siteId))
    .and(TRANSACTIONS.NO_OF_LINES.greaterThan(0))
    .and(TRANSACTIONS.START_TIME
      .between(new Timestamp(reportStartDate.toInstant().toEpochMilli()))
      .and(new Timestamp(reportEndDate.toInstant().toEpochMilli())))
    .and(TRANSACTIONS.TRANSACTION_TYPE_ID.notEqual(cancelledSaleID))
    .fetchOneInto(SiteSalesFigures.class);

This might be quite slower than doing it all in one go, depending on the size of your TRANSACTIONS table.

Solution using a weighted average

Because your join produces repeated TRANSACTIONS rows, you have to calculate a weighted average, not an ordinary average. Given your example, if your TRANSACTIONS row gets repeated 3 times, then you have to divide that particular transaction's contribution by 3. This would normally be quite complicated, but given that you've already denormalised your schema by pre-counting the NO_OF_ITEMS per transaction, you're lucky. If you didn't have this column, you'd have to pre-calculate it in a derived table.

In SQL / jOOQ:

final SiteSalesFigures siteSalesFigures =
dsl.select(
      ...
      count() 
        .divide(countDistinct(TRANSACTIONS.TRANSACTION_ID)).as("averageSalesTotalPerTransaction"),
      sum(TRANSACTIONS.PRICE.divide(TRANSACTIONS.NO_OF_ITEMS))
        .divide(countDistinct(TRANSACTIONS.TRANSACTION_ID)).as("averageSalesTotalPerTransaction"),
      sum(TRANSACTIONS.PROFIT.divide(TRANSACTIONS.NO_OF_ITEMS))
        .divide(countDistinct(TRANSACTIONS.TRANSACTION_ID)).as("averageProfitTotalPerTransaction"))
    .from(TRANSACTIONS)
    .join(...)
    ...
    .fetchOneInto(SiteSalesFigures.class);

Depending on your data types, you may need to cast to DOUBLE or NUMBER.

I've blogged about calculating weighted averages in SQL more in detail here.

5
  • Unfortunately Lukas, neither solution will work. The first solution, will include lines which I'm trying to exclude (the exclusion is based on the line department, found in the LINE table). The second solution doesn't work either using the example table I've now added to my original question Mar 15, 2019 at 10:11
  • @androidUser1993: I'd say these solutions work perfectly fine for the question you've presented. And the general idea will still apply whatever adaptation to your question you're planning to do. I'm happy to update my answer once you've updated your question.
    – Lukas Eder
    Mar 15, 2019 at 10:17
  • @androidUser1993: Well, this doesn't really change the semantics of the question all that much, just requires adaptation of the expressions. Use NUMBER_OF_LINES in my query rather than NUMBER_OF_ITEMS, I guess? Given that one LINE can only have one ITEM_ID, this seems to be a to-one relationship.
    – Lukas Eder
    Mar 15, 2019 at 10:41
  • I've further updated the question with how I have solved the averageSalesTotalPerTransaction and averageProfitTotalPerTransaction, the only issue I'm having now is the averageItemsPerTransaction (this was mis-copied in the original question in terms of name). For the example table I included I would expect this value to be 3 Mar 15, 2019 at 10:49
  • 1
    Again, I will post an easy solution... it turns out the answers were in front of me all along. I appreciate your help Lukas Mar 15, 2019 at 10:51
0

The solution was in front of me all along, I can just use the values that were working to get the values I needed:

final SiteSalesFigures siteSalesFigures =
dsl.select(
      countDistinct(LINE.TRANSACTION_ID).as("transactionCount"),
      sum(LINE.PROFIT).as("totalProfit"),
      sum(LINE.TOTAL).as("totalSalesAmount"),
      sum(LINE.QUANTITY).as("totalItemsSold"),
      sum(LINE.PROFIT).divide(sum(LINE.TOTAL)).multiply(100).as("profitMarginPercentage"),
      sum(LINE.QUANTITY).divide(countDistinct(LINE.TRANSACTION_ID)).as("averageItemsPerTransaction"),
      sum(LINE.TOTAL).divide(countDistinct(LINE.TRANSACTION_ID)).as("averageSalesTotalPerTransaction"),
      sum(LINE.PROFIT).divide(countDistinct(LINE.TRANSACTION_ID)).as("averageProfitTotalPerTransaction"))
    .from(TRANSACTIONS)
    .join(LINE).on(TRANSACTIONS.TRANSACTION_ID.equal(LINE.TRANSACTION_ID))
    .leftJoin(ITEM).on(LINE.ITEM_ID.equal(ITEM.ITEM_CODE))
    .where(TRANSACTIONS.SITE_ID.equal(siteId))
    .and(TRANSACTIONS.NO_OF_LINES.greaterThan(0))
    .and(TRANSACTIONS.START_TIME
      .between(new Timestamp(reportStartDate.toInstant().toEpochMilli()))
      .and(new Timestamp(reportEndDate.toInstant().toEpochMilli())))
    .and(TRANSACTIONS.TRANSACTION_TYPE_ID.notEqual(cancelledSaleID))
    .fetchOneInto(SiteSalesFigures.class);

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.