1

I want to fetch previous date data only from mongodb by using SQL query in my spring boot app code

In mongo db the date is stored in ISOFormat. I need to filter the records only by the date which should be date = previous day's date.

After fetching previous day's data only, I am storing only that data in new dataset.

Simple query used in code is:

  SELECT b._id as _id,a.msisdn as msisdn,
  a.classification as classification, a.event_date_actual as 
  event_date_actual,(IFNULL(a.up_vol_mb,0) + IFNULL(b.up_vol_mb,0)) as 
  up_vol_mb,(IFNULL(a.down_vol_mb,0) + IFNULL(b.down_vol_mb,0)) as 
  down_vol_mb,(IFNULL(a.total_vol_mb,0) + IFNULL(b.total_vol_mb,0)) as 
  total_vol_mb FROM SlaveSchemaTable a LEFT JOIN MasterSchemaTable b ON 
  (a.msisdn = b.msisdn) AND 
  (a.classification = b.classification) AND
  (a.event_date_actual = b.event_date_actual);

Below is the code:

    private boolean writeMongoDBMaster(Dataset<Row> slaveSchemaTable,    
    SparkSession sparkSession) {

    if(slaveSchemaTable.limit(1).count() !=0) {
    StringBuilder joinBuilder = new StringBuilder("SELECT b._id as _id,");
    joinBuilder.append("a.msisdn as msisdn,");
    joinBuilder.append("a.classification as classification,");
    joinBuilder.append("a.event_date_actual as event_date_actual,");
    joinBuilder.append("(IFNULL(a.up_vol_mb,0) + IFNULL(b.up_vol_mb,0)) as 
    up_vol_mb,");
    joinBuilder.append("(IFNULL(a.down_vol_mb,0) + 
    IFNULL(b.down_vol_mb,0)) as down_vol_mb,");
    joinBuilder.append("(IFNULL(a.total_vol_mb,0) + 
    IFNULL(b.total_vol_mb,0)) as total_vol_mb ");
    joinBuilder.append("FROM SlaveSchemaTable a LEFT JOIN 
    MasterSchemaTable b ");
    joinBuilder.append("ON (a.msisdn = b.msisdn) AND ");
    joinBuilder.append("(a.classification = b.classification) AND ");
    joinBuilder.append("(a.event_date_actual = b.event_date_actual)");

    Dataset<Row> masterTable = sparkSession.sql(joinBuilder.toString());
    MongoSpark.save(masterTable.write().mode("append"));** 

Above is the simple query which is fetching data after proper join condition. I want to fetch only data of previous date.

For example: select--------------- where event_date_actual = previous day date. event_date_actual is in ISO format.

2
  • Can you please provide formatted query instead of java code to make query clean Sep 12, 2019 at 6:01
  • HiYogesh!.. I have added the formatted query in the question. Please have a look :)
    – ing
    Sep 12, 2019 at 6:13

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.