-2

I have been often asked these question:

I have the data-sets which contain the following attributes:

     Date_Day    Geography    Avg_Temp    Max_Temp    Min_Temp
      1/01/2018   Delhi          32(C)     35(C)        28(C)
      2/01/2018   Delhi          33(C)     34(C)        29(C)

There are 20 cities and their per day min,max, avg temperatures are given.

The question is:

How can we predict when there will be next heat waves per city is going to occur in coming 1 year?

We can have assumption as required and add any variables.

I thought of approaching these problem with time series forecasting but then I have this challenge that I have to forecast too many data for 1 year/day. And also forecasting will not be good in this case since forecasting period is very long.

Is there any approach which is feasible to solve such problems.

Any help would be appreciated.

1
  • 1
    You are trying to solve the same problems as thousands (or more) of meteorologists all over the world. The thing is that no one so far has managed to come up with a model good enough that we'd have forecasts in spring about the heatwaves in summer. Now, if you just want any model, regardless of quality, there are plenty of regression-based methods that might be suitable to produce some result. The users of the data science stackexchange site might have better recommendations. Sep 24, 2019 at 14:30

2 Answers 2

3

To be a serious research, you may need a lot more information than what you have. And you may need get some ideas from Geographer about what impact the heat wave happening. You even need to use some other cities or areas' attributes to predict each city. The other cities could be from very far away countries. The weather impact facts could come from north pole, south pole, ocean, etc. Of course a lot more data. We don't know what is the relation between impact facts and the heat wave. But that is what we want machine learning to learn for us.

If you just want to train a model and learn to write a machine learning algorithm. It won't be too hard. You can try any RNNs. You can try use every 10 days as sequence to predict the 11th day's temperature. Each day in the 10 days has four or five attributes you listed above. You can train 3 models to predict max min and average. I don't know what you meant actual heat wave. But I think it is easy to define it based on max,min and average. If you have many years of data, you may get some look good results. For instance the heat wave always happen during the summer time.

Again, I don't think it will be helpful to a geography scientific research. For learning machine learning it is fine.

2
  • Make sense. But even RNN and LSTM would suffer longer term dependency if extended for year. Correct me if I am wrong. The more you extend the prediction the more noise you include. Shorter the prediction period, more accurate the result will be. Sep 24, 2019 at 19:26
  • If you only use one year, you will be getting over fitting right away. As every year must be different. Some years getting warm earlier, some years are later. System may find some pattern, for instance, if the year gets warmed up earlier, more likely to have a heat wave in July. Or if the lowest temperature in winter is not so low, next year more like to have heat wave. Something like that. I am not a geographer, I am just guessing. In summary different years are totally different, they are not noise. You may get more truth than noise.
    – George Yu
    Sep 24, 2019 at 20:01
2

The atmosphere is too chaotic to be modeled by simple statistical models!

As an atmospheric scientist, I can tell you in confidence that there is no way you can make reliable weather predictions for the next year based on a purely statistical model, especially in a highly localized area like a city.

You can build a statistical model to understand what events or parameters might be related to extreme weather events such as ENSO, location of high/low-pressure centers, etc., but even if your model could technically make predictions its predictions would be useless because you wouldn't know what will be the values of predictors in your model. Besides, even if you could precisely predict the predictor variables (which is very very unlikely) your statistical model would still likely fail in most cases. You can test this by splitting a past weather data, such as ERA5, to train/test parts to see if you can predict an existing heat wave using the predictor variables over a city. I would be surprised if your model will be more successful than a random guess. However, you could get some meaningful results if you take an average over a much larger area than a city, such as a country like France, and over a longer period of time like a month or the entire season, provided that you already know precisely the state of the atmosphere for the period of prediction.

As an example, such a model might give you an idea of how many heatwaves you might expect to find in your data over southern Europe for the entire 2004 summer. Still, such an analysis wouldn't be useful other than theoretical reasons or climate change perspective since you still wouldn't know the values of predictors for a future time if you stick with a statistical model.

That being said, there are physically based weather/climate models that can be used to predict the future. For instance, WRF is a physically-based (not statistical) atmospheric model that is used to forecast the weather for the next few days with a very high temporal and spatial resolution. It can also be used as a climate model to make climate projections that are only meaningful over like a decade long average and relatively larger area than a city.

If you feel like I sound too discouraging then that's good! Because I am indeed trying to discourage you by all means from trying to predict the future heatwaves in a city using a purely statistical model. Unless you would like to learn from your own mistakes and have days of spare time to spend for only educational purposes, but not for actually achieving real-life applicable results.

Not the answer you're looking for? Browse other questions tagged or ask your own question.