I'm trying to build a CNN regression model. The input data is satellite images for 10 years.
The input shape is [10, 256,256, 10]
which represents [Year, Image shape, Image Shape, Channels/Bands]
The output of the model is a number between 0-1 which is the percentage value of the area in the image.
these are the parameters used
CHANNELS=5
BATCH_SIZE=16
INPUT_SHAPE=(10,IMG_SIZE,IMG_SIZE,CHANNELS)
SAMPLES=100
LR=1e-7
EPOCHES=10
I'm using the Conv3D layer as an input layer as it provides the ability to provide volume data to the model, and the Dense layer as output.
Model: sequential_FLATTEN_100_5_16_SGD_1e-07_30_v1
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv3d_3 (Conv3D) (None, 10, 254, 254, 32) 1472
_________________________________________________________________
max_pooling3d_3 (MaxPooling3 (None, 10, 127, 127, 32) 0
_________________________________________________________________
conv3d_4 (Conv3D) (None, 10, 125, 125, 64) 18496
_________________________________________________________________
max_pooling3d_4 (MaxPooling3 (None, 10, 62, 62, 64) 0
_________________________________________________________________
flatten_3 (Flatten) (None, 2460160) 0
_________________________________________________________________
dense_24 (Dense) (None, 256) 629801216
_________________________________________________________________
dense_25 (Dense) (None, 1) 257
=================================================================
Total params: 629,821,441
Trainable params: 629,821,441
Non-trainable params: 0
_________________________________________________________________
This model gives the following score on the Training set:
mean_absolute_error: 0.09013315520024737
mean_squared_error: 0.11449361186977994
explained_variance_score: -0.2407465861253424
r2_score: -0.9382254392540899
And on the Validation set:
mean_absolute_error: 0.1923245317002776
mean_squared_error: 0.2579017795812263
explained_variance_score: -5.067052299015521
r2_score: -5.4177061135705475
I also tried a different model as follow: In this only the first layer is Conv3D and the rest are Dense layers
Model: "sequential_FLATTEN_100_5_16_Adam_1e-07_30_v1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv3d_1 (Conv3D) (None, 4, 250, 250, 32) 54912
_________________________________________________________________
max_pooling3d_1 (MaxPooling3 (None, 1, 83, 83, 32) 0
_________________________________________________________________
flatten_2 (Flatten) (None, 220448) 0
_________________________________________________________________
dense_16 (Dense) (None, 512) 112869888
_________________________________________________________________
dense_17 (Dense) (None, 256) 131328
_________________________________________________________________
dense_18 (Dense) (None, 128) 32896
_________________________________________________________________
dense_19 (Dense) (None, 64) 8256
_________________________________________________________________
dense_20 (Dense) (None, 32) 2080
_________________________________________________________________
dense_21 (Dense) (None, 16) 528
_________________________________________________________________
dense_22 (Dense) (None, 8) 136
_________________________________________________________________
dense_23 (Dense) (None, 1) 9
=================================================================
Total params: 113,100,033
Trainable params: 113,100,033
Non-trainable params: 0
_________________________________________________________________
Which gave me the following score on the Training set:
mean_absolute_error: 0.08475626941395917
mean_squared_error: 0.1637630610914996
explained_variance_score: 0.19943303382780664
r2_score: 0.19214565669613703
And on the Validation set:
mean_absolute_error: 0.15135902269457854
mean_squared_error: 0.2650686092962602
explained_variance_score: -1.7471740284409094
r2_score: -1.7776585146674124
As you can see the model has very low MAE and MSE but the R2-Score and Explained-Variance-Score are also low at the same time.
How can I improve these results?. Also when the sample size is increased the model starts to predict similar values for all inputs.