I want to make a classification model for a sequence of CT images with Keras. my dataset obtains from 50 patients and each patient has 1000 images. For a patient, each image has a meaningful relationship with the previous image. I want to use these meaningful relationships, so I don't know how to build a model for such this problem. can you please give me an idea or examples?
1 Answer
Your problem is in the context of Sequence Classification. You need to classify sequences of images. In this case, a model is needed to learn two aspects :
- Features of the images
- Features of the sequence ( temporal or time-related features )
This might sound similar to video classification in which a video is a sequence of several frames. See here.
For extracting features from images:
Most real-world cases use Convolutional Neural Networks. They use layers like Max Pooling and Convolution. They are excellent at extracting features from a 3D input like an image. You can learn more from here.
For handling temporal data:
Here's where you will require an RNN ( Recurrent Neural Network ). LSTM ( Long-Short Term Memory ) cells are popular RNN as they can hold a stronger memory than traditional RNNs.
RNNs preserve the hidden layer activations and use them in processing each and every term in a sequence. Hence, while processing the 2nd image in a sequence, the RNN has knowledge or activations of the 1st image in that same sequence.
You can know more from here.
Finally, we require a fusion of both the above networks:
A CNN-LSTM network uses both convolutional as well as LSTM cells to classify the image sequences.
Hope that helps you. :-)
-
1How about loading the data, what can I use instead of train_datagen.flow_from_directory / datagenerator? Jul 17, 2019 at 10:15