7

I want to make a classification model for a sequence of CT images with Keras. my dataset obtains from 50 patients and each patient has 1000 images. For a patient, each image has a meaningful relationship with the previous image. I want to use these meaningful relationships, so I don't know how to build a model for such this problem. can you please give me an idea or examples?

1 Answer 1

14

Your problem is in the context of Sequence Classification. You need to classify sequences of images. In this case, a model is needed to learn two aspects :

  1. Features of the images
  2. Features of the sequence ( temporal or time-related features )

This might sound similar to video classification in which a video is a sequence of several frames. See here.

For extracting features from images:

Most real-world cases use Convolutional Neural Networks. They use layers like Max Pooling and Convolution. They are excellent at extracting features from a 3D input like an image. You can learn more from here.

For handling temporal data:

Here's where you will require an RNN ( Recurrent Neural Network ). LSTM ( Long-Short Term Memory ) cells are popular RNN as they can hold a stronger memory than traditional RNNs.

RNNs preserve the hidden layer activations and use them in processing each and every term in a sequence. Hence, while processing the 2nd image in a sequence, the RNN has knowledge or activations of the 1st image in that same sequence.

You can know more from here.

Finally, we require a fusion of both the above networks:

A CNN-LSTM network uses both convolutional as well as LSTM cells to classify the image sequences.

This is how they look.

You can refer here and here

Hope that helps you. :-)

1
  • 1
    How about loading the data, what can I use instead of train_datagen.flow_from_directory / datagenerator? Jul 17, 2019 at 10:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.