1

I'd like to train a convolutional network to solve a multi-class, multi-label problem on image data. Due to the nature of the data, and for reasons I'll spare you, it would be best if I could use a custom R generator function to feed to the fit_generator command, instead of its built-in image_data_generator and flow_images_from_directory commands (which I was successfully able to get working, just not for this particular problem).

Here (https://www.rdocumentation.org/packages/keras/versions/2.2.0/topics/fit_generator) it says that I can do just that, without giving any examples. So I tried the following. Here is an extremely stripped down example of what I'm trying to do (this code is entirely self contained):

library(keras)
library(reticulate)      #for py_iterator function

play.network = keras_model_sequential() %>%
  layer_dense(units = 10, activation = "relu", input_shape = c(10)) %>%
  layer_dense(units = 1, activation = "relu")

play.network %>% compile(
  optimizer = "rmsprop",
  loss = "mse"
)

mikes.custom.generator.function = function()     #generates a 2-list of a random 1 x 10 array, and a scalar
{
  new.func = function()
  {
    arr = array(dim = c(1,10))
    arr[,] = sample(1:10, 10, replace = TRUE)/10
    return(list(arr,runif(1)))
  }
}

mikes.custom.iterator = py_iterator(mikes.custom.generator.function())          #creates a python iterator object

generator_next(mikes.custom.iterator)                 #correctly returns a 2-member list consisting of a 1 x 10 array, and a scalar
generator_next(mikes.custom.iterator)[[1]]            #a 1 x 10 array
generator_next(mikes.custom.iterator)[[2]]            #a scalar

#try to fit with "fit_generator":

play.network %>% fit_generator(                       #FREEZES.
  mikes.custom.iterator,
  steps_per_epoch = 1,
  epochs = 1
)

The thing freezes at training time, without giving me an error message or anything. I also tried it with a custom image data generator for my original problem, same result.

Note that this network trains just fine if I just use fit and input the training data manually:

play.network %>% fit(generator_next(mikes.custom.iterator)[[1]],generator_next(mikes.custom.iterator)[[2]], epochs = 1, batch_size = 1)      

#trains just fine

I think I know the problem, but I don't know the solution. If you ask it for the class of my custom iterator, it gives

class(mikes.custom.iterator)

[1] "python.builtin.iterator"  "rpytools.generator.RGenerator"  "python.builtin.object" 

whereas if I build an iterator using the builtin image_data_generator and flow_images_from_directory commands, it gives

train_datagen <- image_data_generator(rescale = 1/255)

class(train_datagen)

[1] "keras.preprocessing.image.ImageDataGenerator" "keras_preprocessing.image.ImageDataGenerator" "python.builtin.object"

train_generator <- flow_images_from_directory(
  train_dir,
  train_datagen,
  ....
)

class(train_generator)

[1] "python.builtin.iterator" "keras_preprocessing.image.DirectoryIterator" "keras_preprocessing.image.Iterator"        "tensorflow.python.keras.utils.data_utils.Sequence" "python.builtin.object"

So my guess is that train_datagen and/or train_generator have attributes that mikes.custom.iterator does not, and fit_generator is trying to call upon mikes.custom.iterator using functions other than the basic generator_next (which is in theory all it should really need). But I don't know what they may be, or how to build mikes.custom.iterator correctly, even after searching for two hours online.

Help anyone?

2 Answers 2

Reset to default

Trending sort

Trending sort is based off of the default sorting method — by highest score — but it boosts votes that have happened recently, helping to surface more up-to-date answers.

It falls back to sorting by highest score if no posts are trending.

7

In R, you can build an iterator using <<- operator. This is very helpful to build a custom generator function; and it is compatible with Keras' fit_generator() function.

Some minimal example:

# example data
data <- data.frame(
  x = runif(80),
  y = runif(80),
  z = runif(80)
)

# example generator
data_generator <- function(data, x, y, batch_size) {

  # start iterator
  i <- 1

  # return an iterator function
  function() {

    # reset iterator if already seen all data
    if ((i + batch_size - 1) > nrow(data)) i <<- 1

    # iterate current batch's rows
    rows <- c(i:min(i + batch_size - 1, nrow(data)))

    # update to next iteration
    i <<- i + batch_size

    # create container arrays
    x_array <- array(0, dim = c(length(rows), length(x)))
    y_array <- array(0, dim = c(length(rows), length(y)))

    # fill the container
    x_array[1:length(rows), ] <- data[rows, x]
    y_array[1:length(rows), ] <- data[rows, y]

    # return the batch
    list(x_array, y_array)

  }

}

# set-up a generator
gen <- data_generator(
  data = data.matrix(data),
  x = 1:2, # it is flexible, you can use the column numbers,
  y = c("y", "z"), # or the column name
  batch_size = 32
)

From above function, you can simply check the resulting arrays by calling the generator:

gen()

Or you could also test the generator using a simple Keras model:

# import keras
library(keras)

# set up a simple keras model
model <- keras_model_sequential() %>% 
  layer_dense(32, input_shape = c(2)) %>% 
  layer_dense(2)

model %>% compile(
  optimizer = "rmsprop",
  loss = "mse"
)

# fit using generator
model %>% fit_generator(
  generator = gen,
  steps_per_epoch = 100, # will auto-reset after see all sample
  epochs = 10
)

I have to admit that the process is a little bit complex and requires extensive programming. You should check this featured blog post by François Chollet himself, or kerasgenerator package that I develop personally.

2
  • 1
    Excellent answer which helped me heaps! Thank you!
    – JanLauGe
    Oct 20, 2019 at 16:14
  • 1
    Excellent answer indeed! Thanks for this. Do you know if its possible to shuffle the dataset at the start of each epoch using this approach?
    – Miguel
    Nov 4, 2019 at 7:55
3
sampling_generator <- function(X_data, Y_data, batch_size) {
  function() {
    rows <- sample(1:nrow(X_data), batch_size, replace = TRUE)
    list(X_data[rows,], Y_data[rows,])
  }
}

model %>% 
  fit_generator(sampling_generator(X_train, Y_train, batch_size = 128), 
            steps_per_epoch = nrow(X_train) / 128, epochs = 10)

I found this answer in R keras FAQs which seems to work

https://keras.rstudio.com/articles/faq.html#how-can-i-use-keras-with-datasets-that-dont-fit-in-memory

1
  • Sorry I didn't see this answer until now Dave, I've been offline for a while. Much appreciated! Jan 3, 2019 at 2:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.