0

I will try to be understandable.

I created a DNNClassifier estimator:

classifier = tf.estimator.DNNClassifier(
    feature_columns=feature_column,
    hidden_units=[10, 10],
    n_classes=3)

I trained this model :

classifier.train(input_fn=lambda:train_input_fn(train_features, train_label, 1), steps=1000)

It is here that I have a problem. I want to change a little my evaluate function. I explain :

Target of my program is to know if my model is profitable or not. My features / label are for example :

| 2 | 1 | 6 | / | 2 |

If evaluation is correct I win the value in features[label] (6 for our example). If not, I lost 1.

So, problem is I don't care accuracy because it means nothing for me.

Currently my evaluate function is :

def eval_input_fn(features, labels, batch_size):
"""An input function for evaluation or prediction"""
features=dict(features)
if labels is None:
    # No labels, use only features.
    inputs = features
else:
    inputs = (features, labels)

# Convert the inputs to a Dataset.
dataset = tf.data.Dataset.from_tensor_slices(inputs)

# Batch the examples
assert batch_size is not None, "batch_size must not be None"
dataset = dataset.batch(batch_size)

# Return the dataset.
return dataset

-

eval_result = classifier.evaluate(
    input_fn=lambda:eval_input_fn(test_features, test_label,
                                            1), steps=1000)

and result doesn't suit me :

INFO:tensorflow:Saving dict for global step 1000: accuracy = 0.555, average_loss = 0.96378326, global_step = 1000, loss = 0.96378326

If it is possible I would like at each iteration on my dataset calculate gain or loss if predicition during evaluation is ok or not. I search for hooks but no result.. Or if is there a way to export all good predicitions after evaluate()?

Have you an idea ?

Thanks a lot for your help !

6
  • Is it OK to use predict instead of evaluate for your case? You can then get the prediction first and decide whether you win or loss in your own way.
    – Y. Luo
    May 2, 2018 at 20:39
  • I thought about it but if I haven't the label for each prediction it is a problem for me. I don't know the order of features so I can't match feature and label or I haven't seen how. An idea ? And I am worried about performance of program if I don't use optimized functions. Thanks for your help ! :) May 2, 2018 at 20:45
  • Is there a reason why you don't know the order of features? You don't have to shuffle for evaluation or prediction, right?
    – Y. Luo
    May 2, 2018 at 20:47
  • Yeah I am agree with you but features are result of files concatenation. But if order of features is preserved during predict function it could work. Do you know if order of predicts is the same than order of features injected as input in predict function ? May 2, 2018 at 20:53
  • Yes, it should be the same order if there is nothing extremely special in your code. Try it out yourself.
    – Y. Luo
    May 2, 2018 at 20:59

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.