Hello Stackoverflowers,
I'm using azureml and I'm wondering if it is possible to log a confusion matrix of the xgboost model I'm training, together with the other metrics I'm already logging. Here's a sample of the code I'm using:
from azureml.core.model import Model
from azureml.core import Workspace
from azureml.core.experiment import Experiment
from azureml.core.authentication import ServicePrincipalAuthentication
import json
with open('./azureml.config', 'r') as f:
config = json.load(f)
svc_pr = ServicePrincipalAuthentication(
tenant_id=config['tenant_id'],
service_principal_id=config['svc_pr_id'],
service_principal_password=config['svc_pr_password'])
ws = Workspace(workspace_name=config['workspace_name'],
subscription_id=config['subscription_id'],
resource_group=config['resource_group'],
auth=svc_pr)
y_pred = model.predict(dtest)
acc = metrics.accuracy_score(y_test, (y_pred>.5).astype(int))
run.log("accuracy", acc)
f1 = metrics.f1_score(y_test, (y_pred>.5).astype(int), average='binary')
run.log("f1 score", f1)
cmtx = metrics.confusion_matrix(y_test,(y_pred>.5).astype(int))
run.log_confusion_matrix('Confusion matrix', cmtx)
The above code raises this kind of error:
TypeError: Object of type ndarray is not JSON serializable
I already tried to transform the matrix in a simpler one, but another error occurred as before I logged a "manual" version of it (cmtx = [[30000, 50],[40, 2000]]
).
run.log_confusion_matrix('Confusion matrix', [list([int(y) for y in x]) for x in cmtx])
AzureMLException: AzureMLException:
Message: UserError: Resource Conflict: ArtifactId ExperimentRun/dcid.3196bf92-4952-4850-9a8a- c5103b205379/Confusion matrix already exists.
InnerException None
ErrorResponse
{
"error": {
"message": "UserError: Resource Conflict: ArtifactId ExperimentRun/dcid.3196bf92-4952-4850-9a8a-c5103b205379/Confusion matrix already exists."
}
}
This makes me think that I'm not properly handling the command run.log_confusion_matrix()
. So, again, which is the best way I can log a confusion matrix to my azureml experiments?