4

I have a running MlFlow server on GCS VM instance. I have created a bucket to log the artifacts. This is the command I'm running to start the server and for specifying bucket path-

mlflow server --default-artifact-root gs://gcs_bucket/artifacts --host x.x.x.x

But facing this error:

TypeError: stat: path should be string, bytes, os.PathLike or integer, not ElasticNet

Note- The mlflow server is running fine with the specified host alone. The problem is in the way when I'm specifying the storage bucket path. I have given permission of storage api by using these commands:

gcloud auth application-default login
gcloud auth login

Also, on printing the artifact URI, this is what I'm getting:

mlflow.get_artifact_uri()

Output:

gs://gcs_bucket/artifacts/0/122481bf990xxxxxxxxxxxxxxxxxxxxx/artifacts

So in the above path from where this is coming 0/122481bf990xxxxxxxxxxxxxxxxxxxxx/artifacts and why it's not getting auto-created at gs://gcs_bucket/artifacts

After debugging more, why it's not able to get the local path from VM: enter image description here

And this error I'm getting on VM:

ARNING:root:Malformed experiment 'mlruns'. Detailed error Yaml file './mlruns/mlruns/meta.yaml' does not exist.
Traceback (most recent call last):
 File "/usr/local/lib/python3.6/dist-packages/mlflow/store/tracking/file_store.py", line 197, in list_experiments
   experiment = self._get_experiment(exp_id, view_type)
 File "/usr/local/lib/python3.6/dist-packages/mlflow/store/tracking/file_store.py", line 256, in _get_experiment
   meta = read_yaml(experiment_dir, FileStore.META_DATA_FILE_NAME)
 File "/usr/local/lib/python3.6/dist-packages/mlflow/utils/file_utils.py", line 160, in read_yaml
   raise MissingConfigException("Yaml file '%s' does not exist." % file_path)
mlflow.exceptions.MissingConfigException: Yaml file './mlruns/mlruns/meta.yaml' does not exist.

Can I get a solution to this and what I'm missing?

7
  • 1
    Have a look at mlflow.org/docs/latest/tracking.html#google-cloud-storage . Probably something wrong with permissions. Try to follow instructions from documentation and update your question with details. Mar 10, 2020 at 12:24
  • 1
    Have you given permissions to the account that is running this command? if not, try adding the Storage Object Viewer and Storage Object Creator to check if it works.
    – Ralemos
    Mar 10, 2020 at 15:29
  • I have updated the question, please review and try answering @SerhiiRohoza Mar 11, 2020 at 9:18
  • I have given these permissions already @ralemos Mar 11, 2020 at 10:16
  • Those permissions should be sufficient to allow what you are trying to do, can you try changing to the Storage Admin role so we can check the permissions? Also, can you try doing something like creating the object from the instance with the service account like this?
    – Ralemos
    Mar 11, 2020 at 15:53

1 Answer 1

1

I think the main error is from the structure that you want to deploy. For your use case, the structure is suitable that in here. So you miss the URI path which used to store backend metadata. So please install DB SQL(PostgreSQL,...) first, then add the path to --backend-storage-uri.

In case you want to use MlFlow as a model registry and store images on gcs. You can use this structure in here with adding tag --artifacts-only --serve-artifacts

Hope this can help you.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.