Questions tagged [clearml]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
38 views

Replacing IPs in MongoDB

I have a nested schema for mongoDB collection. It looks smth like this: { "_id":"61d99bf5544f4822bd963bda0a9c213b", "execution": { "test_split":0, ...
Clement Ruhm's user avatar
0 votes
1 answer
58 views

Automating a Script with ClearML

I am using ClearML to automate a script. I'm new to the tool and I haven't found any tutorials for this. The goal I'm trying to accomplish is automate several python scripts so that a server is spun ...
bballboy8's user avatar
  • 400
0 votes
1 answer
129 views

No preview images for dataset in ClearML web UI

After creating a new dataset via CLI for a bunch of images, I closed that dataset and thus uploaded it to our own, newly installed ClearML server. Now in the web UI the new dataset has been created ...
Matthias's user avatar
  • 9,827
0 votes
1 answer
125 views

ClearML webapp is slow

We manage our own ClearML server, on an EC2 instance AWS cloud. Instance type: t3.xlarge (4 vCPUs, 16 GiB Memory). Data disk: gp3 (size: 200 GB, IOPS: 3,000, Throughput: 125). We have 3 ClearML ...
hilel14's user avatar
  • 21
0 votes
2 answers
272 views

How to run the ClearML Cleanup?

I've installed ClearML test manager solution using ClearML Docker-Compose. So now the whole thing is running using 6 containers (webserver,apiserver,redis,elasticsearch,fileserver and mongodb). I'm ...
Shimi's user avatar
  • 3
1 vote
1 answer
377 views

Experiment tracking for multiple ML independent models using WandB in a single main evaluation

Can you recommend from your experience about choosing a convenient tracking experiment tool and versioning only "Multi independent models, but one input->multi-models->one output" in ...
AlexeyPrikhodko's user avatar
1 vote
2 answers
87 views

ClearML: How to merge 2 datasets which the 2 datasets inherited from a main dataset?

First I create a root (A) dataset. Then I created 2 dataset branches derived from root dataset. How do I "merge" these 2 branches to form another dataset ? Basically, the graph looks like ...
koayst's user avatar
  • 2,095
0 votes
1 answer
77 views

How to make ClearML not upload annotations twice when they have the same ID?

The following uploads two annotations, though I expected there to be only one from typing import List from allegroai import Dataset, DatasetVersion, SingleFrame, DataView from allegroai.dataframe....
Gulzar's user avatar
  • 23.7k
0 votes
1 answer
367 views

ClearML, how to query the best performing model for a specific project and metric

I want to download the best performing model for a certain ClearlML project. I have the following content in my ClearML experiment platform: According to: https://clear.ml/docs/latest/docs/...
Mike B's user avatar
  • 2,226
0 votes
2 answers
123 views

ClearML - dynamically updating Plotly plots?

I have a question related to ClearML plot logging. We are currently using: self.task_logger.report_table("TableSpaceName", "Some Info", iteration=0, table_plot=df) To report ...
sdgaw erzswer's user avatar
1 vote
1 answer
76 views

Does ClearML have accounting of information security events

ClearML is one of the most famous MLOps tools existing. It has logging of machine learning processes, however I couldn't find any information regarding its system of accounting of information security ...
alexlakiza's user avatar
2 votes
1 answer
491 views

[Catboost][ClearML] Error: if loss-function is Logloss, then class weights should be given for 0 and 1 classes

Having recently started using ClearML to manage the MLOps, I am facing the following problem: When running a script that trains a CatBoost in a binary classification problem using different class ...
Plinio Zanini's user avatar
0 votes
1 answer
37 views

Where do augmentations in ClearML run?

In ClearML Dataviews, it is possible to add augmentations. Where do these augmentations run? Options Original data gets downloaded to local, then runs (on which device? How is multiprocessing handled?...
Gulzar's user avatar
  • 23.7k
0 votes
1 answer
162 views

clearml-serving-triton | 2022-06-18 16:21:19,790 - clearml.Task - INFO - No repository found, storing script code instead

i found this error when running this command "docker-compose --env-file example.env -f docker-compose-triton.yml up". Actually, when i run this command for the first time, it worked. And ...
Richardles's user avatar
1 vote
1 answer
235 views

ModuleNotFoundError: No module named 'allegroai'

I am trying to use ClearML. The only line in my file is from allegroai import Dataset, DatasetVersion which yields ModuleNotFoundError: No module named 'allegroai' Looks like some pip package is ...
Gulzar's user avatar
  • 23.7k
0 votes
1 answer
195 views

Remotely execute ClearML task using local-only repo

I want to execute ClearML task remotely. According to docs there are 2 options: 1) execute single python file; 2) ClearML would identify that script is part of repo, that repo will be cloned and ...
Clement Ruhm's user avatar
-1 votes
1 answer
413 views

make clearml agent do not install envs for every task

I want make my clearml agent do not install python envs for every task, it take too long. I tried setup config like: package_manager.system_size_packages=true, but it doesn't work. clearml agent won't ...
宋仁杰's user avatar
0 votes
1 answer
315 views

ALB host based routing without domain name

I'm trying to configure host based routing in AWS ALB for ClearML server using this tutorial. However, I don't have a domain name. So can I only use alb's dns for this routing? For example, I will ...
Nghĩa Lê's user avatar
1 vote
0 answers
196 views

clearml hyperparameter optimization task fails on creating python environment

Trying to use clearml HyperparametersOptimizers. Launching task, but each experiment fails while trying to create python environment. Specificaly the error I get is related to pytorch clearml_agent: ...
Nir Schipper's user avatar
0 votes
1 answer
236 views

How to Deploy a ClearML Agent in a k8s setting?

I want to deploy a ClearML agent in the kubernetes environment while using the ClearML's Free Tier Demo server. I was able to deploy the Agent pod in the k8s cluster with the allegroai/clearml-agent ...
Jimit Modi's user avatar
0 votes
1 answer
190 views

Export metrics of ClearML to Prometheus and show them in Grafana

Are there any metrics I can get from the API server? or any docker image I can point to the backend and get some metrics? Most important is the see how many tasks running in real-time (like we can see ...
xdt's user avatar
  • 3
2 votes
1 answer
567 views

What would stop credentials from validation on a ClearML server?

I've set up a ClearML server in GCP using the sub-domain approach. I can access all three domains (https://app.clearml.mydomain.com, https://api.clearml.mydomain.com and https://files.clearml.mydomain....
mmiller-max's user avatar
0 votes
1 answer
280 views

how to capture logger values using clearml

I am using clearml for testing algorithms and it works well with library Stable Baselines 3, in which clearml automatically captures all the output and plot them in the Scalars tab. However, when I ...
Ventose's user avatar
0 votes
1 answer
233 views

ClearML How to get configurable hyperparameters?

How do I get args like epochs to show up in the UI configuration panel under hyperparameters? I want to be able to change number of epochs and learning rate from within the UI.
Fawad Nizamani's user avatar
1 vote
2 answers
1k views

Mounting an S3 bucket in docker in a clearml agent

What is the best practice for mounting an S3 container inside a docker image that will be using as a ClearML agent? I can think of 3 solutions, but have been unable to get any to work currently: Use ...
johnml1135's user avatar
  • 4,539
3 votes
1 answer
443 views

How to manage datasets in ClearML Web UI?

Using a self-deployed ClearML server with the clearml-data CLI, I would like to manage (or view) my datasets in the WebUI as shown on the ClearML webpage (https://clear.ml/mlops/clearml-feature-store/)...
kleka's user avatar
  • 364
3 votes
1 answer
150 views

ClearML get max value from logged values

I use ClearML to track my tensorboard logs (from PyTorch Lightning) during training. At a point later I start another script which connects to existing task and do some testing. But unfortenautly I do ...
Daniel Andres's user avatar
1 vote
1 answer
75 views

ClearML Web UI custom column not persistent

I'm using the experiments page of a project in ClearML Web UI to visualize some custom metrics. Therefore I've customized my table vie (https://allegro.ai/clearml/docs/docs/webapp/webapp_exp_table....
Daniel Andres's user avatar
1 vote
1 answer
529 views

ClearML multiple tasks in single script changes logged value names

I trained multiple models with different configuration for a custom hyperparameter search. I use pytorch_lightning and its logging (TensorboardLogger). When running my training script after Task.init()...
Daniel Andres's user avatar
2 votes
1 answer
681 views

ClearML how to change clearml.conf file in AWS Sagemaker

I am working in AWS Sagemaker Jupyter notebook. I have installed clearml package in AWS Sagemaker in Jupyter. ClearML server was installed on AWS EC2. I need to store artifacts and models in AWS S3 ...
Slava's user avatar
  • 55
1 vote
1 answer
251 views

ClearML SSH port forwarding fileserver not available in WEB Ui

Trying to use clearml-server on own Ubuntu 18.04.5 with SSH Port Forwarding and not beeing able to see my debug samples. My setup: ClearML server on hostA SSH Tunnel connections to access Web App ...
Daniel Andres's user avatar
3 votes
1 answer
539 views

ClearML server IP address not used with localhost and SSH port forwarding

Trying to use clearml-server on own Ubuntu 18.04.5. I use env variables to set the IP Address of my clearml-server. export CLEARML_HOST_IP=127.0.0.1 export TRAINS_HOST_IP=127.0.0.1 But it still is ...
Daniel Andres's user avatar
6 votes
1 answer
2k views

Can ClearML (formerly Trains) work a local server?

I am trying to start my way with ClearML (formerly known as Trains). I see on the documentation that I need to have server running, either on the ClearML platform itself, or on a remote machine using ...
DalyaG's user avatar
  • 2,979
2 votes
1 answer
54 views

How to fix trainserver empty server? [closed]

Im trying to install an allegroai trains-server on a k8s cluster. I tried the following 3 methods bare linux installtion k8s manifest installation helm installation I followed the linux installation ...
Dean Light's user avatar
1 vote
1 answer
306 views

Trains: reusing previous task id

I am using reuse_last_task_id=True to overwrite an existing task (with same project and task name). But the experiment contains the torch model and therefore does not overwrite the existing task but ...
kyc12's user avatar
  • 329
1 vote
3 answers
1k views

pip install trains fails

upon running pip install trains in my virtual env I am getting ERROR: Command errored out with exit status 1: command: /home/epdadmin/noam/code/venv_linux/bin/python -u -c 'import sys, ...
Gulzar's user avatar
  • 23.7k
2 votes
1 answer
145 views

How should Trains be used with hyper-param optimization tools like RayTune?

What could be a reasonable setup for this? Can I call Task.init() multiple times in the same execution?
Michael Litvin's user avatar
1 vote
1 answer
174 views

Will Trains automagically log Tensorboard HParams?

I know that it's possible to send hyper-params as a dictionary to Trains. But can it also automagically log hyper-params that are logged using the TF2 HParams module? Edit: This is done in the HParams ...
Michael Litvin's user avatar
-1 votes
1 answer
54 views

How resilient is reporting to Trains server?

How would Trains go about sending any missing data to the server in the following scenarios? Internet connection breaks temporarily while running an experiment Internet connection breaks and doesn't ...
Michael Litvin's user avatar
2 votes
1 answer
420 views

Trains: Can I reset the status of a task? (from 'Aborted' back to 'Running')

I had to stop training in the middle, which set the Trains status to Aborted. Later I continued it from the last checkpoint, but the status remained Aborted. Furthermore, automatic training metrics ...
Michael Litvin's user avatar
0 votes
1 answer
32 views

Does clone experiment work on sklearn functions?

I'm trying to run a script and I'm constantly getting this while cloning experiment in allegro.ai AttributeError: 'Namespace' object has no attribute 'get' Can anybody help?
deadofwrite's user avatar
1 vote
1 answer
211 views

Can Trains config file be specified dynamically or relative to the running script path?

Suppose I have a server where many users run different experiments, possibly with different Trains Servers. I know about the TRAINS_CONFIG_FILE environment variable, but I wonder if this can be made ...
Michael Litvin's user avatar
2 votes
2 answers
144 views

Tracking separate train/test processes with Trains

In my setup, I run a script that trains a model and starts generating checkpoints. Another script watches for new checkpoints and evaluates them. The scripts run in parallel, so evaluation is just a ...
Michael Litvin's user avatar
2 votes
1 answer
66 views

Parallel Coordinates Plot in TRAINS

Is there a way to create a parallel coordinates plot in TRAINS (https://github.com/allegroai/trains) package to compare several hyper-parameters in respect to a specific metric?
Majd's user avatar
  • 21
1 vote
2 answers
134 views

trains with grid search

I would like to test trains usage during grid search and it not clear how to do so. from trains import Task Task.init(project_name="project name", task_name='name') creates an experiment in the ...
Sefi Erlich's user avatar
1 vote
1 answer
3k views

How to Backup/Restore TRAINS-server when moving from AMI to local machine

I recently started using TRAINS, with the server in AWS AMI. We are currently using v0.9.0. I would like to move the TRAINS-server to run on our on-premises kubernetes cluster. However, I don't want ...
Oren's user avatar
  • 31
1 vote
1 answer
169 views

How to manually register a sci-kit model with TRAINS python auto-magical experiment manager?

I'm working mostly with scikit-learn, as far as I understand, the TRAINS auto-magic doesn't catch scikit-learn model store/load automatically. How do I manually register the model after I have '...
Yael's user avatar
  • 11
1 vote
1 answer
436 views

Is there a way to create a graph comparing hyper-parameters vs model accuracy with TRAINS python package?

I would like to run multiple experiments, then report model accuracy per experiment. I'm training a toy MNIST example with pytorch (v1.1.0), but the goal is, once I can compare performance for the ...
fidlr's user avatar
  • 81