Questions tagged [amazon-sagemaker]
Amazon SageMaker is a fully-managed AWS service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale.
2,173
questions
0
votes
0
answers
19
views
Received client error (400) deploying huggingface bigscience/bloom to SageMaker
I want to deploy Bloom on SageMaker so that I have a Bloom inference API I can use. I started by running the following in a SageMaker jupyter notebook:
from sagemaker.huggingface import ...
0
votes
0
answers
24
views
Sagemaker training job failed - having issues executing user script
I am very new to AWS Sagemaker and am trying to deploy my SKLearn script to an endpoint so that I can call it from within an Android app. I am following the code here, and so far, getting each block ...
0
votes
1
answer
15
views
Distributed Training Terminology: Micro-batch and Per-Replica batch size
I am reading through the Sagemaker documentation on distributed training and confused on the terminology:
Mini-Batch, Micro-batch and Per-replica batch size
I understand that in data parallelism, ...
0
votes
1
answer
25
views
Start execution of existing SageMaker pipeline using Python SDK
SageMaker documentatin explains how to run a pipeline, but it assumes I have just defined it and I have the object pipeline available.
How can I run an existing pipeline with Python SDK?
I know how to ...
-3
votes
1
answer
27
views
AWS Sagemaker model + explainablity + shap value + report of model [closed]
I have created a sagemaker machine model with auto pilot.I need explainablity for my prediction.Sage maker gives explainablity for overall data set and generated report as pdf, html , python format in ...
0
votes
1
answer
17
views
Load estimator from model artifact in s3 bucket aws
I have used estimator for a pytorch model and have saved the artifacts in s3 bucket. using below code
estimator = PyTorch(
entry_point="train_deploy.py",
source_dir="code",
...
0
votes
1
answer
17
views
how to convert binary file to pandas dataframe
I use Amazon Sagemaker for model training and prediction. I have a problem with the returned data with predictions. I am trying to convert prediction data to pandas dataframe format.
After the model ...
0
votes
2
answers
35
views
On sagemaker in multicontainer endpoint ping isn't able to call
I'm also facing the same issue. If I'm deploying one container on one endpoint, then it works perfectly.
But when I'm trying to deploy the multiple containers on an endpoint. Then the serve file is ...
0
votes
1
answer
21
views
Trigger Sagemaker Inference from SQS event
I have previously created Sagemaker Inference end-point and triggered from with API Gateway, but how can I trigger it from SQS event.
Usecase is: We are getting events in SQS, and whenever we get an ...
-1
votes
0
answers
28
views
AWS Sagemaker: NetworkingError - attempting to create a labeling job
Currently trying to set up an AWS Sagemaker Named Entity Recognition (NER) labeling job with this guide: https://aws.amazon.com/blogs/machine-learning/adding-a-data-labeling-workflow-for-named-entity-...
0
votes
2
answers
27
views
AWS Sagemaker integration with mongodb and lambda
I'm looking for some advice from anyone who's tried aws Sagemaker. I'm very new to this and would appreciate anyone kind enough to help me out.
I have created a basic time series project in a ...
0
votes
0
answers
28
views
InternalError when trying to predict with an endpoint on AWS
I use AWS SageMaker for a ML project and my TrainingJob for image classification has successfully finished. I use a separate inference.py for the inference, that looks like this:
import json
import ...
0
votes
1
answer
42
views
I am exploring the Incremental Learning feature in Amazon Sagemaker
I have access to AWS Console and started exploring the Sagemaker Incremental Learning documentation. It is mentioned that only three built-in algorithms currently support incremental training: Object ...
0
votes
1
answer
33
views
sagemaker notebook instance failed to run even a single line of code
l am running a notebook instance as usual.
But suddenly something was wrong. The notebook failed to give out anything. So l use a simple line of code to test it, which returns the following result:
my ...
0
votes
1
answer
26
views
Running ipynb file, but can't import a certain module
l'am running a ipynb file on sagemaker, however the error of occurs.
l have used 'pip install tqdm' in terminals to install the tqdm so l've no idea what's happening. Is it running in a different ...
0
votes
0
answers
16
views
Reinforcement based Auto Scaling Demo works very bad
Question about the reinforcement learning based auto scaling demo.
https://github.com/aws/amazon-sagemaker-examples/blob/main/reinforcement_learning/rl_predictive_autoscaling_coach_customEnv/...
0
votes
0
answers
35
views
Sagemaker model deployment fails: /opt/ml/model fails with OSError: [Errno 30] Read-only file system
I am deploying a trained/customized model on Sagemaker. Container creation and training works perfectly well. I can my model's pickle files but when I run
predictor = model.deploy(1, instance_type='...
0
votes
0
answers
44
views
Is there any workaround for installing packages ? (deepspeed error)
i was trying to use deepspeed, but when i run the training it shows an error. I have to install mpi4py using pip. However if I try to install mpi4py i get an error.
From that error i found out that to ...
1
vote
1
answer
24
views
Sagemaker API online feature store creation not using given kms key
I am calling Sagemaker API from python script inside EC2 instance to create online feature store. I gave required permission and its creating feature group.
However I observed that key I'm passing in ...
0
votes
2
answers
19
views
How to reduce the verbosity of sagemaker training jobs?
I am training a model with tqdm in the dataloader, the log prints every update in a seperate line that fills up the entire notebook. Is there a way to reduce the verbosity or remove the logs entirely ...
0
votes
1
answer
62
views
Sagemaker - No such file or directory
I want to run sagemaker and I adjusted my code according to the following example:
step 3: https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-preprocess-data.html
and
step 4: https://docs.aws....
0
votes
2
answers
18
views
How to make SageMaker XGBoost Hyperparameter tuning work in script mode
I'm following up after this tutorial and wondering if is it possible to combine the script mode with Hyperparameter tuning. If I'm trying to do so, the HPT.fit() runs my script again and again (the ...
0
votes
1
answer
23
views
Deploy Pre-Trained model to SageMaker Endpoint from CloudFormation
This seems to be a tricky thing to do, as I haven't found too much documentation for it. I'm trying to deploy a Huggingface pre-trained model for NLU to a SageMaker endpoint. Naturally, I don't want ...
0
votes
1
answer
30
views
Fb-Prophet, Apache Spark in Colab and AWS SageMaker/ Lambda
I am using Google-Colab for creating a model by using FbProphet and i am try to use Apache Spark in the Google-Colab itself. Now can i upload this Google-colab notebook in aws Sagemaker/Lambda for ...
0
votes
1
answer
23
views
validation exception while featuregroup creation through sagemaker in ec2
I am referring to document https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-featurestore/sagemaker_featurestore_fraud_detection_python_sdk.html#Ingest-Data-into-FeatureStore to create ...
0
votes
0
answers
34
views
stacking cnn output layer with xgboost. Data prep gives OOM error
I have trained a cnn model and I am trying to stack the output layer to an xgboost regressor to reduce mape. I am getting OOM error in Sagemaker training job when I try to include the input data (in ...
0
votes
0
answers
38
views
SagemakerTraining job catboost-classification-model , ErrorMessage "TypeError: Cannot convert 'xxx'' to float
When I performed the following AWS tutorial, I got an error when training the model. https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/...
0
votes
1
answer
47
views
Sagemaker Pipelines - Unable to parse Pipeline Definition
I'm using Sagemaker Pipelines to chain together two consecutive ProcessingJobs. I'm getting a weird error when I call pipeline.upsert()
botocore.exceptions.ClientError: An error occurred (...
-1
votes
1
answer
14
views
Sagemaker pytorch inference stops at model call on gpu
I deployed a pytorch model using sagemaker and can successfully query it on a CPU. Deploying it on a GPU leads to a InternalServerError client-side though. Checking the CloudWatch Logs shows that the ...
0
votes
1
answer
12
views
why Sagemaker Processing have several ouputs
I have been using AWS Sagemaker for years.
I don't understand why processing jobs can have several outputs ? In what kind of scenario, can you use more than one output destination ?
When I say several ...
0
votes
0
answers
18
views
AWS Sagemaker Studio stuck in sign-in pop up
I am stuck in this pop up window when I launch SageMaker Studio from control panel.
I clicked the sign-in button, another window popped saying redirecting me to the domain but then nothing happened. I ...
0
votes
2
answers
50
views
Extract model saved in S3 bucket as tar.gz format to sagemaker notebook instance
I have a tar.gz file inside a S3 bucket, this is a file containing 6 different 'pickled' model zipped together. This was created after training a model with SageMaker docker container in single run.
...
0
votes
1
answer
46
views
Terraform - Optional Nested Variable
I'm trying to create a module for Sagemaker endpoints. There's an optional object variable called async_inference_config. If you omit it, the endpoint being deployed is synchronous, but if you include ...
0
votes
0
answers
16
views
Not able to create condo environment in SageMaker studio
Sagemaker studio terminal shows default python version as "Python 3.7.10"
I am trying to create conda environment with python3.9 version as below.
conda create --name custom_python_39 python=...
1
vote
1
answer
39
views
Access denied for aws public sagemaker xgboost registry
I am trying to pull a prebuilt xgboost image from the public aws xgboost registry specified here: https://docs.aws.amazon.com/sagemaker/latest/dg/ecr-us-west-2.html#xgboost-us-west-2.title, however ...
0
votes
0
answers
40
views
Print message from inside AWS SageMaker endpoint
I've set up an endpoint for a model in AWS SageMaker. I set up my own container and am trying to debug some path related errors. To do this I need to print some information so I know what's going on.
...
0
votes
1
answer
21
views
ModelNotReadyException when calling sagemaker
Last night I received this error twice when invoking a serverless sagemaker endpoint
serverless ClientError: An error occurred (ModelNotReadyException)
when calling the InvokeEndpoint operation (...
0
votes
0
answers
23
views
Using MLflow and Sagemaker with preprocessing steps
I'm deploying my models to Sagemaker using MLflow integration. However, my ML pipeline includes some basic preprocessing steps, such as scalers, and I need it to be part of my inference endpoint. Is ...
0
votes
1
answer
24
views
Using CLI to start using SageMaker in browser
In the case of using AWS only via aws cli, how can I open a sagemaker app in the browser if I already create sagemaker app and have apparn?
0
votes
1
answer
26
views
How to use scikit learn model from inside sagemaker 'model.tar.gz' file?
New to Sagemaker..
Trained a "linear-learner" classification model using the Sagemaker API, and it saved a "model.tar.gz" file in my s3 path. From what I understand SM just used an ...
0
votes
1
answer
21
views
Print SageMaker instance type
Is there a function I can use to get the instance type of my SageMaker instance.
I basically want to do something like this
region = boto3.Session().region_name
but for the instance type.
I know I ...
0
votes
0
answers
56
views
not able to update trust policy for a role
I am trying to create featureGroup using sagemaker API in ec2 instance.
got below error while running python script which creates featureGroup.
botocore.exceptions.ClientError: An error occurred (...
0
votes
1
answer
53
views
Error when deploying cross account Sagemaker Endpoints
I am using cdk to deploy a Sagemaker Endpoint in a cross-account context.
The following error appears when creating the Sagemaker Endpoint:
Failed to download model data for container "...
0
votes
1
answer
29
views
AccessDenied error when pushing docker image from SageMaker to ECR
I've created a docker image using AWS SageMaker and am now trying to push said image to ECR. When I do docker push ${fullname} it retries a couple of times and then errors.
In CloudTrail I can see ...
0
votes
1
answer
37
views
AWS CLI: How to use variable to filter EFS
I want to use the value of the DOMAIN_ID variable to filter the EFS to get a FileSystemId. I used the commands below. The first command works and it stores the domain ID. The second one returns an ...
0
votes
1
answer
23
views
Sagemaker Pipelines with Autopilot Step
I'm looking to combine the Autopilot AutoML training and deployment step into a wider Pipeline using Sagemaker.
I can't find anything in the documentation around using AutoML with Pipelines.
Does ...
0
votes
0
answers
37
views
Issues with deploying spark and mlflow to sagemaker
My goal is to deploy a spark/mlflow to sagemaker with the following command:
mlflow sagemaker deploy ..
I've successfully pushed a image to EC2 with
mlflow sagemaker build-and-push-container
I ...
-1
votes
1
answer
25
views
How does tagging work in sage maker studio. Does the tags applied to users propagate to the resources created in studio
In sage maker studio, How can we apply tags to the resources which are created by the user inside the studio.
As per my knowledge, Tags can be applied to the users created under the domain. Does these ...
1
vote
0
answers
28
views
SageMaker Batch Transform fails to access nginx
I had been trying to deploy a ML model on SageMaker using Docker.
So far I have been able to run the Training job, however the model inference using batch transform is having issue with nginx log:
...
0
votes
1
answer
39
views
How to build a simple spark-test-app.jar to test AWS SageMaker SparkJarProcessing
Does anyone know a repo that shows what a simple HelloWorld java or scala code would look like to build the jar that could be executed using the AWS SageMaker SparkJarProcessing class?
Readthedocs (...