Questions tagged [amazon-sagemaker]

Amazon SageMaker is a fully-managed AWS service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale.

Filter by
Sorted by
Tagged with
0 votes
0 answers
4 views

Opencv Build Wheel Timeout in SageMaker Endpoint Deployment

I am trying to deploy an AWS SageMaker Notebook endpoint. In my requirements.txt file, I include opencv_python. When I try to deploy my endpoint, I get the following error: "...
athanzxyt's user avatar
-4 votes
0 answers
30 views

How do I work through the code in this example? [closed]

I would like to begin playing with some machine learning models from AWS. I see that they recently released a Bedrock service, and I would like to setup an environment to do that. I would also need to ...
chai86's user avatar
  • 437
0 votes
0 answers
6 views

unable to extract text using tika in sagemaker, what is the reason for this?

code I've used: from tika import parser headers = { 'X-Tika-PDFextractInlineImages': 'true', } text = parser.from_file("pdffiles/BG1309020268.pdf",headers=headers) print(text) text = parser....
sky's user avatar
  • 1
0 votes
0 answers
8 views

How to write a lambda function for the Map task in the sagemaker?

I have a machine learning model deployed in SageMaker, and I'm currently working on a process to handle multiple inputs for predictions. I understand that AWS Lambda can be used to implement a Map ...
Nima Sari's user avatar
0 votes
0 answers
19 views

Invoking SageMaker endpoint failed since timm library was not found in the environment

I deployed a Hugging Face facebook/detr-resnet-50 model using the official code provided here: https://huggingface.co/facebook/detr-resnet-50?sagemaker_deploy=true Then, I tried to use the predictor ...
Hai's user avatar
  • 41
0 votes
0 answers
7 views

How to deploy multiple models in amazon sagemaker on various endpoints

I'm recently interested to Amazon Sagemaker and Rekognition. I would like to try AWS instances and deploy ML models. I am going to train and deploy models for diagnoising diseases(Image ...
dayday's user avatar
  • 33
1 vote
0 answers
29 views

How do I use Sagemaker HyperparameterTuner on a SKLearn Estimator?

I'm following an Amazon Sagemaker workshop to try and leverage several of Sagemaker's utilities instead of running everything off a Notebook as I'm currently doing. The thing is, in the workshop they ...
jvmunhoz's user avatar
0 votes
0 answers
18 views

Save pyspark dataframe as RecordIO protobuf

I want to save my pyspark dataframe in RecordIO protobuf format. I am using Amazon EMR to run my pyspark scripts, and I want to use AWS SageMaker to train a machine learning model. SageMaker pipe mode ...
Maokai's user avatar
  • 164
0 votes
0 answers
14 views

Path Structure for Processing inputs to step function "AppManaged" and not app managed for a sagemaker Job

I have a use case where i am running a sagemaker processing job in a step function , now for this use case input is : "ProcessingInputs": [ { "...
divyanayan awasthi's user avatar
1 vote
1 answer
24 views

Sagemaker Studio Notebook - How to access OS Environ

I am trying to set (at the terminal or in the notebook) an environment variable. I am running the following in the terminal or (with %%bash in the cell) notebook cell: echo "export MY_KEY='BLAH-...
B_Miner's user avatar
  • 1,850
0 votes
0 answers
19 views

AWS SageMaker Real-Time inference configure storage size

i'm currently using SageMaker real-time inference (and async inference) since September. Now my endpoint workload increased such that i need to process a couple hundreds of HD photos (each about 30-40 ...
Lorenzo Di Giacomo's user avatar
2 votes
1 answer
100 views

Sagemaker serverless ValidationError: oversize body

I have an image classification model stored in s3 location. If I deploy the model as real-time inference, then I am able to predict any image size, however, if I deploy the model as serverless then I ...
sid8491's user avatar
  • 6,642
0 votes
0 answers
21 views
+100

Building ML Model in AWS Sagemaker autopilot with Time Series feature

I am building a classification model in AWS Sagemaker using Autopilot. I have imported a dataset which has a time series feature. This features includes a list of values in each cell. The data set is ...
Dataminer's user avatar
  • 1,419
1 vote
0 answers
29 views

Sagemaker giving 'OSError: [Errno 28] No space left on device' when separate model and code zip is given

This looks like an interesting problem. When I am zipping my model and code together(like below), things work fine. huggingface_model = HuggingFaceModel( model_data="s3://abc/xyz/model.tar.gz&...
Sid's user avatar
  • 599
-4 votes
0 answers
46 views

Reading local excel files on AWS without uploading it to S3 bucket

How to Read local excel files on AWS Sagemaker notebook without uploading it to S3 bucket. Basically I want to read excel/csv files from local drive, and I don't want to duplicate the same file on s3 ...
mahesh purbia's user avatar
0 votes
0 answers
23 views

Using outputs from TrainingStep in Sagemaker Pipeline

My TrainingStep works as needed in my sagemaker pipeline, however I can't seem to access it in any way in the future steps of my sagemaker pipeline. I am using ProcessingInputs but I can't seem to ...
Fen P.'s user avatar
  • 1
0 votes
0 answers
33 views
+50

Can't train sagemaker model using recordio protobuf data saved in S3

I want to preprocess data in Amazon EMR using PySpark and train a machine learning model in SageMaker using pipe mode. The issue I am having now is saving data in S3 and feeding it to the model. ...
Maokai's user avatar
  • 164
0 votes
0 answers
7 views

Do I have to send all trainging data columns when requesting a prediction from AWS SageMaker?

I'm using the Javascript library @aws-sdk/client-sagemaker-runtime to interface with AWS SageMaker. I'm able to successfully request predictions from our model like this: const params = { ...
user2943799's user avatar
0 votes
0 answers
13 views

`arguments` method of `ProcessingStep` results in unwanted upload to S3

I want to register a model made via SageMaker Pipelines. Following this as an example. Here is the code for model registration: import logging from sagemaker.drift_check_baselines import ...
Francisco C's user avatar
0 votes
1 answer
16 views

Are instances used in sagemaker pipeline destroyed automatically after the pipeline completion?

I saw that for each step in sagemaker pipeline we define instances on which the step will be run. So, after the completion of each step is the instance destroyed, or does it keep on running?
V K's user avatar
  • 1,669
0 votes
0 answers
28 views

How to setup pytesseract path in amazon sagemaker notebook (cloud)?

I have installed pytesseract using pip install pytesseract in sagemaker notebook now I need to setup tesseract path pytesseract.pytesseract.tesseract_cmd = 'home/studio-lab-user/.conda/envs/studiolab/...
sk jainmiah's user avatar
1 vote
1 answer
45 views

Loading a pretrained model from torch.hub in Sagemaker

I'm getting an import error when trying to load a pretrained resnet50 model in Sagemaker Studio. I'm using the provided Pytorch 2.0.0 Python 3.10 GPU optimized image on a t3.medium. My code: import ...
John Brown's user avatar
0 votes
1 answer
19 views

In AWS Sagemaker Pipeline, handing off CreateModelStep to TransformStep results in ModuleNotFoundError: No module named 'inference'

In AWS Sagemaker Pipeline, I have a CreateModelStep that connects to a TransformStep. When I get to the TransformStep, I see the following error thrown. Traceback (most recent call last): File "/...
Jane Wayne's user avatar
  • 8,317
0 votes
1 answer
23 views

Can we use sagemaker model python CLI without using s3 bucket?

I'm trying to train the Anthropic model to understand the table of data and it's meta data(table about how the tables are related). Then use it to generate an SQL to fetch the data from database. I ...
Bhavani Priya's user avatar
-1 votes
1 answer
39 views

JumpStartEstimator.fit ERROR: UnboundLocalError: local variable 'dataset_train' referenced before assignment

I have a simple model tuning python code, when I run I get the fallowing error, appreciate your help. ERROR: sagemaker.exceptions.UnexpectedStatusException: Error for Training job meta-textgeneration-...
Reza Nayeri's user avatar
0 votes
0 answers
33 views

how to download output artifacts from AWS sagemaker (Python)

I have a code similar to this one ''' from sagemaker.experiments import Experiment from sagemaker.experiments import Run import matplotlib.pyplot as plt from sagemaker import Session ...
Hadoop User's user avatar
0 votes
0 answers
44 views

SageMaker Pytorch Job Not Finding Custom Dependency Files

I've been trying to get SageMaker to run my job for a week and keep running into the same issue no matter what I try. Folder Structure Root: __init__.py train.py /scripts (contains my_dataloader.py ...
user3857124's user avatar
0 votes
1 answer
61 views

how to fine-tune stable diffusion in sagemaker jumpstart with different descriptions per image

I'm trying to fine-tune stable diffusion in sagemaker using this method: https://aws.amazon.com/blogs/machine-learning/fine-tune-text-to-image-stable-diffusion-models-with-amazon-sagemaker-jumpstart/ ...
Luis Leal's user avatar
  • 3,356
0 votes
0 answers
27 views

SageMaker batch transform error Requested instance type cannot work with all containers of the model

I'd appreciate your help with this error: botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateTransformJob operation: Requested instance type cannot work ...
Reza Nayeri's user avatar
0 votes
0 answers
31 views

Sagemaker - build your own docker container from a poetry vm

I am learning sagemaker and would like to build my own Docker container from my code. I looked at examples here: https://github.com/aws/sagemaker-training-toolkit, but I have no experience with Docker ...
Ziqi's user avatar
  • 2,465
0 votes
2 answers
98 views

Deploy LLM using Sagemaker and Langchain

I am trying to deploy Generative AI solution built using Langchain (obviously with LLM at it's core) and Sagemaker. So, the code is not just an inference script but inference pipeline (challenge is ...
akshat garg's user avatar
0 votes
0 answers
32 views

How do I define SM_CHANNEL_TRAIN for a training step in AWS Sagemaker Pipelines?

I am using AWS Sagemaker Pipelines, and apparently I cannot correctly define the value for SM_CHANNEL_TRAIN. Here is my pipeline definition. import boto3 import sagemaker import sagemaker.session from ...
Jane Wayne's user avatar
  • 8,317
-1 votes
0 answers
25 views

How to output variable from ProcessingStep to use in the Sagemaker pipeline script?

I'm trying to use factorization machines in sagemaker which need a value for feature_dim which I want for it to be the same number as columns from the training dataset. The problem is that I only ...
Ricardo Ferreira's user avatar
0 votes
0 answers
25 views

AWS Sagemaker Endpoint inference "async-invocations" endpoint does not log of my log / print statements

I've exposed a Sagemaker async-invocations endpoint using Python and Flask as explained here. When I invoke the endpoint it gets triggered with correct inputs sent from the invocation and I can see ...
Temp O'rary's user avatar
  • 5,476
0 votes
1 answer
40 views

ValueError: math domain error in AutoARIMA model of StatsForecast

I am trying to use statsforecast AutoArima for forecasting on below type of data: zip_code product_family week_date rma_count 12198 ABC 2021-01-03 6.0 61022 DEF 2021-01-03 1.0 43106 GHI 2021-01-03 ...
parag mittal's user avatar
0 votes
0 answers
11 views

Aws neo - RuntimeError('PytorchStreamReader failed locating file constants.pkl: file not found')")

I have a checkpoint.pth file . This is working fine in sagemaker and giving results. However when i try to use aws-neo-compilation-job i am seeing below error ClientError: InputConfiguration: We are ...
Jeya Kumar's user avatar
  • 1,022
0 votes
0 answers
37 views

Retry for Sagemaker ModelError

When querying my sagemaker inference endpoint using the following code config = Config(retries={"max_attempts": 4, "mode": "standard"}) sagemaker = boto3.client("...
cotrane's user avatar
  • 159
0 votes
0 answers
9 views

Sagemaker Clarify Job keeps returning ClientError: Threshold values must be provided for continuous features

Data Config from sagemaker import clarify data_config_unbalanced = clarify.DataConfig( s3_data_input_path='s3://a/b.csv', # S3 object path containing the unbalanced dataset s3_output_path='s3:...
PUG's user avatar
  • 4,310
0 votes
0 answers
49 views

Is It Possible to Automatically Enable Code Whisperer Extension Using SageMaker Lifecycle Configurations?

I've been following the AWS blog post on using SageMaker Lifecycle Configurations for customizing SageMaker Studio. According to the post, the second use case involves the installation of custom ...
mjack34's user avatar
  • 11
0 votes
1 answer
48 views

Sagemaker LLama2 endpoint without huggingface token

I have a model.tar.gz archive with the following files in it: LICENSE.txt README.md Responsible-Use-Guide.pdf USE_POLICY.md checklist.chk consolidated.00.pth params.json tokenizer.model ...
Spas Kalaydzhiyski's user avatar
0 votes
2 answers
32 views

How to read large text file in Amazon SageMaker?

I want to read multiple text file from s3 bucket which will make up a data frame of 10M records and 900 columns. But it is taking too long to read data using ml.p3.2xlarge instance. Moreover, I would ...
Mandar's user avatar
  • 11
0 votes
1 answer
47 views

Managing Multiple AWS SageMaker Lifecycle Configurations

I've been exploring SageMaker Lifecycle Configurations, particularly in reference to the setting default LCC in the AWS documentation . According to the documentation, it's possible to attach multiple ...
mjack34's user avatar
  • 11
-1 votes
1 answer
64 views

Sagemaker-huggingface An error occurred (ValidationException) when calling the CreateModel operation: Requested image not found

I'm trying to create an asyncronous sagemaker endpoint as shown here shown here but using a huggingface model, so I need to use one of the huggingface images and for that I check the official list of ...
Luis Leal's user avatar
  • 3,356
0 votes
0 answers
22 views

I've trained an autopilot model using AutoML tool in Sagemaker. I want to load this model outside of sagemaker and score data in local machine

I have trained an aws autopilot model on Sagemaker. I am trying to load this model in local and score data. But I'm getting error when I'm trying to load model using pickle. And when I'm trying to ...
Shivang Singh Kaira's user avatar
1 vote
1 answer
36 views

How do I detach the HuggingFace SageMaker training?

I am training a HuggingFace model remotely on SageMaker integration. My training job takes more than two hours, and I would like to shut my computer off during training. I use the following snippet to ...
lmaooooo's user avatar
  • 3,427
0 votes
1 answer
22 views

Sagemaker Monitor - MonitoringDatasetFormat as gz

I have created a monitoring schedule to monitor predictions from a Batch Transform job. The schedule runs fine when the input dataset_format in BatchTransformInput is csv. However, my batch job is ...
dee's user avatar
  • 26
0 votes
0 answers
25 views

Is there a way to know the structure and code of lamda provided by aws?

I am using aws-sagemaker. Follow the workflow and provide by aws during use You are trying to use Lambda with arn:aws:lambda:ap-southeast-2:454466003867:function:PRE-ImageMultiClass ARN. But I want to ...
Changhoon Lee's user avatar
0 votes
1 answer
164 views

Error in deploying Falcon-7B after fine-tuning to AWS SageMaker endpoint using SageMaker Python SDK

I am currently running into a problem with AWS SageMaker where I cannot deploy my fine-tuned Falcon-7B model to a SageMaker endpoint after training it using an AWS training job. I am roughly following ...
Jacob Brophy's user avatar
0 votes
1 answer
20 views

AWS Sagemaker. How to pass data_capture_config parameter to Pipeline?

I'm trying to monitorize my data so I need to use the DataCaptureConfig object in Sagemaker. The point is that I'm using a Pipeline object that include all my steps. In the documentation it seems that ...
Guillermo Álvarez's user avatar
0 votes
1 answer
111 views

Pytorch-Dataloader throws multiprocessing exception when deployed inside gunicorn/flask server

I am deploying my pytorch model in aws-sagemaker container and using gunicorn server for inference imports i have Dependency versions matplotlib = 3.0; pandas = 2.0; ...
Jeya Kumar's user avatar
  • 1,022

1
2 3 4 5
59