0

I am creating a test Azure Data Factory Pipeline for learning purposes.

For this specific pipeline, I want to move files from one blob to the other. In ADF world, this involves creating a pipeline with the following activities:

  1. Get Metadata: retrieve file list from Blob 1
  2. Filter: from the output of previous activity, filter out folders, and output list of files
  3. Move files: get output from previous activity, move files from Blob 1 to Blob 2

ADF does not implement a Move File activity, so activity 2. above is actually a ForEach activity composed by two sub-activities:

2.1 Copy Files: Copy file from Blob 1 to Blob 2 2.2 Delete Files:Delete file from Blob 1

When debugging this pipeline, I am getting the following error for activity 2.2:

Failed to execute delete activity with data source 'AzureBlobStorage' and error 'The required Blob is missing. Folder path: sensor-sink-aws/test - Copy.json/.'

But the file exists in the container:

enter image description here

In my DeleteFile activity, I have the following data for Source:

enter image description here

Which basically references the Source Dataset that I created for this pipeline - it is a reusable dataset, meaning that I'm able to pass the container name and file name in a dynamic way.

Any idea on what could be going wrong?

1 Answer 1

1

From the path in the error it looks that the path of the blob is not correct "sensor-sink-aws/test - Copy.json/.'"

The file name should had been at the last , but as you can see that it "/." . I think you should check Delete activity -> Source ->Dataset -> Open-> Connection -> It should be some like the below screenshot ( please pay attention that the file name is at the end , my guess is you have put that in the middle text box )

enter image description here

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.