3

I'm using first 100 of lines from The Enron Email Dataset for my experiment in Azure ML Studio, however the Saved Dataset object is being populated with odd 4.8K lines instead of 100. That must be due to "Inaccurate column separation on string data containing commas" issue, which I understand.

However, using same dataset in the Python project locally and/or in Azure ML Jupyter notebook (same imported dataset from ML Studio - not separately imported to Jupyter notebook) the number of lines is being read correctly and the further logic does also work fine.

Jupyter example:

from azureml import Workspace
ws = Workspace()
ds = ws.datasets['The Enron Email Dataset (Minimal)']
emails_df = ds.to_dataframe()

Local example:

import pandas as pd
emails_df = pd.read_csv('C:/enron-email-dataset/emails.csv', nrows=100)

And here is how dataset visualisation looks like in Azure ML Studio enter image description here

It's clear that it get's messed up after it gets moved from saved datasets to an experiment, but my question is - what would be the best way to work around it? Calling dataset from Azure BLOB Storage inside my Python code perhaps?

EDIT 1: Removing commas from CSV file didn't helped either. The dataset is working fine in Jupyter and locally, but it appears messed up when added to the experiment.

EDIT 2: Removing carriage returns resulted in dataset being visualised correctly in the experiment, however that broke the further text pre-processing logic in Python code (email.parser). Which I could amend making it to work across different environments. That's probably as good as it can get. enter image description here

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.