2

I am currently working on IMDB 5000 movie dataset for a class project. The budget variable has a lot of zero values.

They are missing entries. I cannot drop them because they are 22% of my entire data.

What should I do in Python? Some suggested binning? Could you provide more details?

3
  • Are they actually the number 0 or empty fields that should be considered 0?
    – Edeki Okoh
    Feb 20, 2019 at 22:38
  • 3
    Please explain why the zero values are a problem and provide some code which details what you have tried already.
    – Steve Boyd
    Feb 20, 2019 at 22:38
  • The variable is "budget", I have 22% budget input as zero. I cannot remove them directly. Is there anything I can do with it?
    – user10724844
    Feb 24, 2019 at 19:04

1 Answer 1

2

Well there are a few options.

  • Take an average of the non zero values and fill all the zeros with the average. This yields 'tacky' results and is not best practice a few outliers can throw off the whole.
  • Use the median of the non zero values, also not a super option but less likely to be thrown off by outliers.
  • Binning would be taking the sum of the budgets then say splitting the movies into a certain number of groups like say budgets over or under a million, take the average budget then divide that by the amount of groups you want then use the intervals created from the average if they fall in group 0 give them a zero if group 1 a one etc.
  • I think finding the actual budgets for the movies and replacing the bad itemized budgets with the real budget would be a good option depending on the analysis you are doing. You could take the median or average of each feature column of the budget make that be the percent of each budget for a movie then fill in the zeros with the percent of budget the median occupies. If median value for the non zero actor_pay column is budget/actor_pay=60% then filling the actor_pay column of a zeroed value with 60 percent of the budget for that movie would be an option.

  • Hard option create a function that takes the non zero values of a movies budgets and attempts to interpolate the movies budget based upon the other movies data in the table. This option is more like its own project and the above options should really be tried first.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.