2

A sample of my input looks like this , Too give a brief understanding. This is a matrix of User ratings where the columns are product numbers.

User    95  94  97  101 99  87  98  86  103 105 106 100 92  89  91  96
27669   15  19      2   1                                           
27670               12  9   61  51  69  30  32  30  10              
27671                                                   49  7   29  
27672   11  73                      43  47  12                      6
27673           8       14      11                  11              
27674                   1   55                                      
27675                       9           9       10      30  29  11  
27676                               29  50  50                      
27677           31      25          28                              
27678           9   9       27  7   49              7               
27679       28                      27                              7
27680   52  47              40      55      52                      
27681                               11                  9       15  
27682   28  50              27                                      49
27683           9   9   10      8   12              9   10  8       

I am using the code below to read the csv and convert it to a Dictionary

import csv
reader= csv.DictReader(open('test_23.csv'))
next(reader)
users = {}
for row in reader:
    key=row.pop('User')
    if key in users:
        pass
    users[key]=row
print users        

A sample out put of my code is as below

{'31550': {'91': '', '88': '', '89': '', '97': '', '103': '', '100': '', '86': '', '87': '', '101': '', '95': '', '105': '', '99': '', '98': '', '102': '69', '90': '', '93': '', '92': '', '106': '', '94': '', '104': '', '96': ''}, '29443': {'91': '90', '88': '', '89': '69', '97': '', '103': '65', '100': '', '86': '', '87': '74', '101': '', '95': '', '105': '68', '99': '', '98': '', '102': '', '90': '', '93': '', '92': '', '106': '70', '94': '', '104': '74', '96': ''}, '32103': {'91': '', '88': '', '89': '', '97': '', '103': '', '100': '3', '86': '', '87': '', '101': '28', '95': '', '105': '65', '99': '', '98': '4', '102': '', '90': '', '93': '', '92': '', '106': '69', '94': '', '104': '68', '96': ''}, '29687': {'91': '', '88': '9', '89': '7', '97': '', '103': '8', '100': '', '86': '', '87': '', '101': '', '95': '', '105': '', '99': '', '98': '', '102': '9', '90': '25', '93': '', '92': '', '106': '27', '94': '', '104': '9', '96': ''}, '29444': {'91': '69', '88': '60', '89': '71', '97': '', '103': '', '100': '', '86': '51', '87': '', '101': '', '95': '19', '105': '', '99': '', '98': '', '102': '', '90': '', '93': '', '92': '', '106': '', '94': '35', '104': '', '96': '18'}, '28224': {'91': '', '88': '18', '89': '18', '97': '', '103': '', '100': '', '86': '', '87': '17', '101': '', '95': '17', '105': '', '99': '', '98': '', '102': '', '90': '17', '93': '', '92': '19', '106': '', '94': '19', '104': '', '96': '17'}, '31783': {'91': '', '88': '', '89': '', '97': '', '103': '50', '100': '', '86': '', '87': '', '101': '47', '95': '55', '105': '', '99': '', '98': '54', '102': '', '90': '', '93': '', '92': '', '106': '', '94': '', '104': '53', '96': ''},

Is there a way so that Dictreader can skip the blank values ?

Also ,Is there a way to remove the single quotes for all values ?

I have tried quoting=csv.QUOTE_NONE but that doesnt work .

3 Answers 3

4

You could use defaultdict and do it all in one step.

#!/usr/bin/python

from csv import DictReader
from collections import defaultdict

users = defaultdict(dict)

for row in DictReader(open('./file.csv', 'rb')):
    key = row.pop('User')
    tmp_dict = {int(k):int(v) for k,v in row.iteritems() if v != ''}
    users[key] = tmp_dict

>>> users
{'27671':{89:7, 91:29, 92:49}, ... }
2
  • thanks , this seems elegant . but not why I get this in my output defaultdict(<type 'dict'>, {{'27671':{89:7, 91:29, 92:49}, ... }
    – dagan
    Commented Aug 5, 2015 at 4:05
  • you can type dict(users) if you dont like the way the output looks, but it is the same thing ... just a dictionary of dictionaries.
    – o-90
    Commented Aug 5, 2015 at 4:09
0

The single quotes are a python thing so you would have to first convert to a string and then remove them yourself:

str s = str(users).replace("'","")

I don't know a default way to tell the CSV reader not to read blanks but you could just loop over the dictionary and remove all values that are blank:

users = {k: v for k,v in users.iteritems() if v != ""}
3
  • str s = str(users).replace("'","") ^ SyntaxError: invalid syntax I get the aboe error . I am using python 2.7
    – dagan
    Commented Aug 5, 2015 at 3:10
  • sorry drop the first str
    – fdsa
    Commented Aug 5, 2015 at 3:10
  • tried both . both quotes and blank values still remain =)
    – dagan
    Commented Aug 5, 2015 at 3:19
0
import csv
reader= csv.DictReader(open('test_23.csv'))
users = {}
newlist = {}

for row in reader:

    key =row.pop('User')
    if key in users:
        pass      


    for val in row:
        if row[val] != '':
            newlist[val] = int(row[val])
        else:
            pass

    users[key]=newlist

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.