102

I'm trying to set the entire column of a dataframe to a specific value.

In  [1]: df
Out [1]: 
     issueid   industry
0        001        xxx
1        002        xxx
2        003        xxx
3        004        xxx
4        005        xxx

From what I've seen, loc is the best practice when replacing values in a dataframe (or isn't it?):

In  [2]: df.loc[:,'industry'] = 'yyy'

However, I still received this much talked-about warning message:

A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead

If I do

In  [3]: df['industry'] = 'yyy'

I got the same warning message.

Any ideas? Working with Python 3.5.2 and pandas 0.18.1.


EDIT Jan 2023:

Given the volume of visits on this question, it's worth stating that my original question was really more about dataframe copy-versus-slice than "setting value to an entire column".

  • On copy-versus-slice: My current understanding is that, in general, if you want to modify a subset of a dataframe after slicing, you should create the subset by .copy(). If you only want a view of the slice, no copy() needed.
  • On setting value to an entire column: simply do df[col_name] = col_value
2
  • 7
    You must have done something to df prior to calling df.loc[:,'industry']='yyy' as what you posted should've worked. Basically the warning gets raised if you took a slice or sub-section of your starting df which you didn't show
    – EdChum
    Jun 23, 2017 at 14:36
  • Does this answer your question? How to deal with SettingWithCopyWarning in Pandas Oct 27, 2022 at 20:39

11 Answers 11

150

You can use the assign function:

df = df.assign(industry='yyy')
5
  • 5
    This method worked for me without giving me warnings.
    – Miladiouss
    Aug 23, 2019 at 22:00
  • 2
    this should be the answer. a good answer is useful to everybody. not just the one who asked.
    – greendino
    May 22, 2020 at 4:01
  • How can I use this approach with a multiindex?
    – thinwybk
    Jun 22, 2020 at 12:02
  • 2
    This didn't throw a warning. Jul 9, 2020 at 20:54
  • Just to note, if your column name has spaces like "my industry" then it would be df = df.assign(**{"my industry": 'yyy'}) Jul 13 at 20:05
38

Python can do unexpected things when new objects are defined from existing ones. You stated in a comment above that your dataframe is defined along the lines of df = df_all.loc[df_all['issueid']==specific_id,:]. In this case, df is really just a stand-in for the rows stored in the df_all object: a new object is NOT created in memory.

To avoid these issues altogether, I often have to remind myself to use the copy module, which explicitly forces objects to be copied in memory so that methods called on the new objects are not applied to the source object. I had the same problem as you, and avoided it using the deepcopy function.

In your case, this should get rid of the warning message:

from copy import deepcopy
df = deepcopy(df_all.loc[df_all['issueid']==specific_id,:])
df['industry'] = 'yyy'

EDIT: Also see David M.'s excellent comment below!

df = df_all.loc[df_all['issueid']==specific_id,:].copy()
df['industry'] = 'yyy'
3
  • 10
    This can (I think) also be done without the copy module by using the Pandas copy function (e.g. df=df_all.loc[df_call['issueid']==specific_id,:].copy())
    – David M.
    Apr 29, 2019 at 14:11
  • 1
    also df = pd.DataFrame(df) makes a copy. Oct 14, 2022 at 7:23
  • sorry, if it possible, can yo add to your answer explanation of another answer df.iloc[:]['industry'] = 'yyy' is it correct? and why ) Dec 17, 2022 at 10:02
34
df.loc[:,'industry'] = 'yyy'

This does the magic. You are to add '.loc' with ':' for all rows. Hope it helps

1
  • 12
    This still throws the 'SettingwithaCopy' warning. Jul 9, 2020 at 20:54
14

You can do :

df['industry'] = 'yyy'
7
  • 23
    Still the same warning message. Jun 23, 2017 at 14:12
  • How is constructed your dataframe ? Because I don't have this warning when doing this with a random dataframe.
    – HH1
    Jun 23, 2017 at 14:14
  • 1
    df is taken from a broader dataframe df_all. something like df = df_all.loc[df_all['issueid']==specific_id,:]. I think you've got it, because when I do df_all['industry']='yyy' I don't see this message. But I don't know why df is not a "normal" dataframe. Jun 23, 2017 at 14:19
  • 1
    it's your df = df_all.loc[df_all['issueid']==specific_id,:] ; try to use df = df_all[df_all['issueid']==specific_id] instead
    – HH1
    Jun 23, 2017 at 14:22
  • Yes got it. Care to explain what's the difference between the two? Does it have something to do "a copy"? Jun 23, 2017 at 14:26
4

Assuming your Data frame is like 'Data' you have to consider if your data is a string or an integer. Both are treated differently. So in this case you need be specific about that.

import pandas as pd

data = [('001','xxx'), ('002','xxx'), ('003','xxx'), ('004','xxx'), ('005','xxx')]

df = pd.DataFrame(data,columns=['issueid', 'industry'])

print("Old DataFrame")
print(df)

df.loc[:,'industry'] = str('yyy')

print("New DataFrame")
print(df)

Now if want to put numbers instead of letters you must create and array

list_of_ones = [1,1,1,1,1]
df.loc[:,'industry'] = list_of_ones
print(df)

Or if you are using Numpy

import numpy as np
n = len(df)
df.loc[:,'industry'] = np.ones(n)
print(df)
4

This provides you with the possibility of adding conditions on the rows and then change all the cells of a specific column corresponding to those rows:

df.loc[(df['issueid'] == '001'), 'industry'] = str('yyy')
3

Seems to me that:

df1 = df[df['col1']==some_value] will not create a new DataFrame, basically, changes in df1 will be reflected in the parent df. This leads to the warning.
Whereas, df1 = df[df['col1]]==some_value].copy() will create a new DataFrame, and changes in df1 will not be reflected in df. The copy method is recommended if you don't want to make changes to your original df.

3

For anyone else coming for this answer and doesn't want to use copy -

df['industry'] = df['industry'].apply(lambda x: '')
2

Only use them instead:

df.iloc[:]['industry'] = 'yyy'

remember: this only works with exist columns in dataframe

this for people who didn't work .loc

1
  • Thanks, but can you explaine - why? Dec 17, 2022 at 10:01
1

I had a similar issue before even with this approach df.loc[:,'industry'] = 'yyy', but once I refreshed the notebook, it ran well.

You may want to try refreshing the cells after you have df.loc[:,'industry'] = 'yyy'.

-3

if you just create new but empty data frame, you cannot directly sign a value to a whole column. This will show as NaN because the system wouldn't know how many rows the data frame will have!You need to either define the size or have some existing columns.

df = pd.DataFrame()
df["A"] = 1
df["B"] = 2
df["C"] = 3

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.