Questions tagged [pandas-loc]
For questions about the [pandas] loc indexer. This tag should only be used for questions primarily about pandas loc or the behavior of this indexer. Be sure to also include the [pandas] tag on your question.
385
questions
0
votes
1
answer
71
views
Issue setting entire column (and changing dtype) with .loc[:,'col'] in pandas 1.5+
I often use df.loc[:,'col'] = arr to reassign columns rather than df['col'] = arr. This was a recommended practice to avoid the fact that, prior to Copy-on-Write changes in pandas 2.0, we couldn't ...
0
votes
1
answer
48
views
Selecting Rows that only match the column values in another data frame in Python [duplicate]
I have a data frame:
>df_idbank.dtypes
DATASET object
IDBANK object
KEY object
FREQ object
INDICATEUR ...
1
vote
1
answer
23
views
How to stop overwriting values with NA when referenced index missing?
I'm trying to copy the 'Name' column values from two different dataframes stocks and major_indices to another dataframe mport if the index value of mport is present in stocks and major_indices ...
0
votes
0
answers
13
views
In Python 3.8.8, why does read_csv(StringIO()) behavior change depending on where the string is populated and passed to the function?
Depending on whether I declare a string variable within a function, or pass it from outside the function, changes the behavior of the function. Specifically, declaring a string variable and converting ...
2
votes
1
answer
97
views
Replace values in a dataframe column, based on another dataframe
I have two DataFrames:
df1
PRA_Kod;WorkerId
1049;9024
0014;60260
0608;10506
20270/9;20270
9511;64473
0639;60264
0767;10509
.. .. .. . .
df2
WorkerId;Day;Time;W
1049;11;u.wyp;WE
1049;14;u.wyp;WE
64454;...
0
votes
1
answer
56
views
Assign subset of rows to a Dataframe using loc / iloc
I am currently having a database with a set of columns X. I am willing to "update" , using .loc (or .iloc) the content of a row for a certain subset of columns (which we can call Y) from X, ...
0
votes
2
answers
39
views
How can I sum columns in a data frame for rows within a range of values that increases over a specified interval?
Edit: I tried the following code on my actual data and I am getting the incorrect ranges for column 1.
MAX_SIZE = 10_000_000 # max chromosome size
bins = list(range(0, MAX_SIZE, 10_000))
bins[0] = 1
...
0
votes
0
answers
43
views
Why is df.loc['row', 'col'] the same as df['col'].loc['row']?
I can see theyre both referencing the same cell/value, but why does df['col'].loc[row] even work? I learned to use loc as df.loc[rowname,colname].
I saw it in a list comprehension:
fb['Direction'] = [...
0
votes
0
answers
20
views
Accessing pandas datframe within function returns an Attribute error: 'function' object has no attribute 'loc'
I wrote a script to update a pandas dataframe by accessing the relevant index using .loc and assigning a desired value to it. This worked fine, but as I put these lines of script into a function which ...
1
vote
2
answers
69
views
Apply definition on multiple columns based on conditions- python
i have dataframe that i want when one column equal to '02' apply the definition to 5 others columns. this is my code:
mask = (df['A']=='02')
z_valid = df[mask]
df.loc[((mask) & (df['B'] !=0)), 'B']...
1
vote
1
answer
61
views
How filter an inner dataframe based on the row of an outer dataframe
**My goals: **
filter an inner dataframe based on the row (then the values) of an outer dataframe
Let's consider three dataframes:
first a list of shop with detail about the shop:
Shop = pd....
1
vote
2
answers
77
views
KeyError when filter pandas dataframe by column with particular key:value pair
My df looks like the following
col1
col_x
...
{"key_x":"value1"}
...
None
...
{"key_x":"value2"}
How to select all items with {"key_x":"...
0
votes
2
answers
91
views
time series filtering with .loc with two index DataFrame
I have a time series pandas (df) table with many columns and with 2 indexes "date" and "ticker". I would like to use df.loc to select a specific range of dates , let say ("...
1
vote
1
answer
294
views
Why does converting a column from datetime to string in pandas not work using .loc?
I'm using pandas to load data from Excel with the resulting DataFrame containing both strings and dates. The columns containing strings are of dtype "object" while the date-columns are of ...
1
vote
1
answer
37
views
What's the difference between these 2 methods of Series filtering? (with or without lambda)
I have a data Series called Snow (the amount of snow in different months of the year).
These two lines of code produce the same results (at least seems so!)
So I just wanted to know the difference.
...
-1
votes
1
answer
25
views
Trying to find rows which end in .99 cents // Python // Multiple Conditionals // .loc // Manipulation inside .loc
Trying to find rows of a column which end in .99 cents
len(df_official.loc[(df_official.price_duration_days >= 7) & (df_official.shop.isin(official_stores)) & (df_official['Days From ...
0
votes
1
answer
47
views
.isin() returning a blank dataframe
My DataFrame:
Trying...
Nope...
Still nope...
AND YET...
Any idea why I can't filter this dataframe by a list using isin?
I'm expecting it to return one or more rows where the Chord Notes column ...
0
votes
1
answer
68
views
TypeError: unsupported operand type(s) for &: 'float' and 'bool' pandas
Actually i was trying to copy few columns from a dataframe to another dataframe but got an error unsupported operand type for & float and bool pandas
df
df
mask_actor = df[df.cast.str.contains(&...
-2
votes
1
answer
35
views
using an .loc and If statment
enter image description here
Use the .loc method, select the column rating for the rows of df where company_location equals "U.K."
Store it in a variable called uk_ratings.
uk_ratings = ...
0
votes
2
answers
183
views
Faster method for complex .loc in large Pandas dataframe?
For each row, I need to get a) the number of and b) the mean score of all rows 28 days prior to that row's date with the same member_ID.
df:
member_ID score date past28
0 ...
0
votes
0
answers
27
views
How to split dataframe based on first value only while keeping the rest of the corresponding data?
I have a dataset which looks like this:
User Session Score
1 1 5
2 4
2 1 2
2 5
3 1 5
2 4
...
1
vote
1
answer
27
views
How do I change a pandas row value, for a certain column, for a certrain date (datetimeindex) in a dataframe?
I have a pd like this:
DATE delivery
2020-01-01 1
2020-01-01 11
2020-01-01 10
2020-01-01 9
2020-01-01 8
..
2023-03-02 5
2023-03-02 4
2023-03-02 3
...
0
votes
1
answer
55
views
Unexpected results using pandas .loc - trying to concatenate 2 columns based on a condition
I have a df where I am trying to merge 2 columns based on a condition.
Create df
df = em_df[['Redcap_Case_num', 'EV_EM', 'COMP_EM', 'EV_RND', 'COMP_EM_RND'] ].head(3)
df.to_clipboard(excel = False, ...
1
vote
1
answer
74
views
assigning an object to 1 entry of a pandas dataframe with 2 methods
I would like to share a strange behavior of pandas, and find out the reason :
I assign a numpy array as an object to 1 element (cell, entry) of a pandas dataframe in 2 different ways :
first create a ...
0
votes
1
answer
55
views
Pandas loc not filtering properly
I am using pandas to read a csv file and then trying to filter by some condition to remove blank/null/NaN values and then sorting. it still does not filter and returns all the contents. Below is the ...
0
votes
2
answers
30
views
How to search for a specific date within concatenated DataFrame TimeSeries. Same Date would repeat several times in a merged df
I downloaded historical price data for ^GSPC Share Market Index (S&P500), and several other Global Indices. Date is set as index.
Selecting values in rows when date is set to index works as ...
0
votes
0
answers
28
views
.loc only returning first value in a list instead of the full list
I have a data frame and I'm trying to return all the matching values in one column based on another column using loc. The dataframe looks like this.
Col1
Col2
Alpha
Bravo
Alpha
Charlie
Delta
...
0
votes
2
answers
52
views
How do you replace values in one column with values in another when condition is not met with Python
I have some inconsistent data with 5 columns. The two I'm focussing on are - 'Account Name' and 'User ID'. All the Account names should contain a 7 character string. If it doesn't I want to replace if ...
0
votes
0
answers
60
views
df = df.loc yield empty DataFrame in a loop [Python] [Pandas]
Sorry if this is a dumb question, I'm a very noob coder.
I'm trying to itinerate through a folder and create DataFrames for each file (then saving it).
Thing is, I changed nothing from when it was not ...
0
votes
0
answers
11
views
Reshaping a Pandas Dataframe to set values from rows to columns [duplicate]
My question is about rearranging the df1 dataframe to match df2. I was thinking in filtering by Num followed by pd.concat(). My question is whether there is a more clever way to do it. Thanks in ...
0
votes
2
answers
61
views
Remove date from datatime index in pandas
I want to remove the date from datetime function in pandas and the following code works just fine.
df= pd.read_csv('data.csv')
df['Value']= df.Value.astype(float)
df['Time'] = pd.to_datetime(df['Time']...
2
votes
1
answer
31
views
Mapping value based on two dataframe (error: an only compare identically-labeled Series objects)
Table t1
id mins maxs
0 43852 11 11
1 63087 14 15
2 63106 14 15
3 63155 14 15
Table t2
idx cons
0 1 1.00
1 2 0.95
2 3 0.90
3 4 0.85
4 5 0.80
5 6 ...
0
votes
0
answers
23
views
Pandas Module: How to edit specific row values without affecting rest of data frame? (Error when using .at and .loc)
I am trying to update a data frame with details of games played by NBA teams. I am trying to add an ELO rating based on the results of the game and update it across the data frame.
I am using the ...
1
vote
1
answer
44
views
Filter data with it correponding value on another column
I have the following datfarame
ind
Q
haul1
haul2
haul3
haul4
haul5
em1
em2
em3
em4
em5
1
1
medio
NaN
NaN
NaN
NaN
0.165850
NaN
NaN
NaN
NaN
2
1
medio
NaN
NaN
NaN
NaN
0.244922
NaN
NaN
NaN
NaN
3
1
...
0
votes
0
answers
27
views
Error during insertion of a new row in a DataFrame
I made a dataframe from a dictionary and set one of its columns as my index. While inserting new a row, I get this error:
docdf=docdf.loc[sno_value]=[name_value,age_value,special_value,contact_value,...
0
votes
1
answer
61
views
Python Pandas .loc need count of column; null for one column and value in another column
I am trying to create a pandas .loc statement that would get the count of rows in one column where the value is not null and where the value of a different column is 1.
(I could do this same query in ...
0
votes
1
answer
50
views
In Python, what is the difference between df["row_name"] and df.loc["row_name"]
I am trying to add another row to my data Frame
When I use df["new_row"] = [5, True, "joe", 20] , I get the error
ValueError: Length of values (4) does not match length of index (...
0
votes
2
answers
236
views
How do I return the rows of DataFrame where every Country in each Continent has a Population of less of than 100?
df = pd.DataFrame({
"Continent": list("AAABBBCCD"),
"Country": list("FGHIJKLMN"),
"Population": [90, 140, 50, 80, 80, 70, 50, 125, 50]})
...
0
votes
0
answers
14
views
Python Pandas DateFrame - SettingWithCopyWarning: [duplicate]
import pandas as pd
class OrdersList:
def __init__(self):
self.shopping_list = pd.DataFrame()
self.shopping_list = self.shopping_list.assign(Order=0, Type=0, Price=0, Quantity=0)
...
1
vote
1
answer
42
views
Pandas - starting iteration index and slicing with .loc
I'm still quite new to Python and programming in general. With luck, I have the right idea, but I can't quite get this to work.
With my example df, I want iteration to start when entry == 1.
import ...
0
votes
2
answers
27
views
How to update at a specific row, after finding the same value in two tables
import pandas as pd
data1 = [['xx'], ['4']]
data2 = [['4', 'x0'], ['aa', 'bb'], ['cc', 'dd']]
df1 = pd.DataFrame(data=data1, columns=["isin"])
print(df1)
df2 = pd.DataFrame(data=data2, ...
0
votes
1
answer
22
views
Locating column of dataframe in a dataframe
I have a dataframe in a dataframe similar to this one (my real one is much larger)
df_peter = pd.DataFrame({"height": [50,np.nan,65], "weight": [20,25,27]})
df_anna = pd.DataFrame({...
0
votes
1
answer
41
views
Pandas - Blanking out values of one column if value of another column has a specific value
I have the following DataFrame:
import pandas as pd
df = pd.DataFrame({'Status': ['','To Do', '','Completed', 'To Do', 'In Progress', 'Completed'],
'Date': ['','9/1/2022','','12/5/2019','8/12/2020','...
0
votes
2
answers
301
views
Pandas .loc with sort_values assignment does not work
I am trying to assign a value and/or a series of values to a slice of a pandas dataframe selected using .loc after sorting values.
For reference, this is the code I am trying to run, with a set string ...
1
vote
1
answer
323
views
Filter variable not working properly in Pandas
I have a very large dataset, and I am applying multiple filters on many columns. In order to make the code more readable, I assign the filters to some variables - but I noticed that although the ...
1
vote
2
answers
828
views
.loc will not work specifically on a 'Date' column pandas
I have a dataframe in pandas that originally had a parsed date column with values such as: 2020-05-11 02:23:00.
From that I created a 'Date' column using df['Date'] = df.parsed_date_column.dt.date ...
0
votes
5
answers
83
views
Pandas: Transferring column values to separate columns
Relatively new to Python here. I have a dataframe with three columns: Teams, Year and Medals_Won like so:
Teams Year Medals_Won
A 2009 Gold
A 2010 Silver
A 2011 Silver
...
0
votes
0
answers
36
views
Using only .loc[], how can you extract a single index-column in a multi-indexed df?
I know there are many posts on the general theme of extracting a single index-column from a multi-indexed df. I hope this question can be differentiated, as I haven't found something answering this ...
0
votes
2
answers
94
views
Using Pandas df.loc
I have a DataFrame of a csv file which is being read by pandas. What I am attempting to do is use df.loc to add a new column but only insert values into the column when values from another column, ...
0
votes
1
answer
917
views
Combine a variable and a string for naming a column in pandas
I have a dataframe with several column which have similar names, something like col_1, col_2, col_3.
I am trying to create a for loop to go over the 3 columns and want to use .loc inside the loop to ...