Questions tagged [pandas]

Pandas is a Python library for data manipulation and analysis, e.g. dataframes, multidimensional time series and cross-sectional datasets commonly found in statistics, experimental science results, econometrics, or finance. Pandas is one of the main data science libraries in Python.

Filter by
Sorted by
Tagged with
-2 votes
0 answers
15 views

print pandas dataframe using to_markdown() without index column

I want to print a pandas dataframe using to_markdown() method. This is my code: dups = dups.loc[dups["records"] > 1] if len(dups) > 0: raise Exception( f"The following ...
HuLu ViCa's user avatar
  • 5,125
0 votes
2 answers
30 views

Counting value in Pandas

I have a dataframe in Pandas made like this: name domain rank1 rank2 rank3 ... rank_mean foo foo.com 1 1 2 ... 1.34 boo boo.it 5 12 ... 6....
gianluca gabriele's user avatar
2 votes
2 answers
38 views

how to merge the multiple dataframes sequentially?

Although I thought this question should be duplicated, I couldn't find the proper answer. I have some problems merging multiple dataframes sequentially. For example, I have four dataframes as below: ...
Ssong's user avatar
  • 306
-1 votes
0 answers
11 views

i didn't understand how the code pick the main csv file, because first i try to use second csv file data

\`BACKGROUND_COLOR = "#B1DDC6" import pandas import random from tkinter import * word1 = {} french = {} try: data = pandas.read_csv("data/words_to_learn.csv") except ...
Subham Pal's user avatar
0 votes
1 answer
26 views

DataFrame object is not callable. Yfinance

class SAnalysis: def __init__(self, scrip): self.scrip = scrip self.data = yf.Ticker(self.scrip) def get_history(self): data = self.data.history() return data def get_balance_sheet(...
Shimron Alakkal's user avatar
-7 votes
0 answers
32 views

a program that generates a detailed truth table for a specific logical expression entered by the user [closed]

program must strictly accept symbols like ∨, ∧, ⊕, ¬, →, ↔, ( and ) only. program must cater multiple number of variables as entered by the user. Include a column in the truth table for every term ...
Assel Florence Duncano's user avatar
0 votes
1 answer
40 views

Have an equivalent to pandas.merge with huge dataframes and no additional libraries

I am currently dealing with very big dataframes of a few millions rows each. However, my computer is often not powerful enough to perform the merge on both dataframes with pandas.merge I would like to ...
Sadface's user avatar
  • 11
-1 votes
0 answers
38 views

matplotlib.pyplot takes too long than plt.show, what to do? [closed]

I have a code in Python, where I want to create and present a specific graph. It is of numeric type and has a timestamp table. Fwd Packet Length Std Packet Length Std Fwd IAT Mean \ 1700177 ...
Davi Jorge's user avatar
-1 votes
0 answers
27 views

Skipping the first two rows in an excel when importing to pandas dataframe

I am trying to import 18 excel files that has the same structure, and I get a KeyError: '' when reading. I have figured out that it is probably because the first two lines messes up, and I want to ...
Snasegh's user avatar
  • 43
-1 votes
1 answer
47 views

panda, filter DataFrame and sum repeat Value

welcome everybody. i have dataFrame Like this, I want to calculate the value of the row repeated with the next row, and delete the previous row. For values calculated from Date, I want to put them in ...
Ben Davidsson's user avatar
0 votes
1 answer
45 views

Count Instances Until Value Repeats

I have a list/array/series of booleans, and I want a list of the same length that contains NaNs wherever there was a False value in the original, and on True indices I want to plug the number of False ...
Jon Nir's user avatar
  • 507
0 votes
0 answers
26 views

Pandas category is set to anything? after using of apply, but for-loop works [duplicate]

I have a complex df, where I use categories. As example here a small example with leads to the same problem. You see, df and df_cpy look initially like this: import pandas as pd df = pd.DataFrame({&...
teq508's user avatar
  • 13
-4 votes
1 answer
53 views

cut a DataFrame und get the minus value

I have a DataFrame and I need to cut it und get the min from the minus values: Than I need to export the local min value as array like min_lokal = [-1.3, -2, .....] and another qustion please: how I ...
Toma's user avatar
  • 1
0 votes
0 answers
9 views

Create columns based on values in one column, grouped by another colum [duplicate]

My dataframe is similar to below: User type Chris green Chris brown Chris green Paul brown Paul brown Mary yellow Mary green Paul green Chris yellow Mary brown ...
Ed Jefferies's user avatar
0 votes
0 answers
22 views

Cannot export a pandas DataFrame to Excel in Python [duplicate]

I need some help here, please. An excel file needs to be updated, not overwritten. In particular, I need to add data to an Excel file (row by row) from an external data source (data from my own ...
Lorenzo Castagno's user avatar
0 votes
0 answers
16 views

How to resampling OHLC Data in 15-Minute Intervals with Market Hours Consideration and Non-Repainting

I have OHLC (Open, High, Low, Close) data at a 3-minute timeframe, and I want to convert it into 15-minute OHLC intervals while considering market hours (from 9:15 AM to 3:30 PM). I'm using the ...
Shikhar Vaish's user avatar
0 votes
1 answer
32 views

.corr() method for dataframe not returning ideal values only returns either -1 or 1

Ideally it should be returning values between -1 and 1 for every cell except for the cells that have the same column name and row name those need to have a 1 value Tried replacing the NaN with 0 ...
Anirvesh Arcot's user avatar
1 vote
1 answer
24 views

pandas : how to slove "overflow encountered in accumulate return bound "

I would like to calculate cumulative rate of return using a dataframe sorted by date and fund name, nav(net asset value). I tried to solve it using the lambda function and the np.cumprod, but I faced ...
user3685918's user avatar
-1 votes
0 answers
12 views

installed the Anaconda 23.7.4 everything is working, I can use the jupyter labs but while lauching the jupyter notebook, I keep getting erro

Traceback (most recent call last): File "C:\Users\shukl\anaconda3\Lib\site-packages\notebook\traittypes.py", line 235, in _resolve_classes klass = self._resolve_string(klass) ^^^^^^^^^^^^^^^^...
Ashutosh shukla's user avatar
0 votes
0 answers
33 views

split pandas dataframe to list of dataframes based on column value difference [duplicate]

I need to split pandas dataframe to list of dataframes in which value difference between FIRST row value of 'address' column and LAST row value is not greater than 100 What i do now is iterate over ...
HMM's user avatar
  • 1
0 votes
1 answer
20 views

How to combine imported csv to dataframes based on conditional boolean variable?

I'm new to python and am trying to import a large csv file depending on where a checkbox from a user GUI input is checked to True and combine those csv files into a single dataframe. All dataframes ...
SharpeShark's user avatar
0 votes
3 answers
60 views

Date time format - Pandas

I have a Pandas dataframe in which one of the column contains date. However the format of the date is not consistent. Have different type of date formats in the column. When I read the data in ...
Sunil's user avatar
  • 141
0 votes
0 answers
24 views

Melt pandas with two values [duplicate]

I have a data frame with lower and upper bound values over different simulations: Out[71]: lb1 ub1 lb2 ub2 lb3 ... lb11 ub11 lb12 ub12 target 0 0.0005 5.0005 0....
therickster's user avatar
0 votes
2 answers
72 views

Filter Pandas dataframe by columns that are a substring of a string

I am trying to filter a dataframe using the condition that string values in a column are a substring of a string outside of the dataframe. Example below: df = [['a', 'b', 'c'], ['hello', 'bye', 'hello'...
randomwerg's user avatar
0 votes
0 answers
20 views

Expected 2D array, got 1D array instead using SciKit for image classification

I am trying to train a Gaussian Naive Bayes model using scikitlearn. My database contains 656 images along with labels from a csv file. Below is the code I used to both import the images + labels as ...
Ginger's user avatar
  • 1
2 votes
2 answers
30 views

Pandas: How to self-join cross-functionally

I have this dataframe below for airline flights: df = pd.DataFrame({ 'ORIGIN_x':["LGA","ORD"], 'DESTINATION_x':["ORD","LGA"], '...
user19913336's user avatar
-2 votes
0 answers
21 views

Cannot replace non-alphanumeric characters in a string in Pandas [duplicate]

Following a youtube tutorial: https://www.youtube.com/watch?v=bDhvCp3_lYw Stuck on the 'phone numbers' part. I wanted to cleanse the Phone_Number column to replace the specified characters with ...
Jubed96's user avatar
1 vote
1 answer
24 views

how to use @pandas_udf of pyspark for groupby.agg

I am using the pandas API on Spark. And I am using the groupby.add operation. I found a similar issue, but the solution does not work for me. I also checked the official docs. But the docs does not ...
stevelu's user avatar
  • 11
0 votes
1 answer
44 views

Reading csv file in pandas with double quotes and commas separating fields

I have the following csv file: Data: "Id","Student","Job","Satisfied" "10132","Bachelor","Data Engineer(AWS, GCP)","Yes"...
Gonçalo Fontes's user avatar
1 vote
2 answers
34 views

Derive a bool column based on groupby with current row compared to previous

I'm trying to calculate a row number over several groups where the number increments only if the current row has different classification values vs. the previous. I'm thinking calculating a boolean ...
Mark's user avatar
  • 79
0 votes
2 answers
57 views

Why time format changes when write df to excel?

When i try to write the df to one excel sheet the column with dates print with dates and time like 2007-01-02 00:00:00. I want the date to stay with format 2007-01-02 (with NO time) Why this happens? ...
dimzev's user avatar
  • 181
1 vote
1 answer
45 views

pandas groupby, transform and join

I have a dataframe and I would like to groupby and join the strings of a column together. So something like the below. df = pd.DataFrame({ 'id': [1, 1, 2, 2, 3, 3], 'txt': ['sth', 'sth else', ...
corianne1234's user avatar
1 vote
1 answer
52 views

How to plot trips durations while considering their deadlines?

My input is a dataframe : df = pd.DataFrame({'blk1': {'a': '12h45', 'b': None, 'c': None, 'd': None}, 'blk2': {'a': None, 'b': None, 'c': '13h15', 'd': '15h15'}, 'blk3': {'a': None, 'b': '12h10', 'c'...
VERBOSE's user avatar
  • 227
-3 votes
4 answers
71 views

How to filter a dataframe by consecutive value and condition?

I have a pandas dataframe like this: df = pd.DataFrame({'a':[1, 1, 1, 2, 1, 3, 1, 1, 4, 4, 5, 5, 5, 6, 6, 3]}) >>> df a 0 1 1 1 2 1 3 2 4 1 5 3 6 1 7 1 8 4 9 4 10 5 ...
skywave1980's user avatar
1 vote
1 answer
50 views

Pandas get the combined sum of each row and colum

I'm trying to add the "vertical" and "horizontal" sum for each cell. So far I've tired to get the sum of each row and column. But like this I'd have to iterate over every value ...
Lukas Neumann's user avatar
-3 votes
0 answers
31 views

If sales price is greater/less than 2 standard deviation from the mean sale price then delete such rows? [closed]

print("Before deleting outliers length = " , ??) target = ?? target_mean = ?? target_sd = ?? data = ?? print("After deleting outliers length = " , ??) I can't ...
rey02's user avatar
  • 1
1 vote
3 answers
46 views

How to merge pandas columns in a comma separated way

I have a pandas dataframe like below Id Other columns Param1 Param2 Param3 1 29 -- 2 29 35 3 29 74 4 11 34 I want to merge the last three columns using comma as a separator to make the ...
Suhas's user avatar
  • 7,999
2 votes
1 answer
29 views

How to solve a KeyError problem in Pandas

enter code here Gender Value Year Measure Name Ma girl 0 1997 Rank Nabiha girl 1204 2014 Rank Muad. boy 1749 2009 ...
elksie5000's user avatar
  • 7,122
1 vote
1 answer
34 views

Why does dividing columns by another column yield NaN?

There is a Pandas dataframe df. df.info() --- <class 'pandas.core.frame.DataFrame'> Int64Index: 3 entries, 1 to 3 Data columns (total 3 columns): # Column Non-Null Count Dtype --- ------ -...
mon's user avatar
  • 19.1k
0 votes
0 answers
27 views

how to convert pandas dataframe to binary pandas dataframe? [duplicate]

I have a pandas dataframe: 1. sc_RNA seq dataframe (n_cells, a_genes): contains gene expression matrix. Now I need to convert these pandas dataframes into binary pandas data frames, where, if there ...
Irin Sultana's user avatar
2 votes
2 answers
46 views

How to dynamically create dataframes with a for loop

My code currently looks like this: df_1 = portfolio_all[0].rename(columns={'Close': 'Close_1'} ) df_2 = portfolio_all[1].rename(columns={'Close': 'Close_2'} ) df_3 = portfolio_all[2].rename(columns={'...
adrCoder's user avatar
  • 3,177
0 votes
0 answers
58 views

How to extract element from a numeric list which is contained in an array object

I was trying to extract a data point from a cell in a pandas data frame, the data appears to be a list: [1,3,4,6...]. I want to use it as a list or numpy array. I extracted this data by indexing this ...
Duke He's user avatar
0 votes
1 answer
58 views

python3: plot x in function of y

I want to plot a curve graph with in abcisse temperature of patients and in y axes the pourcentage. I did the code below, but i want to update it to get like the curve in the picture attached. Please ...
tamo007's user avatar
  • 277
0 votes
0 answers
40 views

Stop loss mechanics while backstesting on time series

I'm presented with the following problem. I have time series data of prices and through the use of the "loc" method I was able to create a new column "trade" that stores the close ...
Gus's user avatar
  • 193
0 votes
1 answer
54 views

Having an issue converting pandas dataframe 'object' column into a float

I am trying to convert 2 columns in a pandas dataframe into float values as they are currently listed as 'object'. When I try to convert using .astype() the one column is then placed with value 0 ...
just_trying_'s user avatar
0 votes
0 answers
34 views

How to use dynamic (growing) analytics data with Pandas

I want to use Pandas for statistical analysis of analytics data. This data is not fixed, it's growing every day. Saving the analytics data in a database just to convert it to the dataframe every time ...
ddofborg's user avatar
  • 2,038
0 votes
0 answers
12 views

How to locate all rows of a pandas data frame where 'date' is in a particular set? [duplicate]

I have a set of dates in an array called 'signals'. I also have a pandas dataframe, with each row containing a different date's numbers. I want to pull all the 'Adj Close' values from a particular set ...
ak231472's user avatar
-1 votes
0 answers
23 views

produce a pivot table (index is item, columns are users and item ratings are values) for recommendation systems [duplicate]

I am trying to produce a pivot table from a book rating table that contains user ID as pivot table index, ISBN as columns and ratings as values. I trying to create a pivot table to make rating matrix(...
Fainan El sisi's user avatar
0 votes
0 answers
29 views

How to determine the chunk limit based on the contents of the file I want to read in chunks?

I want to work with a huge csv file, but I get OOM errors when trying to read it into a dataframe in one go. The solution would be to read it in chunks, but the issue is that I want to calculate ...
Zegher V's user avatar
  • 123
-1 votes
0 answers
42 views

How to group values by category

How I can group the table by category (percentage should be calculated per group. I need to calculate the % of the category from the total number that got into the TOP 15 (746 markets) and show it on ...
unifly's user avatar
  • 1

1
2 3 4 5
5670