28,108
questions
0
votes
1
answer
20
views
How to calculate difference between dates by group in SAS
I have data that resembles the below, and need to calculate the average time (in months) between dates in the Event_date column, grouped by Subject_ID. In other words, for each subject in the data, I ...
1
vote
2
answers
46
views
Find a value based on all its present combinations with a value in another field in Postgres
I have a PostgreSQL table:
create table test(type_id, m_id)as values
(1, 123)
,(2, 456)
,(3, 123)
,(4, 123)
,(2, 456)
,(1, 789)
;
Basically, one m_id could have ...
-2
votes
0
answers
30
views
group a dataframe by n columns and add a column containing the group number [duplicate]
Sample dataframe:
import pandas as pd
data = {
'Name': ['Alice', 'Bob', 'Alice', 'Mark', 'Bob', 'Sarah', 'Frank', 'Grace', 'Bob', 'Hank'],
'Department': ['HR', 'Engineering', 'Marketing', '...
0
votes
1
answer
29
views
Pandas plot grouped-by frame
I have the following code using exported data from Apple Health. The data is obtained by exporting the Apple Health data to an export.zip file, and then you'll see in the code I'm extracting the ...
0
votes
1
answer
48
views
Aggregation: Category missing in SQL subquery
my base data:
Process ID
Location
Date
Timeliness
2030608
New York
May 24
in time
2067393
Ohio
May 24
overdue
1329306
Ohio
May 24
in time
1740814
Ohio
June 24
overdue
1924676
Chicago
May 24
...
0
votes
1
answer
35
views
Matplotlib: Stacked Bar Diagram
Image in this Image I want to split each bar as a stacked graph.
Sample data:
season venue toss_decision
2011/12 Shere Bangla National Stadium, Mirpur bat ...
0
votes
3
answers
87
views
MYSQL GROUP BY most frequent value for each ID
I have following 2 tables: Sales and Menu.
SALES TABLE
customer_id
product_id
A
1
A
2
A
2
A
3
A
3
A
3
B
1
B
1
B
3
B
3
B
2
B
2
C
3
C
3
C
3
MENU TABLE
product_id
product_name
1
sushi
...
0
votes
2
answers
47
views
Why does pandas Grouper groups edge not align with expectations?
Given a simple series with a DateTime index, I want to group the Series into 4 month bins:
import pandas as pd
import numpy as np
# Create a sample Series with a datetime index
index = pd.date_range('...
1
vote
1
answer
43
views
Pandas groupby is changing column values
I have a multiindex Pandas DataFrame and I'm using groupby to extract the rows containing the first appearances of the first index.
After this operation, however, the output column values does not ...
0
votes
1
answer
31
views
Deducting Values from one DataFrame based on Values from another Table (With common Columns)
I have a simple case where I have two tables, one represents demand and another is supplies. I want to pick up an order, if the order is from the same location and Item as supplies then substract the ...
2
votes
3
answers
102
views
Group elements in dataframe and show them in chronological order
Consider the following dataframe, where Date is in the format DD-MM-YYY:
Date Time Table
01-10-2000 13:00:03 B
01-10-2000 13:00:04 A
01-10-2000 13:00:05 B
01-10-2000 13:00:06 A
01-...
0
votes
1
answer
56
views
Pandas groupby more columns + rolling + adding new column
I have a dataset of transactions with for example the columns: UserID, IBAN, Timestamp, Amount.
I want to consider a rolling window of 24h for each user taking into account one single IBAN, so for ...
0
votes
3
answers
50
views
Pandas Groupby and Filter based on first record having date greater than specific date
I have a dataframe that shows details about employees and the site they are at and the positions they have held. The dataframe has columns for Site Id, Employee ID, and StartDate (plus a lot more ...
-1
votes
3
answers
88
views
Python groupby rank in two different directions
I have a dataframe, d:
Position Operation Side Price Size
9 9 0 1 0.7289 -16
8 8 0 1 0.729 -427
7 7 0 1 0.7291 -267
6 6 0 1 0.7292 -15
5 5 0 1 ...
0
votes
1
answer
38
views
Set a boolean Mask for a Group with a Condition
If the Value '2235' is in column 'Age', the associated Group in the Column 'Name' should be set for all true in a new Column.
My Test-Dataframe is:
import pandas as pd
# intialise data of lists.
data ...
0
votes
1
answer
33
views
Selecting the lowest price values on a 24 hours cycle to charge a battery
I want to select the minimum values over a 24 hour cycle to select the cheapest price per day.
Price Carbon
Date and Hour
2024-01-01 00:00:00 20 40
2024-01-01 01:00:00 ...
0
votes
0
answers
45
views
Count values greater than row value in all previous rows in each group
I want to compare the numerical value of each row of groups with the previous rows of the same group and return the number of previous rows that have a value greater than the current row.
I know the ...
1
vote
2
answers
54
views
Pandas: adding a new column to MultiIndex
I am having trouble working out a way to add a extra column that has the sum of the items in the bin column.
This is because I would like a way to show bins that have less than so many items remanding ...
1
vote
1
answer
61
views
Query Returns Invalid error on Column Already Exists?
I want to Group BY Date and Payment and branch name and code
I don't know why this SQL query return this error:
Column 'FactSalesTable.SalesTotal' is invalid in the select list because it is not ...
1
vote
1
answer
44
views
Count and groupby a specfic value
I have a dataframe where i want to count a specific value that occurs in a row.
This code below gives the right answer and now i want to add a new coluumn to my dataframe
occur = df.groupby(['...
0
votes
1
answer
35
views
Pandas groupby and concat multiple rows
CONTEXT
I want to group by both a rule_id and calc_id and transform multiple columns into one row where each variable is concatenated with a ",'
DATA EXAMPLE
Calc_ID Rule_ID Name Tracked?...
-3
votes
2
answers
73
views
SQL query - GROUP BY usage [duplicate]
In my table I have three columns: Name, Department and Experience. I want to retrieve the name who got min experience from each department along with department.
I used the below query but it's not ...
0
votes
2
answers
121
views
Pandas Dataframe groupby count number of rows quickly [duplicate]
I have a dataframe that looks like
Class_ID Student_ID feature
1 4 31
1 4 86
1 4 2
1 2 11
1 2 0
5 3 ...
0
votes
1
answer
69
views
How to select 'null' if any null or 'max' if otherwise in a group select?
This is a simplified example of the query I want to do:
select
t.id,
case when any(st.end_at is null) then null else max(st.end_at) end as max_end_at
from table as t
join sub_table as st on st....
0
votes
1
answer
85
views
Efficiently Grouping Millions of Records by Field in MongoDB - Performance Optimization
I'm working with a large dataset in MongoDB where I need to group approximately 15 million records by a specific field (name). The goal is to find all records where the name field is duplicated. I'm ...
0
votes
1
answer
42
views
How to Merge Multiple Rows with Same Unique ID into One in Python with Pandas [duplicate]
I have seen some questions similar to this, but none I could find tackle how to unstack multiple columns. I have successfully unstacked a single column using.
df = df.groupby(['id', 'entity'])['...
1
vote
2
answers
131
views
Sql TRANSFORM with calculation time in MS Access
I'm Trying to make Sql TRANSFORM with calculation time in column in and out.
I want the answer in 1 query and I don't want to use the function in ms access because I want to use the sql in vb.net
...
1
vote
3
answers
43
views
Drop rows based on conditions (longitudinal dataset)
I have a dataset with the following structure:
ID
drug
drug sequence
drup type
A
X
1
AB
A
X
1
AB
A
Y
2
CD
A
Z
3
CD
A
V
4
EF
A
V
4
EF
B
W
1
GH
B
W
1
GH
B
W
2
GH
C
Z
1
CD
C
V
2
EF
C
V
2
EF
...
0
votes
0
answers
19
views
How do I aggregate this column because I know the right thing is to use group by so what am I doing wrong?
Question:
Portfolio Performance Analysis
Which portfolios have achieved the highest average return rates over the last year?
I have attached how the document looks and this is my locution but I don't ...
0
votes
0
answers
104
views
Pandas how to groupby close values with large range
I have dataframes for which I want to group relatively close values of a column.
I tried using percentile but often there are huge differences between values in a group.
I.E. column has 1000 values ...
1
vote
5
answers
106
views
Python Pandas Grouping data between two specific dates
I'm trying to group data from a dataframe considering a specific date range that repeats monthly.
2024-01-26 to 2024-02-25
2024-02-26 to 2024-03-25
2024-03-26 to 2024-04-25
2024-04-26 to 2024-05-25
...
2
votes
1
answer
46
views
Groupby index and keep the max column value given a single column
Scenario: With a dataframe with duplicated indices, I want to groupby while keeping the max value. I found the solution to this in Drop duplicates by index, keeping max for each column across ...
-1
votes
3
answers
31
views
Filling NaN in dataframe column based on groupby dataframe value [duplicate]
I want to impute values to replace NaNs in a dataframe. The dataframe contains Hour, Location and multiple numeric columns associated with speeds, times etc.
In each of the numeric columns containing ...
1
vote
1
answer
51
views
How select query from record count double in MS Access
I'm trying to select query from row count double based on ID and DATE in MS Access.
I want to keep it as a single query. What are the possibilities of this being applied? Or another method?
Please ...
1
vote
1
answer
45
views
How to efficiently select the top column by grouping for each row of a pandas DataFrame?
This is a continuation of a question I asked previously in How to efficiently select the top N columns by grouping for each row of a pandas DataFrame?. My needs have evolved as I work with my dataset -...
2
votes
2
answers
59
views
Preserving DataFrame subclass type during pandas groupby().aggregate()
I'm subclassing pandas DataFrame in a project of mine. Most pandas operations preserve the subclass type, but df.groupby().agg() does not. Is this a bug? Is there a known workaround?
import pandas as ...
2
votes
3
answers
83
views
New rows in Pandas Dataframe by considering combination of rows in the original dataframe and applying function to it
I have a Pandas dataframe df that looks like
Class_ID Student_ID theta
6 1 0.2
6 2 0.2
6 4 0.1
6 3 0.5
3 2 ...
0
votes
2
answers
89
views
How to efficiently select the top N columns by grouping for each row of a pandas DataFrame?
Let's say that I have a pandas DataFrame representing the coolness score for each "contestant" of a hypothetical contest by date:
import numpy as np
import pandas as pd
rng = np.random....
2
votes
1
answer
47
views
Pandas create complex rolling calculation in groupby
I have a dataset in Pandas consisting of phone call logs, from several caller numbers into several destination numbers. Calls are logged as soon as they are completed.
Main features:
unix_timestamp (...
0
votes
0
answers
35
views
How to add row for a groupby dataframe? [duplicate]
I have a dataframe which comes from groupby. it looks like:
pnl amt
ticker ex
1000RATS_USDT kn 1.0 666.0
AAVE_USDT bg -7.4 5724.0
...
1
vote
1
answer
41
views
groupby with conditions .nth()
I am trying to get an additional column ['readmit'] that gives the time elapsed in days since the indexed admission discharge date(first occurance) for each new episode of the same keys admit date.
...
1
vote
1
answer
37
views
Add Optional Variables in Groupby Function Pandas
Let's say I have the following data frame, and I want to have var_a be optional, so that if it is not supplied there will not be a default value; it will only group by the variable vehicle. How do I ...
0
votes
0
answers
18
views
Iterate a group_by summarise function over multiple variables
I have written a group_by summarise function to avoid typing it out every time. I want to run for multiple variables (columns) in my dataframe. Is there a way I can iterate this, as opposed to typing ...
1
vote
1
answer
46
views
Pandas: Returning Blank Array after Groupby Aggregate
I have a dataframe:
data = {
"Key": ["A1", "A2", np.nan, "A3", "A4"],
"Name": ["Candy A", "Candy B", np.nan, "...
2
votes
1
answer
49
views
Applying function that takes a list as input and a list as output to a pandas dataframe
I have defined the following function:
def my_function(inputList):
intermediateList = []
outputList = []
S = 0
for x in inputList:
S += x
y = 6*x
intermediateList.append(y)
for ...
0
votes
1
answer
59
views
Pandas: How do I use a function with groupby on multiple columns when the function depends on sequential rows within each group?
I would like to impute some missing values in a pandas data frame using .bfill() or .ffill(). I want to use .groupby() first. The problem is that .bfill() and .ffill() depend on previous or next ...
-1
votes
1
answer
175
views
KeyError: 0 # If we have a listlike key, _check_indexing_error will raise after applying a function to a pandas dataframe
I have a rather complicated function f(featureList) that takes a list of arbitrary length as input and gives another list of the same length as output:
import math
import random
import time
def ...
0
votes
0
answers
36
views
Pandas how to do groupby + resample + linear interpolation at once?
I have a huge dataset in a dataframe with 1 column having location name, 1 column having a depth value and then a set of other columns with values measured at those locations for those depths.
The ...
0
votes
0
answers
27
views
TypeError: Could not convert string to numeric during groupby mean operation in Pandas
I'm working on a data analysis project using Pandas to analyze housing prices in London boroughs. I'm trying to group the data by borough and year and then calculate the mean of the average housing ...
2
votes
1
answer
103
views
MySQL ask to add column to GROUP BY which I don't want to use in GROUP BY
I have a query that should create a view that will be used to be exported as csv file. Here's the query:
CREATE OR REPLACE VIEW coupons_export AS
SELECT
coupons.code AS coupon,
...