Questions tagged [aggregate]

Aggregate refers to the process of summarizing grouped data, commonly used in Statistics.

Filter by
Sorted by
Tagged with
0 votes
1 answer
35 views

Aggregate data frame using less repetitive code than a bunch of select and merge operations

I am trying to summarize/aggregate a data frame as follows. While the code gives the correct result, it is very repetitive and I would like to to avoid that. I think that I need to use something like ...
Timur Shtatland's user avatar
0 votes
1 answer
37 views

Frequency of A list in a Pyarrow Table

Find myself in an interesting conundrum, in that I need to compute the frequency of a series of lists by participant ID. Data looks not unlike the following: test_table_orig = pa.table([ pa.array([...
TDeramus's user avatar
  • 117
0 votes
1 answer
28 views

Django annotate with Count on subquery with OuterRef failing

TLDR: Lets hit overall problem here 1st: How in django annotate queryset with a Subquery that uses OuterRef and count how many objects subquery returned (for every object in original query) Pseudocode:...
Johnemperor's user avatar
1 vote
1 answer
38 views

Difference Between Placing `$match` Before or After `$unwind` in MongoDB Aggregate for My BooksDB

Hello MongoDB community, I am trying to understand the difference in the results and performance implications between two aggregate queries in MongoDB. Both queries aim to count the number of books ...
omersk's user avatar
  • 13
0 votes
1 answer
24 views

How can I filter an annotated query set

I have the code below that returns an aggregate of authors that have a book and the total amount of authors. This works well, but I want to get the count of authors that have a particular type of book ...
Tife's user avatar
  • 33
-2 votes
1 answer
38 views

Mongoose: filter option in a aggregation query

I'm having problems filtering out data in a query. My original query works, but I get 0 results if I add a match statement at the end of the pipepline (if Base in code). Base is a field in users. ...
Martin Wiklund's user avatar
2 votes
2 answers
44 views

How to lookup a sub-array in mongodb?

I tried to use aggregate and lookup but look like I cannot keep role in the members item. Any idea how to resolve it? db.groups.insertMany([ { title: 'Hi', members: [ { ...
Lam Lee's user avatar
  • 129
0 votes
1 answer
20 views

how to use aggregate sum to create a new variable that is correct df size

I am trying to create a new variable in this data set that is the total of the weights in each level of the cluster variable (i.e., sum of weights in cluster 1, cluster 2, etc.) I get this error, that ...
four77's user avatar
  • 185
-3 votes
1 answer
90 views

How to find and count number of matches in an array of objects for a 4 digit number with MongoDB (i.e. lottery numbers) [closed]

I'm creating some sort of lottery game where users can submit a 4 digit number between numbers 1-9 (e.g. "1, 4, 5, 8"). Then the game concludes with a winning 4 digit number using Math....
user874737's user avatar
-1 votes
3 answers
69 views

How does arithmetic sequence of addition with SUM works in SQL server?

I have an issue with the SUM operation on float data type on SQL Server 2016. I know addition with float data type is not associative means, so the order of additions matters and it will affect the ...
mangerbang's user avatar
0 votes
0 answers
29 views

Merge duplicate rows and get the nth row from each group [duplicate]

i found below solution from stackoverflow, but i need some change in this. how can i use nth(n) aggregate function in below example. i want to catch nth position of a single group. in this given ...
user22891413's user avatar
1 vote
1 answer
36 views

MongoDB group aggregate with nested array of objects

I have a collection with the following structure (i'll post a link to playground because it's quite big): https://mongoplayground.net/p/WNga0mze0WS Just a brief explanation, we are looking for ...
DAVI FONSECA's user avatar
0 votes
2 answers
29 views

psql: How can I get a single json column in aggregation after a group by query

I am trying to get a single row for each group by clause in the query. Each SELECT output has a json in it. I am trying this: SELECT MIN(C.json_column->'key') AS Key, A.field1, B.field2 FROM ...
bcsta's user avatar
  • 2,032
1 vote
1 answer
46 views

How to aggregate values in Data Frame for a few columns to have NaN in case of sum NaN instead of 0 in Python Pandas?

I have Data Frame in Python Pandas like below: Input data: df = pd.DataFrame({ 'id' : [999, 999, 999, 185, 185, 185, 999, 999, 999], 'target' : [1, 1, 1, 0, 0, 0, 1, 1, 1], 'event': ['2023-...
dingaro's user avatar
  • 2,292
0 votes
1 answer
32 views

Conditionally aggregating data while also grouping by site

I'm working with a large case-level dataset and am brand new to python. The dataset in general looks something like this: Site Type Value A Red 10 A ...
BMcBee's user avatar
  • 1
2 votes
1 answer
90 views

Customized data aggregation using vanilla JavaScript

Could you please help me with the following data aggregation problem? Here's my initial array: const data = [ { date: '2023-11-01', type: 'A', casualties: 10, state: 'NY', country: 'USA', site: '...
user1352042's user avatar
0 votes
0 answers
41 views

MongoDB. How to use sorting, search and limit in aggregation correctly

Suppose I have 10 million documents. I need to skip the last 5 documents and get the last 10 documents. This code works fast documents.find({user: 'demo'}).sort({ _id: -1 }).skip(5).limit(10) This ...
Александр Миронов's user avatar
0 votes
0 answers
21 views

$lookup in aggregate but array is empty

I used $lookup to join a connection. But it takes empty list in joining ``` var userId="65421d801c128f2ed899a7a4";`` aggregate([ { $unwind: { path: "$reserve", ...
Iman Kamali's user avatar
-1 votes
2 answers
53 views

Mongodb: Unsupported conversion from array to objectId

I used a $lookup to join a connection. There was no problem under normal circumstances. But after using pipeline, the program gives an error : "Unsupported conversion from array to objectId in $...
Iman Kamali's user avatar
0 votes
2 answers
49 views

what is the proper way to use aggregate in a complicated structured Django models

I have a Django app that represents a football league and should show scores points and other stuff, I need to create a function based on the models in this app to get the sum of goals, points, ...
Ahmed Wagdi's user avatar
  • 3,925
-1 votes
0 answers
25 views

aggregate on annotate inside subquery

return queryset.annotate( sum_amount=Subquery( RI.objects.filter(r_queryset=OuterRef('pk')) .annotate( sum_amount=Sum( ...
yalda's user avatar
  • 1
1 vote
1 answer
20 views

Expression within condition

This is a follow up question: I want to aggregate the average of a collection only for documents with type 'buy'. So the problem is how to get the if condition around the sum expression and same for ...
OliverKhl's user avatar
1 vote
1 answer
41 views

Aggregate average

I want to aggregate the average of a collection times the amount. These are example documents: [ { _id: new ObjectId("653c4c017800342a3bedb82e"), item: 555, amount: 1, ...
OliverKhl's user avatar
0 votes
0 answers
20 views

R Combine Excel files into a single Excel file with multiple sheets, where the sheets are sourced from different files but share the same structure

I have 4 Excel files (a,b,c,d), each contains 6 Excel sheets. All the files have the same structure (i.e., column names, sheet names). I would like to have 1 final Excel Sheet with the aggregated data ...
J. Pasqualini's user avatar
0 votes
0 answers
20 views

Unique Monthly Count in Spotfire Visualization that Ignores Coloring but can Still be Filtered

I cannot figure out how to make a stacked bar chart which outlines the average cost of a set of jobs on a monthly basis, and how those costs are divided between different cost categories, in such a ...
cambry's user avatar
  • 3
0 votes
2 answers
25 views

aggregating/summing variables for each ID observations and storing in corresponding new total variables in SAS

so i created a macro for this but issue is i have a lot of variables; more than 50. I was hoping there could be an efficient way to create so i can pass a list of new variables without having to ...
jean's user avatar
  • 1
0 votes
1 answer
31 views

I have error after using aggregate mongoose mongoDb in my Api

"error": { "ok": 0, "code": 5107201, "codeName": "Location5107201", "$clusterTime": { "clusterTime": { "$timestamp": &...
Donisher Djumaqulov's user avatar
0 votes
0 answers
15 views

Aggregate and count the status in field using FME Workbench

I have two layers point and a polygon, I wanted to have the aggregate of a column name "Progress" with its respective polygon name and polygon id. The progress field in the point layer has ...
Vicky's user avatar
  • 51
0 votes
0 answers
23 views

Elasticsearch using to filter detailed data out of a bif demand list

I have a huge CSV file with three columns: number of product, KindOfProduct, FinalSubmissionTime The data is for a whole month and the FinalSubmissionTime is specified with the date and hour. Now I ...
Andreas K's user avatar
0 votes
1 answer
36 views

MongoDB $lookup on doubly nested array of objects

I have a data model where each product has many variants and each variant has many modifications. In the database, it looks like this: const mods = db.modifications.insertMany([ { title: '...
crabvk's user avatar
  • 131
0 votes
0 answers
14 views

I have one array in mongoDb want to populate the ids of that array using aggregate

I have a MongoDB collection containing documents with an embedded array of products. Each product is represented by an productId, brandId, packSizeId, competitorId, and other related fields. I want to ...
Sameer Shaikh's user avatar
1 vote
2 answers
33 views

Merge weighted averages in groupby

I have a Pandas DataFrame: import pandas as pd df = pd.DataFrame({ "commodity":["Potatos","Potatos","Apples","Apples","Apples"], &...
sds's user avatar
  • 58.9k
1 vote
3 answers
62 views

Aggregation by two columns, excel. power query

I have a dataframe like this idvac idcand event status 1 1 call yes 1 1 meeting no I'd like to get like this idvac idcand call meeting 1 1 yes no seems like the best solution is ...
Edward's user avatar
  • 4,513
1 vote
1 answer
58 views

MS Access- Count the values in a table row that are equal to 1 but not the other values of 0 and -1 then add them

We have a simple database with a table that has 53 Columns in it, each column can contain a 1,-1, or 0. We need to count the number of 1's and -1 in each record to get the total of each. I would like ...
Chrisemtpa's user avatar
0 votes
0 answers
10 views

Inventory Problem: How to group ungrouped and aggregated data

I am trying to group data into a dataframe. The data I imported is an excel sales report from Quickbooks Enterprise (14th Edition). The format contains Q1, Q2, and Q3 data for this year. The columns ...
Frank G. Rojas's user avatar
0 votes
0 answers
28 views

Maintaining time dimension (Z) for an aggregated Raster

I'm aggregating a large RasterBrick by lat, lon, and layer. My original RasterBrick object has a set Z dimension (i.e. there are dates assigned to each layer), but I lose these dates when I aggregate ...
Kaila Frazer's user avatar
2 votes
3 answers
62 views

in r how to sort the lowest numeric value in multiple numeric columns and by group?

** There are NAs in the min.range column** In R, I have a dataframe with a list of species, each has two related columns with numeric values (overall.percentage and min.range). For each species, there ...
Yitzchak Ben-Mocha's user avatar
-1 votes
1 answer
48 views

Trailing 12 months calculation not giving expected results

I have a table of daily data that i'm trying to add trailing twelve months column to with the sum of the revenue by id, product, currency, and date truncated to month. I put together some random ...
Raksha's user avatar
  • 1,619
0 votes
0 answers
38 views

How to aggregate ORs from the same study, and after that use these aggregated effect sizes to do a meta-analysis in R

I am trying to do a meta-analysis with a random effects model. The data includes: study, Odds Ratio (OR), and 95% confidence interval. For the meta-analysis, some categories were made consisting from ...
Noa's user avatar
  • 1
0 votes
0 answers
35 views

Pandas datframe: Read in multiple textfiles, map them together, aggregate (unique), and return as dictionary

Set-up and available data: I have 3 csv files describing some measurements I have: 1st file: descriptive weather station assets (metgeography.csv),containing for instance: station number ("stNo&...
Marco's user avatar
  • 11
1 vote
1 answer
68 views

Polars group_by_rolling only retains index and by columns

Using the same dataframe as in a previous question, pl.DataFrame( [ pl.Series("Time", ['02/01/2018 07:05', '02/01/2018 07:07', '02/01/2018 07:08', '02/01/2018 07:09', '02/01/2018 ...
Ted Black's user avatar
  • 148
0 votes
1 answer
39 views

collapse / aggregate multiple rows into single row with shared percentage based on same IDs

I need some advice/hint on how to aggregate rows shared IDs to get percentage data. I have the data frame formatted as: id county_1 country_2 country_3 .... country_x sum_by_id 1 10 ...
Sam 's user avatar
  • 1
0 votes
1 answer
22 views

How to group by counted elements using aggregation in MongoDB?

I have to write some aggregation using MQL. The case is: There is a chat application Collection contains info about: thread, sent message, sent time, read time Sample records are shown below. ...
Harord's user avatar
  • 59
0 votes
2 answers
32 views

Aggregate works with complete cases

I would like to aggregate multiple columns by a character vector. However I think that base R aggregate works on complete cases so if a summation variable is missing for one variable it is excluded ...
Stephen Clark's user avatar
0 votes
1 answer
40 views

Pandas Groupby-- How to extract first, last, or first non-NaN value from a group

This comes up for me repeatedly, and I'm sure there a way to do it as part of the groupby().agg() command, but I can't figure it out. I often have to resample time series and need to keep a value from ...
JackyChan's user avatar
0 votes
0 answers
52 views

Fill values in a matrix from column 4 based on a value in column 3 for all categories in column 1 and column 2 in R

Dataframe df is the given dataframe and we want to obtain the matrix below for value in VAR3 as "101". Use of xtabs() does not produce the desired output. The value inside 2X2 matrix is the ...
vp_050's user avatar
  • 672
1 vote
1 answer
38 views

match/filter/project nested fields from array in array in object

I need a little help , I attempt to extract some fields from nested array but yet struggeling , can somebody help? Here is the example document: [ { _id: 1, "_a": [ { "_p": [ ...
R2D2's user avatar
  • 9,630
0 votes
1 answer
29 views

Return links which do not have an associated document in MongoDB

I have a collection of webpages stored in MongoDB. Each document contains at least a hostname and pathname value as well as an array of all links found in a field 'a'. The document only exists after ...
Patrick Sturm's user avatar
2 votes
1 answer
31 views

select a grouped row based on another variable

I have a data set where for each region there is a value pos, but some regions don't have a value of 1. region <- c('A','A','B','B','C','D','D','E') pos <- c(0,1,0,1,0,0,1,0) df <- data.frame(...
bhopalstiffs's user avatar
1 vote
1 answer
46 views

Aggregate dataframe by dimension pairs for heatmap

I have a table with rows being individual visits from users to an online shop. There are multiple columns that are attributes of the visit (could be booleans or categorical with more than 2 possible ...
fredibeni's user avatar

1
2 3 4 5
166