Questions tagged [aggregate]
Aggregate refers to the process of summarizing grouped data, commonly used in Statistics.
8,282
questions
0
votes
1
answer
35
views
Aggregate data frame using less repetitive code than a bunch of select and merge operations
I am trying to summarize/aggregate a data frame as follows. While the code gives the correct result, it is very repetitive and I would like to to avoid that. I think that I need to use something like ...
0
votes
1
answer
37
views
Frequency of A list in a Pyarrow Table
Find myself in an interesting conundrum, in that I need to compute the frequency of a series of lists by participant ID. Data looks not unlike the following:
test_table_orig = pa.table([
pa.array([...
0
votes
1
answer
28
views
Django annotate with Count on subquery with OuterRef failing
TLDR:
Lets hit overall problem here 1st:
How in django annotate queryset with a Subquery that uses OuterRef and count how many objects subquery returned (for every object in original query)
Pseudocode:...
1
vote
1
answer
38
views
Difference Between Placing `$match` Before or After `$unwind` in MongoDB Aggregate for My BooksDB
Hello MongoDB community,
I am trying to understand the difference in the results and performance implications between two aggregate queries in MongoDB. Both queries aim to count the number of books ...
0
votes
1
answer
24
views
How can I filter an annotated query set
I have the code below that returns an aggregate of authors that have a book and the total amount of authors. This works well, but I want to get the count of authors that have a particular type of book ...
-2
votes
1
answer
38
views
Mongoose: filter option in a aggregation query
I'm having problems filtering out data in a query. My original query works, but I get 0 results if I add a match statement at the end of the pipepline (if Base in code). Base is a field in users.
...
2
votes
2
answers
44
views
How to lookup a sub-array in mongodb?
I tried to use aggregate and lookup but look like I cannot keep role in the members item. Any idea how to resolve it?
db.groups.insertMany([
{
title: 'Hi',
members: [
{
...
0
votes
1
answer
20
views
how to use aggregate sum to create a new variable that is correct df size
I am trying to create a new variable in this data set that is the total of the weights in each level of the cluster variable (i.e., sum of weights in cluster 1, cluster 2, etc.) I get this error, that ...
-3
votes
1
answer
90
views
How to find and count number of matches in an array of objects for a 4 digit number with MongoDB (i.e. lottery numbers) [closed]
I'm creating some sort of lottery game where users can submit a 4 digit number between numbers 1-9 (e.g. "1, 4, 5, 8"). Then the game concludes with a winning 4 digit number using Math....
-1
votes
3
answers
69
views
How does arithmetic sequence of addition with SUM works in SQL server?
I have an issue with the SUM operation on float data type on SQL Server 2016. I know addition with float data type is not associative means, so the order of additions matters and it will affect the ...
0
votes
0
answers
29
views
Merge duplicate rows and get the nth row from each group [duplicate]
i found below solution from stackoverflow, but i need some change in this.
how can i use nth(n) aggregate function in below example.
i want to catch nth position of a single group.
in this given ...
1
vote
1
answer
36
views
MongoDB group aggregate with nested array of objects
I have a collection with the following structure (i'll post a link to playground because it's quite big):
https://mongoplayground.net/p/WNga0mze0WS
Just a brief explanation, we are looking for ...
0
votes
2
answers
29
views
psql: How can I get a single json column in aggregation after a group by query
I am trying to get a single row for each group by clause in the query. Each SELECT output has a json in it. I am trying this:
SELECT MIN(C.json_column->'key') AS Key, A.field1, B.field2
FROM ...
1
vote
1
answer
46
views
How to aggregate values in Data Frame for a few columns to have NaN in case of sum NaN instead of 0 in Python Pandas?
I have Data Frame in Python Pandas like below:
Input data:
df = pd.DataFrame({
'id' : [999, 999, 999, 185, 185, 185, 999, 999, 999],
'target' : [1, 1, 1, 0, 0, 0, 1, 1, 1],
'event': ['2023-...
0
votes
1
answer
32
views
Conditionally aggregating data while also grouping by site
I'm working with a large case-level dataset and am brand new to python. The dataset in general looks something like this:
Site Type Value
A Red 10
A ...
2
votes
1
answer
90
views
Customized data aggregation using vanilla JavaScript
Could you please help me with the following data aggregation problem?
Here's my initial array:
const data = [
{ date: '2023-11-01', type: 'A', casualties: 10, state: 'NY', country: 'USA', site: '...
0
votes
0
answers
41
views
MongoDB. How to use sorting, search and limit in aggregation correctly
Suppose I have 10 million documents.
I need to skip the last 5 documents and get the last 10 documents.
This code works fast
documents.find({user: 'demo'}).sort({ _id: -1 }).skip(5).limit(10)
This ...
0
votes
0
answers
21
views
$lookup in aggregate but array is empty
I used $lookup to join a connection. But it takes empty list in joining
``` var userId="65421d801c128f2ed899a7a4";``
aggregate([
{
$unwind: {
path: "$reserve",
...
-1
votes
2
answers
53
views
Mongodb: Unsupported conversion from array to objectId
I used a $lookup to join a connection.
There was no problem under normal circumstances. But after using pipeline, the program gives an error : "Unsupported conversion from array to objectId in $...
0
votes
2
answers
49
views
what is the proper way to use aggregate in a complicated structured Django models
I have a Django app that represents a football league and should show scores points and other stuff, I need to create a function based on the models in this app to get the sum of goals, points, ...
-1
votes
0
answers
25
views
aggregate on annotate inside subquery
return queryset.annotate(
sum_amount=Subquery(
RI.objects.filter(r_queryset=OuterRef('pk'))
.annotate(
sum_amount=Sum(
...
1
vote
1
answer
20
views
Expression within condition
This is a follow up question:
I want to aggregate the average of a collection only for documents with type 'buy'.
So the problem is how to get the if condition around the sum expression and same for ...
1
vote
1
answer
41
views
Aggregate average
I want to aggregate the average of a collection times the amount.
These are example documents:
[
{
_id: new ObjectId("653c4c017800342a3bedb82e"),
item: 555,
amount: 1,
...
0
votes
0
answers
20
views
R Combine Excel files into a single Excel file with multiple sheets, where the sheets are sourced from different files but share the same structure
I have 4 Excel files (a,b,c,d), each contains 6 Excel sheets.
All the files have the same structure (i.e., column names, sheet names).
I would like to have 1 final Excel Sheet with the aggregated data ...
0
votes
0
answers
20
views
Unique Monthly Count in Spotfire Visualization that Ignores Coloring but can Still be Filtered
I cannot figure out how to make a stacked bar chart which outlines the average cost of a set of jobs on a monthly basis, and how those costs are divided between different cost categories, in such a ...
0
votes
2
answers
25
views
aggregating/summing variables for each ID observations and storing in corresponding new total variables in SAS
so i created a macro for this but issue is i have a lot of variables; more than 50. I was hoping there could be an efficient way to create so i can pass a list of new variables without having to ...
0
votes
1
answer
31
views
I have error after using aggregate mongoose mongoDb in my Api
"error": {
"ok": 0,
"code": 5107201,
"codeName": "Location5107201",
"$clusterTime": {
"clusterTime": {
"$timestamp": &...
0
votes
0
answers
15
views
Aggregate and count the status in field using FME Workbench
I have two layers point and a polygon, I wanted to have the aggregate of a column name "Progress" with its respective polygon name and polygon id. The progress field in the point layer has ...
0
votes
0
answers
23
views
Elasticsearch using to filter detailed data out of a bif demand list
I have a huge CSV file with three columns:
number of product, KindOfProduct, FinalSubmissionTime
The data is for a whole month and the FinalSubmissionTime is specified with the date and hour. Now I ...
0
votes
1
answer
36
views
MongoDB $lookup on doubly nested array of objects
I have a data model where each product has many variants and each variant has many modifications. In the database, it looks like this:
const mods = db.modifications.insertMany([
{
title: '...
0
votes
0
answers
14
views
I have one array in mongoDb want to populate the ids of that array using aggregate
I have a MongoDB collection containing documents with an embedded array of products. Each product is represented by an productId, brandId, packSizeId, competitorId, and other related fields. I want to ...
1
vote
2
answers
33
views
Merge weighted averages in groupby
I have a Pandas DataFrame:
import pandas as pd
df = pd.DataFrame({
"commodity":["Potatos","Potatos","Apples","Apples","Apples"],
&...
1
vote
3
answers
62
views
Aggregation by two columns, excel. power query
I have a dataframe like this
idvac
idcand
event
status
1
1
call
yes
1
1
meeting
no
I'd like to get like this
idvac idcand call meeting
1 1 yes no
seems like the best solution is ...
1
vote
1
answer
58
views
MS Access- Count the values in a table row that are equal to 1 but not the other values of 0 and -1 then add them
We have a simple database with a table that has 53 Columns in it, each column can contain a 1,-1, or 0. We need to count the number of 1's and -1 in each record to get the total of each. I would like ...
0
votes
0
answers
10
views
Inventory Problem: How to group ungrouped and aggregated data
I am trying to group data into a dataframe. The data I imported is an excel sales report from Quickbooks Enterprise (14th Edition). The format contains Q1, Q2, and Q3 data for this year. The columns ...
0
votes
0
answers
28
views
Maintaining time dimension (Z) for an aggregated Raster
I'm aggregating a large RasterBrick by lat, lon, and layer. My original RasterBrick object has a set Z dimension (i.e. there are dates assigned to each layer), but I lose these dates when I aggregate ...
2
votes
3
answers
62
views
in r how to sort the lowest numeric value in multiple numeric columns and by group?
** There are NAs in the min.range column**
In R, I have a dataframe with a list of species, each has two related columns with numeric values (overall.percentage and min.range). For each species, there ...
-1
votes
1
answer
48
views
Trailing 12 months calculation not giving expected results
I have a table of daily data that i'm trying to add trailing twelve months column to with the sum of the revenue by id, product, currency, and date truncated to month.
I put together some random ...
0
votes
0
answers
38
views
How to aggregate ORs from the same study, and after that use these aggregated effect sizes to do a meta-analysis in R
I am trying to do a meta-analysis with a random effects model. The data includes: study, Odds Ratio (OR), and 95% confidence interval. For the meta-analysis, some categories were made consisting from ...
0
votes
0
answers
35
views
Pandas datframe: Read in multiple textfiles, map them together, aggregate (unique), and return as dictionary
Set-up and available data:
I have 3 csv files describing some measurements I have:
1st file: descriptive weather station assets (metgeography.csv),containing for instance: station number ("stNo&...
1
vote
1
answer
68
views
Polars group_by_rolling only retains index and by columns
Using the same dataframe as in a previous question,
pl.DataFrame(
[
pl.Series("Time", ['02/01/2018 07:05', '02/01/2018 07:07', '02/01/2018 07:08', '02/01/2018 07:09', '02/01/2018 ...
0
votes
1
answer
39
views
collapse / aggregate multiple rows into single row with shared percentage based on same IDs
I need some advice/hint on how to aggregate rows shared IDs to get percentage data.
I have the data frame formatted as:
id county_1 country_2 country_3 .... country_x sum_by_id
1 10 ...
0
votes
1
answer
22
views
How to group by counted elements using aggregation in MongoDB?
I have to write some aggregation using MQL. The case is:
There is a chat application
Collection contains info about: thread, sent message, sent time, read time
Sample records are shown below.
...
0
votes
2
answers
32
views
Aggregate works with complete cases
I would like to aggregate multiple columns by a character vector. However I think that base R aggregate works on complete cases so if a summation variable is missing for one variable it is excluded ...
0
votes
1
answer
40
views
Pandas Groupby-- How to extract first, last, or first non-NaN value from a group
This comes up for me repeatedly, and I'm sure there a way to do it as part of the groupby().agg() command, but I can't figure it out. I often have to resample time series and need to keep a value from ...
0
votes
0
answers
52
views
Fill values in a matrix from column 4 based on a value in column 3 for all categories in column 1 and column 2 in R
Dataframe df is the given dataframe and we want to obtain the matrix below for value in VAR3 as "101". Use of xtabs() does not produce the desired output. The value inside 2X2 matrix is the ...
1
vote
1
answer
38
views
match/filter/project nested fields from array in array in object
I need a little help , I attempt to extract some fields from nested array but yet struggeling , can somebody help?
Here is the example document:
[
{
_id: 1,
"_a": [
{
"_p": [
...
0
votes
1
answer
29
views
Return links which do not have an associated document in MongoDB
I have a collection of webpages stored in MongoDB. Each document contains at least a hostname and pathname value as well as an array of all links found in a field 'a'. The document only exists after ...
2
votes
1
answer
31
views
select a grouped row based on another variable
I have a data set where for each region there is a value pos, but some regions don't have a value of 1.
region <- c('A','A','B','B','C','D','D','E')
pos <- c(0,1,0,1,0,0,1,0)
df <- data.frame(...
1
vote
1
answer
46
views
Aggregate dataframe by dimension pairs for heatmap
I have a table with rows being individual visits from users to an online shop. There are multiple columns that are attributes of the visit (could be booleans or categorical with more than 2 possible ...