477
questions
0
votes
1
answer
19
views
How to remove dynamic field from mapping with Elastic Reindex API
We have a dynamic field defined in multiple indexes that is of type geo_shape, and uses the points_only param. Due to a) the deprecation of points_only in version 7.x, and b) the fact that we don't ...
2
votes
1
answer
80
views
How to resample a dataframe to stretch from start to enddate in intervals (containing 0 for not available values)
I have the following setup:
I have sparse information about queries hitting my endpoint at certain timepoints in a csv file. I parse this csv file with dates according to date_format='ISO8601' in the ...
0
votes
0
answers
50
views
SOLR 504 - bad gateway
When trying to reindex SOLR, at some point I only get failed tasks in my resque:queue. I try to retry them, but after 60 seconds (default bad gateway time imposed by ALB) I receive error 504. What ...
1
vote
2
answers
78
views
How to update de value of the column headers on a pandas DataFrame?
I'm having problems on sorting my header in a way that shows all the meetings that happened on that year, than move on to the next year (M being meeting):
Given the Data frame:
Meeting M1/2023 M1/...
0
votes
2
answers
153
views
Postgresql Primary Key Index Maintenance
I refresh my indexes in my Postgresql database with the "reindex index concurrently" command. However, I never renewed my primary key indexes because it might damage my primary key indexes ...
0
votes
0
answers
31
views
Elasticsearch reindexing - missing child mapping
I have elastic index with join mapping, I created new index, performed reindexing. The issue is related to mapping on new index, I have all fields in new mapping except fields from document that is ...
0
votes
0
answers
326
views
Is a full reindex required after adding a new field to Opensearch mapping? (Searchkick)
I am using the Searchkick gem to communicate with Opensearch.
The Searchkick docs state:
Control what data is indexed with the search_data method. Call Product.reindex after changing this method.
It ...
1
vote
0
answers
81
views
Opensearch - Reindex periodically using Query DSL
I need to reindex data periodically like every 1 hour using a custom Query DSL.
I reindex a subset of documents, i use an ingest pipeline and a custom script to update the new index.
It looks like ...
0
votes
2
answers
80
views
How to create index with new fields, parsing values from field in existing index?
I have an index that contains 1 document. This document has a field1 with value "B" and a field2 with value "C, D, E" (that is, the value in field2 is comma separated and can have ...
0
votes
0
answers
26
views
How to Reindex Solr in Production with Data Changes/Updates
I have deployed Solr in Solrcloud mode (Operator) and although I am pretty good with K8s I am fairly new to Solr. I have worked with nosql darabases but Solr interests me and so I deployed it in my ...
-1
votes
1
answer
74
views
_reindex has suddenly stopped working inexplicably
I've been using the _reindex endpoint for some time now, about 2 months, without any problems.
I'm running ES 8.6.2 on port 9500. The OS is W10.
For some reason, as of this morning, any attempts to ...
1
vote
1
answer
101
views
Reconcile channels with varying data acquisition frequencies
My dataframe has 8 columns for 4 different data recorded as a function of depth. However, each channel came with a different acquisition frequency. I would like to reconcile all these 4 data using a ...
0
votes
0
answers
144
views
PostgreSQL Scheduled Reindexing
I want to prepare a job that checks for fragmentation and reindexes those with fragmentation over 60. But I couldn't figure out how to do it. Currently, I have a query that only checks the index and ...
0
votes
1
answer
44
views
How to change an Index of DataFrame?
I have a DataFrame with an index column as [0, 1,2,3]. I would like to change it to ['Q1','Q2','Q3','Q4']. How do I do this? I do not want to introduce a new column but change index only.
I tried
qtrs ...
0
votes
0
answers
35
views
Same Mapping Retained After Reindexing in Elasticsearch
I recently attempted to update the mapping of an Elasticsearch index by creating a new index with a specific mapping and then reindexing the data from the old index into the new one. However, despite ...
1
vote
1
answer
135
views
I would like to know how to reindex Express program (.CDX) files using PHP
Currently, I can insert data using the xbase library in PHP, but I'm facing an issue where I have to manually reindex the data in the Express program after inserting it. I would like to know how to ...
0
votes
0
answers
355
views
Elasticsearch conflict during reindex with opType Create
I've used reindex API with opType=create and I've expected for conflicts to be impossible because only new records will be inserted. But I get conflict message. Whats wrong? Does elasticsearch has ...
0
votes
1
answer
156
views
Magento CLI fails to return any output when I type magento commands like reindex
Magento CLI fails to return any output but instead goes exits to the next line, I am trying to execute reindex but I am unsure it is even executing.
I really need it to work because it is causing ...
0
votes
0
answers
44
views
Getting mapper_parsing_exception on creating a deleted elastic index again
I have deleted an existing elastic index as part of reindexing, created a temp index and moved all the documents to the temp index before deleting this index. Now I'm getting this error when I'm ...
1
vote
1
answer
69
views
Looking to reindex dates based on a groupby of Grid Cell
I am attempting to do something similar to the reindexing in Pandas reindex date index by group revisited
except I have one extra level of indexing involved. My data is in the following structure: [...
0
votes
1
answer
1k
views
Does Re-index in Elasticsearch has zero-down time?
Currently, I'm working with Elasticsearch, I'm interested in reindexing because it has no time down. I have a question if my index has very much data (1 million documents for example), when I call ...
0
votes
1
answer
120
views
How to manage scroll context on reindex with Elasticsearch and Dotnet
I have a method in .NET which reindexes indices in Elasticsearch.
When I execute it, the tasks generated are in error with the reason
Trying to create too many scroll contexts. Must be less than or ...
0
votes
0
answers
97
views
How does Pandas DataFrame reindex function work with the Copy Parameter when it is set to 'False'?
I could not quite understand the functionality of copy using reindex. When I tested, copy=False is working only if I am giving exactly same index/columns, in sequence. But this not work if I am ...
1
vote
2
answers
94
views
Pandas df.reindex() doesn't work for level>0 for MultiIndex
I've got two DataFrames A and B, both with a MultiIndex in the columns. B however has the order of the columns switched up. Now I would simply like to get the columns of B in the same order as in A.
...
1
vote
1
answer
26
views
Need to have non NaN value within same sampling window in a DataFrame
I am trying to write an implementation of the GAPS operator in Kairos in Python.
What the GAPS operator does is introduces NaN values where there is no data point based on the sampling frequency.
`
...
0
votes
1
answer
57
views
Is there anyway to skip `Reindexing in Elastic Search with database
Currently working with a large database that typically takes 2-3 days to reindex completely, and I only need to add a new field to the mapping for a specific table.
What could be a possible solution ...
1
vote
3
answers
90
views
Sort/reindex a pandas multiindex dataframe when level 1 indicies are different for each level 0 index. Only one level 0 group needs to be sorted
I am struggling with finding a way to sort the rows in a grouped pandas dataframe with two levels of indicies. This is an example of what a dataframe df I have:
I could not paste the table in a nice ...
0
votes
3
answers
97
views
trouble reindexing python dictionary
I have a simple dictionary I deleted an index entry in the middle. I'm having trouble re-ordering my_dict index keys to be in sequence after I deleted an entry in the middle.
my_dict = {
0: {
...
1
vote
1
answer
71
views
Manipulating indices in a pandas dataframe
The problem:
I want to shift the index by adding 2 (index+=2) to match it with the indices of a .csv file.
I have a dataframe which looks as follows:
Counts
Counts
7
10951.0
870.0
8
8653.0
1772.0
...
0
votes
2
answers
51
views
Pandas: Enforcing consistent values for inner index across all outer index values
I have a dataset indexed by entity_id and timestamp, but certain entity_id's do not have entries at all timestamps (not missing values, just no row). I'm trying to enforce consistent timestamps across ...
0
votes
2
answers
37
views
reindex df on axis with duplicates
I have the following df:
df = pd.DataFrame({"col1": [0,0,0,1,1,1], "col2": ["a", "b", "c", "D", "E", "F"] })
and want ...
0
votes
1
answer
39
views
Is it expected behavior to manually reindex a Frame after sorting? Is this a bug?
I have a time series of stellar flux that is periodic in nature. For this data I created a DataFrame with a time column and flux column.
file: Path = localdir / file_path.csv
est_period: float = #...
1
vote
2
answers
149
views
Reindexing Pandas based on daterange
I am trying to reindex the dates in pandas. This is because there are dates which are missing, such as weekends or national hollidays.
To do this I am using the following code:
import pandas as pd
...
0
votes
1
answer
166
views
Reindex Pyspark Dataframe with dates in Year-Week format for each group
I have the following Pyspark dataframe:
id1 id2 date col1 col2
1 1 2022-W01 5 10
1 2 2022-W02 2 5
1 3 2022-W03 3 8
1 5 2022-...
2
votes
2
answers
491
views
Pandas reindex date index by group revisited
This has been asked before and a working solution has been proposed here Pandas reindex dates in Groupby, which worked for me in the past, but it does not work any more.
So, to recap I need to reindex ...
1
vote
2
answers
346
views
Add different prefix to index by column value in pandas dataframe
I have a dataframe, with duplicated index, which I'm trying to reindex according to rowname of another dataframe. However, due to the duplicated index problem it's telling me cannot reindex on an axis ...
0
votes
1
answer
78
views
reindex a dataframe based on suffix
Assume we have a dataframe with 150 rows and 47 columns. The first 2 columns are always named 'name' and 'country'. The rest of the columns might have a different name but the suffices will always ...
0
votes
1
answer
551
views
Elasticsearch merge data from multiple indexes into merged index
My company uses an out of the box software, and that software export logs to Elasticsearch (and uses these logs). The software create an index per day for every data type, for example:
"A" ...
0
votes
3
answers
69
views
formating file with hours and date in the same column
our electricity provider think it could be very fun to make difficult to read csv files they provide.
This is precise electric consumption, every 30 min but in the SAME column you have hours, and date,...
0
votes
1
answer
261
views
What is the best approach to use remote reindexing API of elasticcluster when migration millions of documents?
I have approx. 100million documents in an index and i want to migrate it to new cluster using reindex API. I want to do it in the throttling manner.
I tried using request_per_seconds to 100000 but it ...
0
votes
0
answers
142
views
Merging conflicts of reindex api in elasticsearch
I need to merge conflicts in reindex api. But it just completele removes the dest old document and replaces it with what I provide in source instead of just changing the fields that source has and not ...
2
votes
1
answer
1k
views
How to reindex and change _type
We need to migrate a number of indexes from ElasticSearch 6.8 to ElasticSearch 7.x. To be able to do this, we now need to go back and fix a large number of documents are the _type field of these ...
0
votes
1
answer
151
views
Reindex pandas DataFrame to match index with another DataFrame
I have two pandas DataFrames with different (float) indices.
I want to update the second dataframe to match the first dataframe's index, updating its values to be interpolated using the index.
This is ...
1
vote
1
answer
552
views
Reindex selected _source fields using Rest high level client in java
I want to re_index only selected fields from my document in elasticsearch using Rest High level client.
I know the elasticsearch query to achieve that but I don't know it's equivalent query using rest ...
0
votes
1
answer
155
views
Pandas: Reindex/merge or re-allocate time series (of different time stamps) without interpolation
I have two pandas dataframes of different time series stamps, and I want to merge them together. The first dataframe is data from model and called df_model and the second one is observation which is ...
0
votes
1
answer
287
views
Sort pandas dataframe by two columns using key in one of them, "kind mergesort", not working
I am trying to sort a dataframe by 2 consecutive conditions, col[0] and col[2], the input data looks like this:
My desired output looks like this (sorted by col[0] and then by col[2]):
The second ...
0
votes
1
answer
111
views
Pandas reindex Dates To Subset of Dates from List
I am sorry, but there is online documentation and examples and I'm still not understanding. I have a pandas df with an index of dates in datetime format (yyyy-mm-dd) and I'm trying to resample or ...
3
votes
1
answer
628
views
In pandas, how to reindex(fill 0) in level 2 in multiindex
I have a dataframe pivot table with 2 level of index: month and rating. The rating should be 1,2,3 (not to be confused with the columns 1,2,3). I found that for some months, the rating could be ...
0
votes
1
answer
156
views
Sorting so that x matches y in data frame
I have two columns in a single data frame, ID_exp and ID_off. I want the order in ID_exp to match ID_off. From what I've read I need to reindex rows to get this to work. Since I can't simply reindex ...
1
vote
1
answer
108
views
Pandas resample with multiindex start- and enddate
Suppose I have a multi-index Pandas data frame with two index levels: month_begin and month_end
import pandas as pd
multi_index = pd.MultiIndex.from_tuples([("2022-03-01", "2022-03-31&...