Skip to main content
Filter by
Sorted by
Tagged with
0 votes
1 answer
19 views

How to remove dynamic field from mapping with Elastic Reindex API

We have a dynamic field defined in multiple indexes that is of type geo_shape, and uses the points_only param. Due to a) the deprecation of points_only in version 7.x, and b) the fact that we don't ...
wonder95's user avatar
  • 4,165
2 votes
1 answer
80 views

How to resample a dataframe to stretch from start to enddate in intervals (containing 0 for not available values)

I have the following setup: I have sparse information about queries hitting my endpoint at certain timepoints in a csv file. I parse this csv file with dates according to date_format='ISO8601' in the ...
glades's user avatar
  • 4,446
0 votes
0 answers
50 views

SOLR 504 - bad gateway

When trying to reindex SOLR, at some point I only get failed tasks in my resque:queue. I try to retry them, but after 60 seconds (default bad gateway time imposed by ALB) I receive error 504. What ...
Krystian Janas's user avatar
1 vote
2 answers
78 views

How to update de value of the column headers on a pandas DataFrame?

I'm having problems on sorting my header in a way that shows all the meetings that happened on that year, than move on to the next year (M being meeting): Given the Data frame: Meeting M1/2023 M1/...
Ramiro's user avatar
  • 41
0 votes
2 answers
153 views

Postgresql Primary Key Index Maintenance

I refresh my indexes in my Postgresql database with the "reindex index concurrently" command. However, I never renewed my primary key indexes because it might damage my primary key indexes ...
Abdullah Ergin's user avatar
0 votes
0 answers
31 views

Elasticsearch reindexing - missing child mapping

I have elastic index with join mapping, I created new index, performed reindexing. The issue is related to mapping on new index, I have all fields in new mapping except fields from document that is ...
MSZMC's user avatar
  • 144
0 votes
0 answers
326 views

Is a full reindex required after adding a new field to Opensearch mapping? (Searchkick)

I am using the Searchkick gem to communicate with Opensearch. The Searchkick docs state: Control what data is indexed with the search_data method. Call Product.reindex after changing this method. It ...
user23579986's user avatar
1 vote
0 answers
81 views

Opensearch - Reindex periodically using Query DSL

I need to reindex data periodically like every 1 hour using a custom Query DSL. I reindex a subset of documents, i use an ingest pipeline and a custom script to update the new index. It looks like ...
Amaury Lecoq's user avatar
0 votes
2 answers
80 views

How to create index with new fields, parsing values from field in existing index?

I have an index that contains 1 document. This document has a field1 with value "B" and a field2 with value "C, D, E" (that is, the value in field2 is comma separated and can have ...
gbs's user avatar
  • 57
0 votes
0 answers
26 views

How to Reindex Solr in Production with Data Changes/Updates

I have deployed Solr in Solrcloud mode (Operator) and although I am pretty good with K8s I am fairly new to Solr. I have worked with nosql darabases but Solr interests me and so I deployed it in my ...
ghraz0r's user avatar
  • 31
-1 votes
1 answer
74 views

_reindex has suddenly stopped working inexplicably

I've been using the _reindex endpoint for some time now, about 2 months, without any problems. I'm running ES 8.6.2 on port 9500. The OS is W10. For some reason, as of this morning, any attempts to ...
mike rodent's user avatar
  • 15.2k
1 vote
1 answer
101 views

Reconcile channels with varying data acquisition frequencies

My dataframe has 8 columns for 4 different data recorded as a function of depth. However, each channel came with a different acquisition frequency. I would like to reconcile all these 4 data using a ...
roudan's user avatar
  • 4,052
0 votes
0 answers
144 views

PostgreSQL Scheduled Reindexing

I want to prepare a job that checks for fragmentation and reindexes those with fragmentation over 60. But I couldn't figure out how to do it. Currently, I have a query that only checks the index and ...
Abdullah Ergin's user avatar
0 votes
1 answer
44 views

How to change an Index of DataFrame?

I have a DataFrame with an index column as [0, 1,2,3]. I would like to change it to ['Q1','Q2','Q3','Q4']. How do I do this? I do not want to introduce a new column but change index only. I tried qtrs ...
Vivek Dabholkar's user avatar
0 votes
0 answers
35 views

Same Mapping Retained After Reindexing in Elasticsearch

I recently attempted to update the mapping of an Elasticsearch index by creating a new index with a specific mapping and then reindexing the data from the old index into the new one. However, despite ...
user avatar
1 vote
1 answer
135 views

I would like to know how to reindex Express program (.CDX) files using PHP

Currently, I can insert data using the xbase library in PHP, but I'm facing an issue where I have to manually reindex the data in the Express program after inserting it. I would like to know how to ...
Jib Jib's user avatar
  • 11
0 votes
0 answers
355 views

Elasticsearch conflict during reindex with opType Create

I've used reindex API with opType=create and I've expected for conflicts to be impossible because only new records will be inserted. But I get conflict message. Whats wrong? Does elasticsearch has ...
xander27's user avatar
  • 3,094
0 votes
1 answer
156 views

Magento CLI fails to return any output when I type magento commands like reindex

Magento CLI fails to return any output but instead goes exits to the next line, I am trying to execute reindex but I am unsure it is even executing. I really need it to work because it is causing ...
Patricode's user avatar
0 votes
0 answers
44 views

Getting mapper_parsing_exception on creating a deleted elastic index again

I have deleted an existing elastic index as part of reindexing, created a temp index and moved all the documents to the temp index before deleting this index. Now I'm getting this error when I'm ...
DSM's user avatar
  • 1
1 vote
1 answer
69 views

Looking to reindex dates based on a groupby of Grid Cell

I am attempting to do something similar to the reindexing in Pandas reindex date index by group revisited except I have one extra level of indexing involved. My data is in the following structure: [...
arctic_climate_science's user avatar
0 votes
1 answer
1k views

Does Re-index in Elasticsearch has zero-down time?

Currently, I'm working with Elasticsearch, I'm interested in reindexing because it has no time down. I have a question if my index has very much data (1 million documents for example), when I call ...
Astart's user avatar
  • 11
0 votes
1 answer
120 views

How to manage scroll context on reindex with Elasticsearch and Dotnet

I have a method in .NET which reindexes indices in Elasticsearch. When I execute it, the tasks generated are in error with the reason Trying to create too many scroll contexts. Must be less than or ...
Maxime Dousselaere's user avatar
0 votes
0 answers
97 views

How does Pandas DataFrame reindex function work with the Copy Parameter when it is set to 'False'?

I could not quite understand the functionality of copy using reindex. When I tested, copy=False is working only if I am giving exactly same index/columns, in sequence. But this not work if I am ...
Aswin K's user avatar
1 vote
2 answers
94 views

Pandas df.reindex() doesn't work for level>0 for MultiIndex

I've got two DataFrames A and B, both with a MultiIndex in the columns. B however has the order of the columns switched up. Now I would simply like to get the columns of B in the same order as in A. ...
Nicoleue's user avatar
1 vote
1 answer
26 views

Need to have non NaN value within same sampling window in a DataFrame

I am trying to write an implementation of the GAPS operator in Kairos in Python. What the GAPS operator does is introduces NaN values where there is no data point based on the sampling frequency. ` ...
nkstack's user avatar
  • 11
0 votes
1 answer
57 views

Is there anyway to skip `Reindexing in Elastic Search with database

Currently working with a large database that typically takes 2-3 days to reindex completely, and I only need to add a new field to the mapping for a specific table. What could be a possible solution ...
Hamza Dabeer khan's user avatar
1 vote
3 answers
90 views

Sort/reindex a pandas multiindex dataframe when level 1 indicies are different for each level 0 index. Only one level 0 group needs to be sorted

I am struggling with finding a way to sort the rows in a grouped pandas dataframe with two levels of indicies. This is an example of what a dataframe df I have: I could not paste the table in a nice ...
confusedstudent's user avatar
0 votes
3 answers
97 views

trouble reindexing python dictionary

I have a simple dictionary I deleted an index entry in the middle. I'm having trouble re-ordering my_dict index keys to be in sequence after I deleted an entry in the middle. my_dict = { 0: { ...
Brendan's user avatar
  • 11
1 vote
1 answer
71 views

Manipulating indices in a pandas dataframe

The problem: I want to shift the index by adding 2 (index+=2) to match it with the indices of a .csv file. I have a dataframe which looks as follows: Counts Counts 7 10951.0 870.0 8 8653.0 1772.0 ...
philiVanilli25's user avatar
0 votes
2 answers
51 views

Pandas: Enforcing consistent values for inner index across all outer index values

I have a dataset indexed by entity_id and timestamp, but certain entity_id's do not have entries at all timestamps (not missing values, just no row). I'm trying to enforce consistent timestamps across ...
Brian Burrows's user avatar
0 votes
2 answers
37 views

reindex df on axis with duplicates

I have the following df: df = pd.DataFrame({"col1": [0,0,0,1,1,1], "col2": ["a", "b", "c", "D", "E", "F"] }) and want ...
user21402593's user avatar
0 votes
1 answer
39 views

Is it expected behavior to manually reindex a Frame after sorting? Is this a bug?

I have a time series of stellar flux that is periodic in nature. For this data I created a DataFrame with a time column and flux column. file: Path = localdir / file_path.csv est_period: float = #...
Gerald's user avatar
  • 166
1 vote
2 answers
149 views

Reindexing Pandas based on daterange

I am trying to reindex the dates in pandas. This is because there are dates which are missing, such as weekends or national hollidays. To do this I am using the following code: import pandas as pd ...
Slartibartfast's user avatar
0 votes
1 answer
166 views

Reindex Pyspark Dataframe with dates in Year-Week format for each group

I have the following Pyspark dataframe: id1 id2 date col1 col2 1 1 2022-W01 5 10 1 2 2022-W02 2 5 1 3 2022-W03 3 8 1 5 2022-...
Marco's user avatar
  • 1,235
2 votes
2 answers
491 views

Pandas reindex date index by group revisited

This has been asked before and a working solution has been proposed here Pandas reindex dates in Groupby, which worked for me in the past, but it does not work any more. So, to recap I need to reindex ...
izhako's user avatar
  • 143
1 vote
2 answers
346 views

Add different prefix to index by column value in pandas dataframe

I have a dataframe, with duplicated index, which I'm trying to reindex according to rowname of another dataframe. However, due to the duplicated index problem it's telling me cannot reindex on an axis ...
Joy Zheng's user avatar
0 votes
1 answer
78 views

reindex a dataframe based on suffix

Assume we have a dataframe with 150 rows and 47 columns. The first 2 columns are always named 'name' and 'country'. The rest of the columns might have a different name but the suffices will always ...
alexntou11's user avatar
0 votes
1 answer
551 views

Elasticsearch merge data from multiple indexes into merged index

My company uses an out of the box software, and that software export logs to Elasticsearch (and uses these logs). The software create an index per day for every data type, for example: "A" ...
Amit Assa's user avatar
0 votes
3 answers
69 views

formating file with hours and date in the same column

our electricity provider think it could be very fun to make difficult to read csv files they provide. This is precise electric consumption, every 30 min but in the SAME column you have hours, and date,...
BenjiBoy's user avatar
  • 185
0 votes
1 answer
261 views

What is the best approach to use remote reindexing API of elasticcluster when migration millions of documents?

I have approx. 100million documents in an index and i want to migrate it to new cluster using reindex API. I want to do it in the throttling manner. I tried using request_per_seconds to 100000 but it ...
Kartik Luthra's user avatar
0 votes
0 answers
142 views

Merging conflicts of reindex api in elasticsearch

I need to merge conflicts in reindex api. But it just completele removes the dest old document and replaces it with what I provide in source instead of just changing the fields that source has and not ...
Ali Zarean's user avatar
2 votes
1 answer
1k views

How to reindex and change _type

We need to migrate a number of indexes from ElasticSearch 6.8 to ElasticSearch 7.x. To be able to do this, we now need to go back and fix a large number of documents are the _type field of these ...
mieliespoor's user avatar
  • 1,053
0 votes
1 answer
151 views

Reindex pandas DataFrame to match index with another DataFrame

I have two pandas DataFrames with different (float) indices. I want to update the second dataframe to match the first dataframe's index, updating its values to be interpolated using the index. This is ...
Grismar's user avatar
  • 30.5k
1 vote
1 answer
552 views

Reindex selected _source fields using Rest high level client in java

I want to re_index only selected fields from my document in elasticsearch using Rest High level client. I know the elasticsearch query to achieve that but I don't know it's equivalent query using rest ...
Aman Mehta's user avatar
0 votes
1 answer
155 views

Pandas: Reindex/merge or re-allocate time series (of different time stamps) without interpolation

I have two pandas dataframes of different time series stamps, and I want to merge them together. The first dataframe is data from model and called df_model and the second one is observation which is ...
Xu Shan's user avatar
  • 305
0 votes
1 answer
287 views

Sort pandas dataframe by two columns using key in one of them, "kind mergesort", not working

I am trying to sort a dataframe by 2 consecutive conditions, col[0] and col[2], the input data looks like this: My desired output looks like this (sorted by col[0] and then by col[2]): The second ...
Fcallejas's user avatar
0 votes
1 answer
111 views

Pandas reindex Dates To Subset of Dates from List

I am sorry, but there is online documentation and examples and I'm still not understanding. I have a pandas df with an index of dates in datetime format (yyyy-mm-dd) and I'm trying to resample or ...
user2100039's user avatar
  • 1,366
3 votes
1 answer
628 views

In pandas, how to reindex(fill 0) in level 2 in multiindex

I have a dataframe pivot table with 2 level of index: month and rating. The rating should be 1,2,3 (not to be confused with the columns 1,2,3). I found that for some months, the rating could be ...
Summer's user avatar
  • 251
0 votes
1 answer
156 views

Sorting so that x matches y in data frame

I have two columns in a single data frame, ID_exp and ID_off. I want the order in ID_exp to match ID_off. From what I've read I need to reindex rows to get this to work. Since I can't simply reindex ...
pidge gunderson's user avatar
1 vote
1 answer
108 views

Pandas resample with multiindex start- and enddate

Suppose I have a multi-index Pandas data frame with two index levels: month_begin and month_end import pandas as pd multi_index = pd.MultiIndex.from_tuples([("2022-03-01", "2022-03-31&...
W. Walter's user avatar
  • 349

1
2 3 4 5
10