Questions tagged [lucene]
The term Lucene refers to the open source Java fulltext search engine library, but also to the entire eco-system that grew around it, including lucene.net, solr, elasticsearch and zend-search-lucene.
11,832
questions
0
votes
1
answer
15
views
Lucene NET throwing Fatal Error: AccessViolationException
I am using Lucene NET v4.8 beta, and I have a method that is calling MaybeRefresh on a SearcherManager every 5 seconds. 99.9% of the time, everything works fine. However, 0.1% of the time, I am ...
0
votes
0
answers
14
views
Lucene indexes are getting deleted while adding a document
I am using Lucene.Net.
On some clients' machines all the indexes are getting deleted when we are trying to add a new document.
Process:
Check if document already exists
If Exists delete
Else add a ...
0
votes
1
answer
24
views
Elasticsearch - Match all arraylist field
I have few documents with array "items" , i want to only pick those documents where "items.name" is equal to "red". If there is any document with one red and another ...
0
votes
0
answers
16
views
how to use bm25 in spark
I have more than 1 million documents to search, and more than 100,000 keywords. Each keyword needs to search 10 most similar documents in the offline way.
So how to use BM25 in spark2, thanks ahead.
0
votes
1
answer
15
views
SOLR Exact Match with Numerics and "OR"
I have a field in SOLR (Completion_FY__TYPE_LIST) that I'm trying to perform an exact "OR" match on but as soon as I put a second value in it does a partial match.
Completion_FY__TYPE_LIST: ...
2
votes
0
answers
42
views
Match financial transactions and merchants
I have an Elastic index with billions of transactions.
An example tx:
{
"id": "54bfa9af-009a-437d-bd21-caaf651f7218",
"amount": 100.0,
"currency": &...
0
votes
0
answers
26
views
ElasticSearch: Sorted index vs inverted index
I wanna know whats the difference between the sorted vs inverted index in an elastic search (lucene).
From an ElasticSearch talk I read that the inverted index is kinda like a sorted dictionary: https:...
0
votes
0
answers
17
views
Memory consumption in my Jersey application keeps growing with time
Memory consumption in my application keeps growing with time. This app uses Lucene and search is performed using Rest endpoints that searched Lucene directories. Around 10 different directories are ...
0
votes
0
answers
8
views
About Lucene index postings list, why are all deltas between 0 and 255 during FOR encoding?
From this blog, it says that postings lists are split into blocks of 256 docs and then each block is compressed separately. But what if a term's postings list is [72, 373]? Is there anything that ...
0
votes
1
answer
17
views
Modifying fields(index) in solrcloud after loading data
solr - solr-8.9.0, java version "11.0.12" 2021-07-20 LTS
I am using external zookeeper ensemble with 3 apache zookeeper instance (apache-zookeeper-3.6.1-bin.tar.gz)
Solr cloud have been ...
-2
votes
0
answers
21
views
Converting regex to lucene in Elasticsearch Query DSL
I am trying to convert this regex to work in Kibana in a query search and can't seem to get it to work
((\+[2-9]|[2-9]){1}\d{10,}$)|((\+[2-9]){1}\d{8,}$)|((\+{0,1}1){0,1}((340)|(670)|(671)|(684)|(787)|...
0
votes
0
answers
34
views
High CPU usage elasticsearch
I have created an elasticsearch plugin. It is working absolutely fine. But, in the case of bulk indexing, it is getting a request timeout, and when I checked the CPU usage, it was showing 101.76% ...
1
vote
1
answer
28
views
ElasticSearch / OpenSearch term search with logical OR
I have been scratching my head for a while looking at OpenSearch documentation and stackoverflow questions. How can I do something like this:
Select documents WHERE studentId in [1234, 5678] OR ...
0
votes
0
answers
19
views
Example of new PatternTypingFilter
There's a new token filter called PatternTypingFilter which was added in Lucene in 8.8.0. I am trying to use this inside my TokenFilterFactory. I tried finding the implementation of this token filter ...
0
votes
0
answers
43
views
Changing type of the tokens generated by pattern tokenizer
I am creating a custom Pattern Tokenizer to change the type of the generated tokens. By incrementToken() function looks like the below code:
public boolean incrementToken() {
if (index >= str....
0
votes
1
answer
47
views
How to join data in unidirectional ManyToMany and ManyToOne relation with Hibernate Search
I'm new to Hibernate Search, I'm using Hibernate Search 6.1.4.Final along with Spring Boot 2.6.7, Spring Data JPA 2.6.7 and PostgreSQL 14. I have the following schema:
CREATE TABLE manufacturer
(
...
0
votes
0
answers
14
views
How to do partial text search in Cloudsearch
I have an indexed field of type "text" called obj_title in cloudsearch. It's value is
queryevents@abc.com
If I search for ts or *ts* or events or *events*. I get no results. I get results ...
-1
votes
0
answers
18
views
Download and Evict Lucene Index files once Index reader is created
Working on a scenario where we have heavy lucene indexes of size say 2GB+. They are stored in S3. If running an lucene server on a container have local cache path, it has finite memory say 25 MB. Need ...
0
votes
1
answer
42
views
Text search in similar clauses that contains misspelled words
We need to extract some information from a free text clause. Let's think we have a clause about a a ship leaving a port and going another port. The same meaning can be expressed in several ways like ...
0
votes
0
answers
42
views
Understanding the java code (Pattern Capture token filter)
I am currently making some changes in the default pattern capture group token filter code to meet my requirement. I am a beginner in Java, so I'm finding it a bit hard to fully understand the code and ...
0
votes
1
answer
72
views
Boost search score from data in another collection
I use Atlas Search to return a list of documents (using Mongoose):
const searchResults = await Resource.aggregate()
.search({
text: {
query: searchQuery,
path: ["...
2
votes
1
answer
61
views
SOLR MoreLikeThis with date field returns Invalid Date String
Tested with SOLR 6.6.6 and now with SOLR 7.7.3 we get the following error:
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
&...
0
votes
0
answers
34
views
Fix for highlight issue pattern capture group token filter
The default pattern capture token filter in elasticsearch gives the same start and end offset for each generated token, that is, the start and end offset as that of the input string. Is there any way ...
0
votes
0
answers
16
views
Lucene Query doesn't work after delete and adding same doc
I have two index folder, then delete all docs from second folder documents which storing in first directory, then adding, but without few documents.
In my case, I delete, then add the same document. ...
0
votes
0
answers
30
views
Elastic search doesn't find document with "-" in it
I have a query in ES:
POST <index>/_search
{
"_source":[
<list_of_fields_to_return>
],
"size":"6",
"from":"0",
&...
0
votes
1
answer
33
views
Lucene to use different files to search text
I am working on a task where I need to search a text using lucene. But here the requirement is to use the already created segment, .si, .cfe and .cfs files by other application.
I am able to get ...
1
vote
0
answers
32
views
Should I hit zookeeper or Solr node for indexing?
I read the following documents
https://solr.apache.org/guide/8_11/shards-and-indexing-data-in-solrcloud.html
https://solr.apache.org/guide/8_11/distributed-requests.html
https://solr.apache.org/guide/...
1
vote
1
answer
23
views
does replica shard has translog files
Elasticsearch maintain translog files for each shard , does replica shard has translog files ?
{
"_shards" : {
"total" : 2,
"successful" : 2,
"failed&...
0
votes
2
answers
29
views
Incrementing Datetime field by one day in Elasticsearch Production Cluster using painless script
I am facing difficulty in debugging a production level Elastic Search Index Date time field in yyyy-MM-dd format & i want to update/increment the datetime field by one day Eg- 2009-07-01 i want to ...
0
votes
1
answer
38
views
What is elasticsearch index , Lucene index and inverted index
Sometime i got confused between lucene index vs inverted index.From my knowledge inverted index contains individual tokens and elasticsearch index include shards.
Then what is lucene index what does ...
0
votes
0
answers
13
views
Solr - Return Ordinal/Index of the Documents in a Result as a Field on the Document
If I had a document collection in Solr like:
{ "id": "953427", "name": "Foo", "home": "A bar", "value": 42 },
{ "id": &...
0
votes
1
answer
28
views
Solr - Using the Result of a Raw Query Param Function in a Filter (fq)
I am trying to work with a function's results like the measure of distance.
I have a raw query param
distance=query({!func v='mul(geodist(store_location,45.98,-110.65),1.60934)'})
I know I can use it ...
0
votes
0
answers
20
views
How to export lucene index to other formats
Is there any utility to dump a lucene index to a file in CSV/JSON/XML or any other easily parsable format?
Or is there any API that can let me to iterate the entire index w/o using search query?
-...
0
votes
1
answer
38
views
Hibernate search with lucene does not index similar names correctly
I'm learning Hibernate Search 6.1.3.Final with Lucene 8.11.1 as backend and Spring Boot 2.6.6. I'm trying to create a search for product names, barcodes and manufacturers. Currently, I'm doing an ...
0
votes
0
answers
43
views
Java Regex returning only first match and not matching globally
I am using an apache lucene built in token filter called "PatternCaptureGroupTokenFilter." Basically, I am creating an elasticsearch plugin and needed this token filter for fulfilling a use ...
0
votes
0
answers
43
views
Indexing and searching text containing special characters
I have built a custom Analyzer and Tokenizer for indexing and searching words with special characters like $, (, ), {, }, :, &, _, ~, *, |, <, >, =, -, +, ,, !, ^, ", \, /, [, ], ?, @, %...
1
vote
0
answers
39
views
Lucene Index converting gen0 memory to unmanaged resource
I am using Lucene NET v4.8 beta, and I have noticed that my service consume more and more memory whenever I add huge amounts of data to the index.
I analyzed this through dotMemory, seen below. The ...
0
votes
0
answers
15
views
lucene - inheritance search
I have the below entities:
@Entity
@Indexed
@DiscriminatorValue("A")
public class CPAEntity extends ServiceEntity {
private static final long serialVersionUID = 1L;
@OneToOne(...
1
vote
1
answer
55
views
Using a Java pattern Globally PatternCaptureGroupTokenFilter
I am currently using the Pattern Capture Group Token Filter inside an Elasticsearch plugin. Currently, the plugin is working absolutely fine and is emitting the tokens also. But, the issue is that the ...
1
vote
1
answer
63
views
Lucene index files changing constantly even when there is no adding, updating, or deletion operations performed on it
I have noticed that, my lucene index segment files (file names) are always changing constantly, even when I am not performing any add, update, or delete operations. The only operations I am ...
0
votes
0
answers
49
views
Why is Lucene.Net.Store.Azure throwing IndexNotFounException "no segments* file found in AzureDirectory" exception after modifying the index?
For some reason when I delete a document from a Lucene index it is throwing a "no segments* file found in AzureDirectory" exception. This only happens when trying to load the data stored ...
0
votes
0
answers
13
views
Kibana search query using regular expression
I want to search url include a-id and b-id.
But i can't write search query for data has b-id
example url: "/aaa/{a-id}/bbb/{b-id}"
my search query(but isn't not working): "/aaa" ...
0
votes
0
answers
27
views
Can UUID v5 be Lucene Friendly
Based on the results of some tests here it is obvious that the worst case for Lucene UUID is UUID version 4 because of it's random nature.
I wanted to know if any one has any experience using UUID v5 ...
0
votes
1
answer
22
views
ES query to Kibana lucene query conversion
I am trying to convert the below ES query to Kibana lucene query but I dont understand the basics, unable to find the examples too.
{
"query": {
"bool": {
&...
0
votes
1
answer
30
views
Zend Framework compability with Lucene 8.4.1
I am using Lucene 3.5.0 for index creation. Zend-Framework-1.8.2 with PHP-7.1.12 is being used to search index.
I have upgraded Lucene from 3.5.0 to 8.4.1. The indexes are now created with Lucene 8.4....
0
votes
2
answers
36
views
How to deal with broad and heavy Elastic Search index
I have fat ElasticSearch index with plenty of fields. I'd like to enrich this index with another field containing arrays of 1024 dimensionality. However, I find it way too much.
Thus, I'm curious if ...
0
votes
0
answers
8
views
GraphDB Lucence Index
Any reason when creating an index the query
PREFIX elastic: <http://www.ontotext.com/connectors/elasticsearch#>
SELECT ?cntUri ?cntStr {
?cntUri elastic:listConnectors ?cntStr .
}
doesn't ...
1
vote
1
answer
91
views
Solr corrupt index exception
So, i have a solr version 7.7.2 running in two different nodes. i wanted to copy the the index data from one to another. all the other config files remain the same (including core and solrconfig.xml). ...
0
votes
1
answer
66
views
org.apache.lucene.queryparser.classic.ParseException: Cannot parse
I'm developing my own service based on Apache Lucene. This is a spring server, on the endpoint of which the search request comes - the fields to search for, and their values. This query needs to be ...
0
votes
0
answers
12
views
How to compare an existing time field with current time using elasticsearch in grafana
In the query tab of grafana, we need to use lucene query in order to query the elasticsearch database. I have a field called time already, I want to count the number of values in the time field tab ...