Questions tagged [marklogic-dhf]
Use for questions about the MarkLogic Data Hub Framework. More people will see your question if you also tag it with "marklogic".
190
questions
0
votes
0
answers
19
views
mlDeploy with Gradle fails offline, succeeds online
We've upgraded our setup to Gradle 6.4, DHF 5.2.0, and MarkLogic 11 from Gradle 5.2.1, DHF 5.2.0, and MarkLogic 10.9.2. The upgrades went smoothly, and gradlew mlDeploy runs successfully in online ...
1
vote
0
answers
39
views
Not able to access Marklogic DHF, Getting 8010 access error
I am just trying to setup Marklogic DHF using the quick start guide and as soon as I enter the user/name password at localhost:8080 and hit enter, I got the below error.
2023-11-11 23:20:40.029 [http-...
1
vote
1
answer
47
views
mDeploy failing after DHF, Gradle and Marklogic upgrades
We have upgraded MarkLogic (from 10.07 to 11.0.0), Gradle(5.2.1 to 6.4) and DHF(5.2.0 to 5.8.1).
All the upgrades were successful and mlDeploy for 3 out of our 4 code bases also executed successfully.
...
0
votes
2
answers
66
views
MarkLogic: Find documents where at least one parent element does not have a particular child
Using the built-in MarkLogic cts functions, I'd like to be able to write a query which can find a document like the following -- where exists at least 1 element that does not have a <child1> ...
0
votes
1
answer
26
views
Is it possible to create TDE View in XML metadata
I am adding set of metadata to XML and i want to expose this data to Power BI through a View (Basically creating a TDE).Is it possible to create TDE View in XML metadata
0
votes
0
answers
36
views
How to identify the document in MarkLogic having special characters
How to identify the documents having special characters in Marklogic. Documents are in UTF-8 format
1
vote
0
answers
40
views
MarkLogic : DHF5 mlDeploy XDMP-VALIDATEBADTYPE: (err:XQDY0027) validate
We are trying to deploy our codebase which is currently on DHF 5.2.0 and we are getting below error :-
Local message: failed to apply resource at internal/apply-transform: Bad Request. Server Message: ...
1
vote
1
answer
84
views
MarkLogic directory-query with wildcard
I want to search in MarkLogic with using term-query in direcories that meet the pattern:
/example/*/folder/
i.e.
/example/foo/folder/
/example/foo/bar/folder/
and so on...
I have something like this:...
0
votes
2
answers
49
views
How to read log file of all the hosts in a marklogic cluster and search for a particular keyword and count the results
I had a requirement where I need to read the application log files of all host in cluster and search for a particular keyword and get the count of all.
Log file in question is 9005_ErrorLog.txt and ...
0
votes
1
answer
63
views
writing some script in sourceQuery in DHF flow of Marklogic
I have one requirement while running DHF flow I can't use cts.uris in place of sourceQuery. I want to use some script.
How can I achieve this?
I tried like this.. but not working
I have placed my ...
0
votes
0
answers
39
views
Strategy for updating document state in MarkLogic Data Hub processing
Is there a best practice/strategy for updating document state when processing docs through MarkLogic Data Hub? I want to manage state in order to prevent re-harmonizing docs. Here's what I'm doing ...
1
vote
1
answer
80
views
MarkLogic cts query to return distinct values in Data Hub
I'm looking for a MarkLogic cts query that I can use as a source query in Data Hub to return the distinct values on a combination of json property paths.
For example, I 200K+ docs with this structure:
...
1
vote
1
answer
76
views
MarkLogic DHF: How to get dynamic input from the command line, while executing any flow with steps
Step Definition is as below,
{
"lang" : "zxx",
"name" : "test_harmonization",
"description" : "",
"type" : "CUSTOM&...
0
votes
1
answer
45
views
How to add a Dynamic value as a collection in DHF5 flows source query
I have a requirement to add current date as a collection in DHF5 flows source query. How to achieve this
1
vote
1
answer
35
views
How to call collector customize module in MARKLOGIC-DHF 5.6.1
I need to call external collector customize module in DHF 5.6.1.
How can I do that?
When I passed module in source query, it works but in main.sjs, I can only see 'uri' is coming within 'content' ...
1
vote
0
answers
77
views
How do I change the default timeout setting in node.js in MarkLogic Grove?
I'm developing a UI application using MarkLogic Grove (React).
I'm calling the REST API from the UI, but it times out in 2 minutes.
I want to change this timeout of 2 minutes, how can I change it on ...
1
vote
1
answer
101
views
How to select documents using cts query that are in CollectionA and not in CollectionB
I am pretty new to using cts queries so I need some help with something that is, I would think, pretty simple. I need to select document that are in CollectionA but are not in CollectionB. How ...
2
votes
1
answer
32
views
Marklogic DHF5:- Can we return multiple content for one document in main.sjs
We are migrating from MarkLogic DHF4 to DHF5 (Data Hub Framework)
We are having scenario where for one entity, depending on criteria we need to create more than one harmonize document for a single ...
1
vote
1
answer
36
views
Add attribute to a header element via makeEnvelope
I have an ingest flow in DHF5, using JavaScript, that calls makeEnvelope() to actually construct the XML envelope. The headers parameter is an object that gets turned into name/value pairs. Is there ...
1
vote
2
answers
45
views
Is the code available for the MarkLogic Pharma Research Hub example?
I'd like to use the MarkLogic Pharma Research Hub as an example to demonstrate the data hub functionality and end point. Is the code and documentation available anywhere?
1
vote
0
answers
57
views
How can I get more detail about an error on a Data-hub job that's getting an Invalid Date error?
When I run a step in a Data-Hub 5.0 flow, I'm getting an "Invalid Date" error. The data I'm ingesting is from a csv file. It's being put into a database where all the fields are defined as ...
1
vote
1
answer
47
views
Marklogic include function in the custom transformation module
In the given function of custom transformation module how to call the DHF transformation module for MLCP(/data-hub/5/transforms/mlcp-flow-transform.sjs) in order to add the envelope in the output ...
1
vote
1
answer
102
views
MarkLogic Data Hub document metadata added by steps
According to the documentation:
Document metadata added by steps
For every content object outputted by a Data Hub step, regardless of the step type, Data Hub will add the following document metadata ...
1
vote
1
answer
138
views
which port number is ml-gradle calling when performing a mlDeploy
Which is the port number is used by gradle internally while performing mlDeploy, mlRelaodModules, and mlLoadModules tasks in MarkLogic?
1
vote
0
answers
27
views
Not able to pass option parameter in Legacy Flows in Datahub 5.2
I am using Legacy Flows in Datahub 5.2 and passing parameter for harmonization in gradle command.
When I am passing parameter like -Pdhf.source, I am getting error like:
Task '.source=ABC' not found ...
0
votes
1
answer
57
views
What are the different ways to do real time ingestion in MarkLogic
I am using MarkLogic-10 and DHF5 in my project. I have a requirement to do real time ingestion and harmonization. My requirement is to do step1 & 2 harmonization if any changes in the document(...
2
votes
0
answers
655
views
DHF mlDeploy : Plugin [id: 'net.saliman.properties', version: '1.4.6'] was not found
While I was trying to deploy marklogic DHF project 5.4.x in my local using gradle mlDeploy, I was getting UnknownPlugin exception for net.saliman.properties. I am using gradle version 6.4.
And I could ...
1
vote
1
answer
41
views
How to setup ODBC server using ml-gradle?
What parameters should I add to gradle.properties to automate ODBC server setup?
I could not find in the reference site.
1
vote
0
answers
36
views
Marklogic Datahub: Problem with the JSON file content while creating a "Mapping" step in Data Hub
I am using Data Hub Quick Start V5.4.0 and Marklogic Version 10.0-6.0
I have a JSON file with the following content in data-hub-STAGING,
{
"A1_Key": "A1_Value",
"A2_Key":...
1
vote
0
answers
83
views
Marklogic Datahub: How to edit the TDEX file and generate the triples?
I am using Data Hub Quick Start V5.4.0 and Marklogic Version 10.0-6.0
I have a few tdex files in data-hub-FINAL-SCHEMAS.
In the Marklogic console, When I select data-hub-FINAL and execute the SPARQL ...
0
votes
0
answers
68
views
Quickstart 4.3.2 "Bad Request", "messageCode":"XDMP-ELEMRIDXNOTFOUND", "message":"XDMP-ELEMRIDXNOTFOUND: cts.jsonPropertyReference(\"id\")
Ive been working through this tutorial https://marklogic.github.io/marklogic-data-hub/tutorial/4x/harmonize-order-data-by-custom-code/ and have followed it exactly as per the tutorial.
The server ...
0
votes
0
answers
18
views
How to stop all the IN/OUT flows in Marklogic Database
I want to stop all the IN/OUT flows in marklogic, It includes both ingestion and harmonization
0
votes
2
answers
35
views
How to run mlunittest in adidfferent database in marklogic
I am running mlunittest in my FINAL database currently and it causing failure of some unittest cases. Is there any way to run the test cases in a different database
0
votes
2
answers
109
views
How To Run A Nifi Template As Part Of A DH Ingestion Flow
We have a nifi template that used to extract data from a relational SQL Server DB and ingest every row of the tables as documents in Marklogic.
Now we want to add DH in the mix, and run the nifi ...
2
votes
1
answer
89
views
How to check mandatory fields are present in the CSV while data ingestion in marklogic mlcp
I want to check few mandatory fields are available in CSV while ingestion of data through MLCP in marklogic. If those fields are not available, I need to ignore those records has to be ingested in ...
0
votes
1
answer
157
views
Is it possible to make a doubly-inherited custom task in Gradle?
I'm working on a Gradle (Groovy, not Kotlin) library using a bunch of external libraries, and we have a case where we want to implement double-inheritance in our code off of a custom Task provided by ...
0
votes
1
answer
34
views
MarkLogic Data Hub cluster upgrade
We are looking at upgrading our v4.1 Data Hub to v5.2 on our 2 node ML v9.x cluster ( upgrading server v9.x to v10.x ) and need to understand the mechanism for the Data Hub upgrade process. For the ...
2
votes
1
answer
41
views
How to do arithmetic operation in TDE
I have requirement to get previous date of one of the date field, for that I am doing below operation in TDE. But I am getting error on this operation. How can I achieve this
<tde:column>
<...
1
vote
1
answer
72
views
Marklogic upgrade - creating local plugin repo
Our servers are not allowed to access internet, and I need to create and use a local repository ( the C:\Plugins directory ) located on the server having its MarkLogic Data Hub v4.1.1 upgraded.
We ...
1
vote
1
answer
36
views
Marklogic DHF 5.2.5 Lexicon based collector for step
Using Marklogic DHF 5 and trying to run a step for every value of a lexicon lookup (e.g cts.values). In DHF 4 this could be achieved by modifying the collector but DHF 5 seems to have eliminate that ...
1
vote
1
answer
2k
views
Gradle and proxy : Negotiate authentication error :Invalid name provided ( Mechanism level: KrbException : cannot locate default realm )
I have a problem getting gradle to get out to the internet from behind a proxy while trying to perform a MarkLogic data hub upgrade.
I execute the following code as a root level user :
./gradlew ...
1
vote
0
answers
50
views
Why are Attributes in Nested Enities not evaluated when matching?
we are using Marklogic DataHub QuickStart Version 5.1.0 for Mastering customer data into a Golden Record. The Entity Structure where we have problems with looks like this
{
"envelope": {
...
4
votes
2
answers
122
views
Can ml-gradle deploy a query options file to an app server?
I have a MarkLogic cluster configured with DHF 5 and ml-gradle. I have multiple HTTP app servers and a couple of query options file.
When I do an mlDeploy, the query options files are deployed to the ...
2
votes
1
answer
155
views
How to get the uri of the records returned by an optic query
I have written an optic query to get a list of documents and I need the URIs of those documents.
let $sql := "select * from table1"
let $results := op:from-sql($sql)
I need the URIs of all ...
0
votes
0
answers
60
views
not able to deploy Marklogic Modules through Jenkins Pipeline
I am trying to build CI in jenkins for marklogic but in Deploy step i am getting below error
Execution failed for task ':ml-modules:mlDeployApp'.
unsupported auth scheme: [Basic realm=public]
How to ...
0
votes
1
answer
135
views
Marklogic DataHub Management GUI
When you install the quick start Data Hub, it usually installs on port 8080 or 9000 ( which is user configurable )
e.g. https://developer.marklogic.com/learn/data-hub-quickstart/
I have installed a 5....
1
vote
1
answer
20
views
Unable to Create range indexes using final-database.xml using marklogic datahub 5.1
I added few element range indexes in data-hub/src/main/ml-config/database-fields/final-database.xml
<range-path-index xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="...
1
vote
1
answer
307
views
unable to uprade marklogic data hub framework using gradle
I am trying to follow the recommendation to upgrade the DHF using gradle but I am running through an issue that I cannot het my head around.
The build succeeds but the redeployment fails
Any idea on ...
1
vote
1
answer
50
views
How do I call the backend MarkLogic API from my own API in MarkLogic Grove?
This question is a continuation of the question below.
How do I handle the REST API in node.js in Marklogic Grove?
I was able to implement a sample REST API that can be called when authenticated with ...
2
votes
1
answer
99
views
How do I handle the REST API in node.js in Marklogic Grove?
I am developing a UI application using MarkLogic Grove(React).
By default, REST API calls from web browser will be forwarded directly to MarkLogic.
For example: http://hostname/v1/resources/foo/
If ...