Questions tagged [marklogic-dhf]

Use for questions about the MarkLogic Data Hub Framework. More people will see your question if you also tag it with "marklogic".

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

mlDeploy with Gradle fails offline, succeeds online

We've upgraded our setup to Gradle 6.4, DHF 5.2.0, and MarkLogic 11 from Gradle 5.2.1, DHF 5.2.0, and MarkLogic 10.9.2. The upgrades went smoothly, and gradlew mlDeploy runs successfully in online ...
Ashish Patil's user avatar
1 vote
0 answers
39 views

Not able to access Marklogic DHF, Getting 8010 access error

I am just trying to setup Marklogic DHF using the quick start guide and as soon as I enter the user/name password at localhost:8080 and hit enter, I got the below error. 2023-11-11 23:20:40.029 [http-...
Dixit Singla's user avatar
  • 2,560
1 vote
1 answer
47 views

mDeploy failing after DHF, Gradle and Marklogic upgrades

We have upgraded MarkLogic (from 10.07 to 11.0.0), Gradle(5.2.1 to 6.4) and DHF(5.2.0 to 5.8.1). All the upgrades were successful and mlDeploy for 3 out of our 4 code bases also executed successfully. ...
Ashish Patil's user avatar
0 votes
2 answers
66 views

MarkLogic: Find documents where at least one parent element does not have a particular child

Using the built-in MarkLogic cts functions, I'd like to be able to write a query which can find a document like the following -- where exists at least 1 element that does not have a <child1> ...
DarthDell's user avatar
0 votes
1 answer
26 views

Is it possible to create TDE View in XML metadata

I am adding set of metadata to XML and i want to expose this data to Power BI through a View (Basically creating a TDE).Is it possible to create TDE View in XML metadata
ramz123's user avatar
  • 199
0 votes
0 answers
36 views

How to identify the document in MarkLogic having special characters

How to identify the documents having special characters in Marklogic. Documents are in UTF-8 format
ramz123's user avatar
  • 199
1 vote
0 answers
40 views

MarkLogic : DHF5 mlDeploy XDMP-VALIDATEBADTYPE: (err:XQDY0027) validate

We are trying to deploy our codebase which is currently on DHF 5.2.0 and we are getting below error :- Local message: failed to apply resource at internal/apply-transform: Bad Request. Server Message: ...
SinghVK's user avatar
  • 313
1 vote
1 answer
84 views

MarkLogic directory-query with wildcard

I want to search in MarkLogic with using term-query in direcories that meet the pattern: /example/*/folder/ i.e. /example/foo/folder/ /example/foo/bar/folder/ and so on... I have something like this:...
Aspir's user avatar
  • 149
0 votes
2 answers
49 views

How to read log file of all the hosts in a marklogic cluster and search for a particular keyword and count the results

I had a requirement where I need to read the application log files of all host in cluster and search for a particular keyword and get the count of all. Log file in question is 9005_ErrorLog.txt and ...
SinghVK's user avatar
  • 313
0 votes
1 answer
63 views

writing some script in sourceQuery in DHF flow of Marklogic

I have one requirement while running DHF flow I can't use cts.uris in place of sourceQuery. I want to use some script. How can I achieve this? I tried like this.. but not working I have placed my ...
Jyothi MS's user avatar
0 votes
0 answers
39 views

Strategy for updating document state in MarkLogic Data Hub processing

Is there a best practice/strategy for updating document state when processing docs through MarkLogic Data Hub? I want to manage state in order to prevent re-harmonizing docs. Here's what I'm doing ...
Abhai Raj Llewellyn's user avatar
1 vote
1 answer
80 views

MarkLogic cts query to return distinct values in Data Hub

I'm looking for a MarkLogic cts query that I can use as a source query in Data Hub to return the distinct values on a combination of json property paths. For example, I 200K+ docs with this structure: ...
Abhai Raj Llewellyn's user avatar
1 vote
1 answer
76 views

MarkLogic DHF: How to get dynamic input from the command line, while executing any flow with steps

Step Definition is as below, { "lang" : "zxx", "name" : "test_harmonization", "description" : "", "type" : "CUSTOM&...
DevNinja's user avatar
  • 1,469
0 votes
1 answer
45 views

How to add a Dynamic value as a collection in DHF5 flows source query

I have a requirement to add current date as a collection in DHF5 flows source query. How to achieve this
ramz123's user avatar
  • 199
1 vote
1 answer
35 views

How to call collector customize module in MARKLOGIC-DHF 5.6.1

I need to call external collector customize module in DHF 5.6.1. How can I do that? When I passed module in source query, it works but in main.sjs, I can only see 'uri' is coming within 'content' ...
Manish's user avatar
  • 59
1 vote
0 answers
77 views

How do I change the default timeout setting in node.js in MarkLogic Grove?

I'm developing a UI application using MarkLogic Grove (React). I'm calling the REST API from the UI, but it times out in 2 minutes. I want to change this timeout of 2 minutes, how can I change it on ...
masakazu.higashino's user avatar
1 vote
1 answer
101 views

How to select documents using cts query that are in CollectionA and not in CollectionB

I am pretty new to using cts queries so I need some help with something that is, I would think, pretty simple. I need to select document that are in CollectionA but are not in CollectionB. How ...
Doug Gibbs's user avatar
2 votes
1 answer
32 views

Marklogic DHF5:- Can we return multiple content for one document in main.sjs

We are migrating from MarkLogic DHF4 to DHF5 (Data Hub Framework) We are having scenario where for one entity, depending on criteria we need to create more than one harmonize document for a single ...
SUB's user avatar
  • 23
1 vote
1 answer
36 views

Add attribute to a header element via makeEnvelope

I have an ingest flow in DHF5, using JavaScript, that calls makeEnvelope() to actually construct the XML envelope. The headers parameter is an object that gets turned into name/value pairs. Is there ...
eaolson's user avatar
  • 14.8k
1 vote
2 answers
45 views

Is the code available for the MarkLogic Pharma Research Hub example?

I'd like to use the MarkLogic Pharma Research Hub as an example to demonstrate the data hub functionality and end point. Is the code and documentation available anywhere?
Tim Meagher's user avatar
1 vote
0 answers
57 views

How can I get more detail about an error on a Data-hub job that's getting an Invalid Date error?

When I run a step in a Data-Hub 5.0 flow, I'm getting an "Invalid Date" error. The data I'm ingesting is from a csv file. It's being put into a database where all the fields are defined as ...
Doug Gibbs's user avatar
1 vote
1 answer
47 views

Marklogic include function in the custom transformation module

In the given function of custom transformation module how to call the DHF transformation module for MLCP(/data-hub/5/transforms/mlcp-flow-transform.sjs) in order to add the envelope in the output ...
Den_Alex's user avatar
1 vote
1 answer
102 views

MarkLogic Data Hub document metadata added by steps

According to the documentation: Document metadata added by steps For every content object outputted by a Data Hub step, regardless of the step type, Data Hub will add the following document metadata ...
Den_Alex's user avatar
1 vote
1 answer
138 views

which port number is ml-gradle calling when performing a mlDeploy

Which is the port number is used by gradle internally while performing mlDeploy, mlRelaodModules, and mlLoadModules tasks in MarkLogic?
ramz123's user avatar
  • 199
1 vote
0 answers
27 views

Not able to pass option parameter in Legacy Flows in Datahub 5.2

I am using Legacy Flows in Datahub 5.2 and passing parameter for harmonization in gradle command. When I am passing parameter like -Pdhf.source, I am getting error like: Task '.source=ABC' not found ...
ramz123's user avatar
  • 199
0 votes
1 answer
57 views

What are the different ways to do real time ingestion in MarkLogic

I am using MarkLogic-10 and DHF5 in my project. I have a requirement to do real time ingestion and harmonization. My requirement is to do step1 & 2 harmonization if any changes in the document(...
Ramz's user avatar
  • 99
2 votes
0 answers
655 views

DHF mlDeploy : Plugin [id: 'net.saliman.properties', version: '1.4.6'] was not found

While I was trying to deploy marklogic DHF project 5.4.x in my local using gradle mlDeploy, I was getting UnknownPlugin exception for net.saliman.properties. I am using gradle version 6.4. And I could ...
sharu's user avatar
  • 71
1 vote
1 answer
41 views

How to setup ODBC server using ml-gradle?

What parameters should I add to gradle.properties to automate ODBC server setup? I could not find in the reference site.
user avatar
1 vote
0 answers
36 views

Marklogic Datahub: Problem with the JSON file content while creating a "Mapping" step in Data Hub

I am using Data Hub Quick Start V5.4.0 and Marklogic Version 10.0-6.0 I have a JSON file with the following content in data-hub-STAGING, { "A1_Key": "A1_Value", "A2_Key":...
Sanjeeva Kumar Kanakam's user avatar
1 vote
0 answers
83 views

Marklogic Datahub: How to edit the TDEX file and generate the triples?

I am using Data Hub Quick Start V5.4.0 and Marklogic Version 10.0-6.0 I have a few tdex files in data-hub-FINAL-SCHEMAS. In the Marklogic console, When I select data-hub-FINAL and execute the SPARQL ...
Sanjeeva Kumar Kanakam's user avatar
0 votes
0 answers
68 views

Quickstart 4.3.2 "Bad Request", "messageCode":"XDMP-ELEMRIDXNOTFOUND", "message":"XDMP-ELEMRIDXNOTFOUND: cts.jsonPropertyReference(\"id\")

Ive been working through this tutorial https://marklogic.github.io/marklogic-data-hub/tutorial/4x/harmonize-order-data-by-custom-code/ and have followed it exactly as per the tutorial. The server ...
steve's user avatar
  • 393
0 votes
0 answers
18 views

How to stop all the IN/OUT flows in Marklogic Database

I want to stop all the IN/OUT flows in marklogic, It includes both ingestion and harmonization
ramz123's user avatar
  • 199
0 votes
2 answers
35 views

How to run mlunittest in adidfferent database in marklogic

I am running mlunittest in my FINAL database currently and it causing failure of some unittest cases. Is there any way to run the test cases in a different database
ramz123's user avatar
  • 199
0 votes
2 answers
109 views

How To Run A Nifi Template As Part Of A DH Ingestion Flow

We have a nifi template that used to extract data from a relational SQL Server DB and ingest every row of the tables as documents in Marklogic. Now we want to add DH in the mix, and run the nifi ...
user7222287's user avatar
2 votes
1 answer
89 views

How to check mandatory fields are present in the CSV while data ingestion in marklogic mlcp

I want to check few mandatory fields are available in CSV while ingestion of data through MLCP in marklogic. If those fields are not available, I need to ignore those records has to be ingested in ...
ramz123's user avatar
  • 199
0 votes
1 answer
157 views

Is it possible to make a doubly-inherited custom task in Gradle?

I'm working on a Gradle (Groovy, not Kotlin) library using a bunch of external libraries, and we have a case where we want to implement double-inheritance in our code off of a custom Task provided by ...
ahwitz's user avatar
  • 13
0 votes
1 answer
34 views

MarkLogic Data Hub cluster upgrade

We are looking at upgrading our v4.1 Data Hub to v5.2 on our 2 node ML v9.x cluster ( upgrading server v9.x to v10.x ) and need to understand the mechanism for the Data Hub upgrade process. For the ...
steve's user avatar
  • 393
2 votes
1 answer
41 views

How to do arithmetic operation in TDE

I have requirement to get previous date of one of the date field, for that I am doing below operation in TDE. But I am getting error on this operation. How can I achieve this <tde:column> <...
ramz123's user avatar
  • 199
1 vote
1 answer
72 views

Marklogic upgrade - creating local plugin repo

Our servers are not allowed to access internet, and I need to create and use a local repository ( the C:\Plugins directory ) located on the server having its MarkLogic Data Hub v4.1.1 upgraded. We ...
steve's user avatar
  • 393
1 vote
1 answer
36 views

Marklogic DHF 5.2.5 Lexicon based collector for step

Using Marklogic DHF 5 and trying to run a step for every value of a lexicon lookup (e.g cts.values). In DHF 4 this could be achieved by modifying the collector but DHF 5 seems to have eliminate that ...
Alexandru Mortan's user avatar
1 vote
1 answer
2k views

Gradle and proxy : Negotiate authentication error :Invalid name provided ( Mechanism level: KrbException : cannot locate default realm )

I have a problem getting gradle to get out to the internet from behind a proxy while trying to perform a MarkLogic data hub upgrade. I execute the following code as a root level user : ./gradlew ...
steve's user avatar
  • 393
1 vote
0 answers
50 views

Why are Attributes in Nested Enities not evaluated when matching?

we are using Marklogic DataHub QuickStart Version 5.1.0 for Mastering customer data into a Golden Record. The Entity Structure where we have problems with looks like this { "envelope": { ...
Andreas Felix's user avatar
4 votes
2 answers
122 views

Can ml-gradle deploy a query options file to an app server?

I have a MarkLogic cluster configured with DHF 5 and ml-gradle. I have multiple HTTP app servers and a couple of query options file. When I do an mlDeploy, the query options files are deployed to the ...
eaolson's user avatar
  • 14.8k
2 votes
1 answer
155 views

How to get the uri of the records returned by an optic query

I have written an optic query to get a list of documents and I need the URIs of those documents. let $sql := "select * from table1" let $results := op:from-sql($sql) I need the URIs of all ...
ramz123's user avatar
  • 199
0 votes
0 answers
60 views

not able to deploy Marklogic Modules through Jenkins Pipeline

I am trying to build CI in jenkins for marklogic but in Deploy step i am getting below error Execution failed for task ':ml-modules:mlDeployApp'. unsupported auth scheme: [Basic realm=public] How to ...
ramz123's user avatar
  • 199
0 votes
1 answer
135 views

Marklogic DataHub Management GUI

When you install the quick start Data Hub, it usually installs on port 8080 or 9000 ( which is user configurable ) e.g. https://developer.marklogic.com/learn/data-hub-quickstart/ I have installed a 5....
steve's user avatar
  • 393
1 vote
1 answer
20 views

Unable to Create range indexes using final-database.xml using marklogic datahub 5.1

I added few element range indexes in data-hub/src/main/ml-config/database-fields/final-database.xml <range-path-index xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="...
Pragya Kapoor's user avatar
1 vote
1 answer
307 views

unable to uprade marklogic data hub framework using gradle

I am trying to follow the recommendation to upgrade the DHF using gradle but I am running through an issue that I cannot het my head around. The build succeeds but the redeployment fails Any idea on ...
Rad's user avatar
  • 1,009
1 vote
1 answer
50 views

How do I call the backend MarkLogic API from my own API in MarkLogic Grove?

This question is a continuation of the question below. How do I handle the REST API in node.js in Marklogic Grove? I was able to implement a sample REST API that can be called when authenticated with ...
masaru.uemura's user avatar
2 votes
1 answer
99 views

How do I handle the REST API in node.js in Marklogic Grove?

I am developing a UI application using MarkLogic Grove(React). By default, REST API calls from web browser will be forwarded directly to MarkLogic. For example: http://hostname/v1/resources/foo/ If ...
masaru.uemura's user avatar