Questions tagged [hpcc-ecl]
ECL is a declarative programming language for High Performance Computing Cluster (HPCC). Do not confuse with Embeddable Common Lisp (ECL).
59
questions
0
votes
2
answers
55
views
Turn json string to data set
I'm able to get a json object back from Python, but it's a string. What's the best way to turn it into a data set in ECL. I feel I would have to save it to a file, then read it as json, but I'm ...
0
votes
1
answer
58
views
Is there a way to return a 3D list from embedded Python back to ECL?
I have a 2D array in Python which I want to return back to ELC, for further usage.
The recordset is of the following format
layout:=RECORD
INTEGER RED;
INTEGER GREEN;
INTEGER BLUE;
...
1
vote
1
answer
37
views
Running ECL from Visual Studio Code with no local repo
We have a HPCC Cluster that is connected to a Git Repository.
Is there any way to bypass a local compile when invoking ECL code from VSC?
This is a use case where teams of individuals will need to run ...
0
votes
1
answer
29
views
Unexpected results from a WHEN function
I have a BWR that executes a FUNCTIONMACRO. Within the FUNCTIONMACRO I execute a FUNCTION that reads a dataset, sorts and deduplicates that dataset and returns the results. Before it is returned, I ...
0
votes
1
answer
28
views
How to increase dali result limitation
In hpcc-ecl, when trying to increase the date size limit from 10 MB to approx ~5 GB using the option command (#OPTION('outputlimitMb',5048)), I got the following error.
Dali result outputs are ...
0
votes
1
answer
38
views
Is there a way visualise BLOB data in ECL?
I have an image in Python embedded in ECL(through the EMBED() function). The image is in the form of a numpy array. How do I return it back to ECL and are there any methods to visualize the image?
I ...
1
vote
2
answers
44
views
How to send POST request using sendgrip api in HPCC ECL?
Could someone kindly assist me in making a POST request call using the SendGrid API in HPCC ECL?
I have diligently referred to the documentation and examples provided in the forum, but I have not come ...
1
vote
1
answer
34
views
In my HPCC Cluster, what does this error in a workunit "Error receiving actinit data for graph: xx" really means?
Note: I am running code in a cluster with 16 slaves, HPCC version 6.4.40
I am running some ECL code that returns this error:
System error: 0: Graph graph2[14], SLAVE #1 [10.313.316.31:20100]: Error ...
0
votes
2
answers
33
views
How can I dynamically generate and output multiple files in ECL?
Let's say we want to dynamically create multiple files containing some data. How can that be achieved?
I tried the following for example, to output some data in multiple files on the fly:
file_names :=...
0
votes
1
answer
34
views
How to install Python libraries on a university cluster using Ubuntu terminal?
I have been working on a project that requires the installation of certain Python libraries onto a cluster. Initially, I was able to install these libraries on my local cluster by following the steps ...
0
votes
0
answers
20
views
How to write an ECL launch configuration for a university cluster?
I have been working with a local cluster for some time, but now I need to expand my work and use a university cluster. I have written a JSON file for the ECL launch configuration, but I am unsure if ...
0
votes
2
answers
24
views
Delay Transaction on Roxie
Scenario: a client wants to test a scenario where they run a transaction against a Roxie service, and they would like the transaction to take at least 50 seconds before it returns anything. What is a ...
1
vote
2
answers
39
views
Unable to get the filelist using STD.File.RemoteDirectory
We are trying to use a unix machine as landing zone for reading files into an HPCC Thor environment. Initially we would get the error
System error: -1: Failed to connect (setting host down) to ...
1
vote
1
answer
62
views
ModuleNotFoundError: No module named 'matplotlib' when using plt.imread() in embedded Python code
I am trying to read an image using plt.imread() in embedded Python code within my DATA function. The code is as follows:
IMPORT PYTHON3 AS PYTHON;
IMPORT Std;
IMPORT Image;
myRec := RECORD
STRING ...
1
vote
1
answer
51
views
How to convert hex data to byte data in ECL?
I am working on a code in ECL and trying to convert hex data into byte data. I am using the following line of code for this:
hex_data := Std.Str.ToHexPairs((>DATA<)ds.contents);
d1:=Std.Str....
0
votes
2
answers
58
views
Ouput To CSV Format
Is there a way to output to csv while keeping the original schema of the dataset? Output to csv will make the original dataset under the "line" column, and will introduce two additional ...
1
vote
1
answer
29
views
Linear Regression bundle OLS.ecl is not recognizing the Beta in math library
When calling the OLS Module in LinearRegression bundle for training the model. It prompts the error that "Object math doesnot have a member named Beta" and "unknown identifier Beta"...
1
vote
1
answer
55
views
ECL Equivalent to "rank over partition " SQL Statement
I am wondering what is the ECL equivalent to "rank over partition" sql statement. For a project that I will be working on, I need to assign a rank according to date. I know how to do this in ...
1
vote
1
answer
105
views
Linking multiple C++ files in ECL
I have C++ code spread across multiple C++ and header files and wanted to know how to link them in ECL. Should I install them as libraries (.cpp and header files) into the local cluster I am running ...
0
votes
1
answer
37
views
ECL Client Tool Despray using dfuplus
My colleagues and I have been exploring the ECL client tools, and we have been wondering if we can use the dfuplus command to despray dataset directly to our azure virtual machines? If this is ...
0
votes
0
answers
35
views
Write a script in ecl hpcc using Data management bundle GenIndex to create superfile
Using Data Mgmt module and GenIndex function. I want to create a superfile for roxie.
Following this link ( https://github.com/hpcc-systems/DataMgmt ) I tried the example method, but failed.
0
votes
1
answer
41
views
Send dataset to a postgress database from HPPSystem - ECL
I need to send an uploaded dataset from a csv inside the ECL Watch of HPCCSystem.
I don't know where the ODBC driver is installed on the server, I downloaded a VM template that contains the HPCCSystem ...
1
vote
2
answers
38
views
ROXIE: How to get a list of files associated with a query?
Probably a multipronged question coming from a Thor Dev-Apologies if the title is misleading.
I am looking to do the following:
Find all the queries on a given Roxie environment.
Find all the files ...
1
vote
2
answers
114
views
How to send POST request in HPCC ECL?
Can someone help me understand how to make a REST POST request in HPCC? I've read through the documentation but I couldn't find any examples. Any help would be appreciated.
0
votes
1
answer
38
views
Remote call to Roxie server from Thor
Assuming that we have the end-point to a Roxie server of interest, I was wondering if it is possible to make a remote call to it from a bwr script on Thor, and get the number of nodes that Roxie ...
0
votes
1
answer
64
views
Error when trying to hit an API using HTTPCALL in ECL
I try to hit the API by using HTTPCALL in ECL and it is giving following error.
I have tried hitting it using Postman, its working fine in it.
Here is the URL for API - 'https://api.waqi.info/feed/...
1
vote
1
answer
92
views
Passing a dataset as an argument for C++ embedding in ECL
I want to pass a dataset as an argument to a C++ embedded function. This code snippet seems to be working when we hardcode the dataset. The dataset is a BLOB type sprayed blk.dat file having ...
0
votes
1
answer
47
views
Why does STD.FILE.LogicalFileList() results not completely correlate with STD.FILE.GetLogicalFileAttribute(x,y)?
I execute the statement:
NrmFiles := NOTHOR(STD.File.LogicalFileList());
This produces a dataset of all the normal files contained in my HPCC environment.
It has the record layout:
string ...
1
vote
1
answer
51
views
Issue while setting up local HPCC cluster using WSL
When I start the cluster ( using the command etc/init.d/hpcc-init start). Starting mydali shows a TIMEOUT message. Something like this:
How do I solve this problem?
Thank You.
2
votes
3
answers
131
views
In HPCC ECL, when running a LOCAL, LOOKUP JOIN. Does the RHS dataset gets copied to all nodes, or kept distributed due to LOCAL?
Say I have a cluster of 400 machines, and 2 datasets. some_dataset_1 has 100M records, some_dataset_2 has 1M. I then run:
ds1:=DISTRIBUTE(some_dataset_1,hash(field_a));
ds2:=DISTRIBUTE(some_dataset_2,...
1
vote
2
answers
87
views
'Cannot read properties of undefined (reading 'eclccPath')' in HPCC Systems VS Code extension
How to resolve this error 'Cannot read properties of undefined (reading 'eclccPath')' in HPCC Systems VS Code extension.
I have installed ecl client tools. While installing I tried with both options '...
1
vote
0
answers
36
views
Updated support schedule for HPCC releases 8.6, 8.8 and 8.10
The schedule defined for the support of the HPCC releases, found at "https://wiki.hpccsystems.com/display/hpcc/Latest+release+and+documentation", seems outdated – 8.8 & 8.10 not present, ...
1
vote
1
answer
24
views
Roxie Query Returns (Failed to get response from server for dynamic file callback)
I have a roxie service that using package map to read the virtual files that are stored on roxie, so basically you pass a name and it finds the match adn returns the result. It is working for most of ...
1
vote
1
answer
45
views
How to determine the source of a job submission
I am searching for a way to determine how a job was initiated on the HPCC cluster.
There are several ways to submit a job. For example:
1- a manual submission via the ECL IDE / ECL Watch
2- an ...
1
vote
1
answer
33
views
HPCC Platform and Ganglia / Nagios plugin
With most users moving toward cloud-based deployments, will the Nagios / Ganglia plugin continue to be supported?
1
vote
1
answer
42
views
Find a specific index read
I'm having an issue with a roxie query when being called via an API. I've tested the query in HThor and the web service on 8002 and it runs perfectly. When calling it from an API I'm getting memory ...
1
vote
1
answer
51
views
ECL Decimal Casting
I'd like to ask the reason why this is happening.
When I was casting a result to DECIMAL5_2 when using COUNT, it works as planned when the COUNT is in a separate line from the casting:
c1 = COUNT($....
1
vote
1
answer
122
views
How can I split CSV files into multiple CSV files in runtime based on file size?
I have been trying to split CSV files into 1GB parts and the file size of the product varies such that it won't be helpful to split in a fixed number.
I have managed to find a way to estimate the ...
1
vote
1
answer
118
views
Iterate using the dataset columns
I have a dataset with more than 30 columns and I would like to calculate the correlation between each of them (and preferably put the results in a correlation matrix). Is there any way to create a ...
1
vote
1
answer
65
views
How do I manage FileScopes in HPCC version 7/8*?
I got a HPCC community version 8.0.0.1 installation running. Whenever I try to add a new FilScope trhoug ECL Watch nothing happens. It simply doesn't create it.
I got some version 6.4 clusters running ...
1
vote
1
answer
61
views
Monitoring a cron running on HPCC cluster
I have a cron scheduled to run on Thor cluster. Is there a way to monitor a cron running on HPCC Cluster and send a notification if the cron is not running due to a failure or system shutdown?
1
vote
1
answer
41
views
Encoding issues HPCC
I have been sent a dataset that contains data for US, UK, France, and Germany product dictionaries. With the German data, I'm having trouble displaying accents, etc.
I've sprayed the data as ASCII and ...
0
votes
1
answer
61
views
I wanted to return the obtained function output into a variable that I added into the RECORD using TRANSFORM. What does this error in image here mean?
distcomp (REAL4 lat1, REAL4 lon1,REAL4 lat2, REAL4 lon2, REAL4 dist) := FUNCTION
REAL4 lo1 := lon1/57.29577951;
REAL4 lo2 := lon2/57.29577951;
REAL4 la1 := lat1/57.29577951;
REAL4 la2 := lat2/57....
0
votes
1
answer
109
views
Unable to run the output for ECL in VSCODE. (HPCC Workunit)
I'm using HPCC workunit.
Every time I submit, the result is always empty (...empty...). I have tried creating new file and then working on it but nothing works.
1
(image)
What should I do ?
1
vote
1
answer
55
views
Fixed vs variable length STRING field definitions in ECL
I have a question regarding STRING field definitions.
Am I better off to fully qualify my STRING fields or allow them to be variable length?
For example I am working with a data file which contains ...
1
vote
1
answer
107
views
HPCC - ECL: Module "name" does not EXPORT an attribute main()
I have been explaining to others what "Module name does not EXPORT an attribute main()" error means and how to fix it.
Every time I answer this question I say it's because MODULE is a ...
0
votes
1
answer
93
views
HPCCSystems - ECL, Does NOTHOR runs on hthor or one node of thor?
Using ECL/HPCC, I'm working with a team on an issue. They're trying to run actions within child queries. During trying different solution a question came up:
Does ECL code executing with a NOTHOR ...
0
votes
1
answer
137
views
ECL - HPCC Testing Roxie query
I'm trying to write a Roxie query using ECL language. Is there a way to write and test the code without constantly publishing the query?
0
votes
1
answer
258
views
Migration Tips moving from ECL IDE to VSCode
I have been using the Windows based ECL IDE for 11 years, but would now like to migrate to using the Visual Studio Code with the ECL extension. Is there any available documentation, or can you share ...
2
votes
3
answers
137
views
LEFT and RIGHT keywords in ECL
I'm new to ECL and having trouble trying to understand RIGHT and LEFT keywords (used in DEDUP, GROUP and other functions). How do they work? The ECL reference guide has a single paragraph about what ...