Questions tagged [hpcc]
High Performance Computing Cluster (HPCC)
51
questions
-1
votes
1
answer
43
views
Is it always necessary to define a record structure while using ECL?
Every time I make use of ECL, I have to define a record structure for any file in use (CSV, XML, JSON). However, is there a way of making use of these files on ECL without defining the record ...
0
votes
1
answer
59
views
Can we make use of AWS S3 connectivity with HPCC Cluster without administrative access to the cluster?
So I wanted to look on how one can develop AWS S3 connectivity with a HPCC Cluster. Tools like s3cmd and aws cli work well if I am accessing files from my S3 bucket onto my localhost cluster by ...
0
votes
1
answer
30
views
MongoDB plugin for ECL language
Trying to load all json docs from a mongodb server in a dataset using the mongodb plugin for the ECL language.
IMPORT MongoDB;
server := 'cluster0.smthing.mongodb.net';
user := 'uname';
pwd := '...
0
votes
2
answers
64
views
Turn json string to data set
I'm able to get a json object back from Python, but it's a string. What's the best way to turn it into a data set in ECL. I feel I would have to save it to a file, then read it as json, but I'm ...
0
votes
1
answer
61
views
Is there a way to return a 3D list from embedded Python back to ECL?
I have a 2D array in Python which I want to return back to ELC, for further usage.
The recordset is of the following format
layout:=RECORD
INTEGER RED;
INTEGER GREEN;
INTEGER BLUE;
...
0
votes
1
answer
43
views
Is there a way visualise BLOB data in ECL?
I have an image in Python embedded in ECL(through the EMBED() function). The image is in the form of a numpy array. How do I return it back to ECL and are there any methods to visualize the image?
I ...
1
vote
2
answers
52
views
How to send POST request using sendgrip api in HPCC ECL?
Could someone kindly assist me in making a POST request call using the SendGrid API in HPCC ECL?
I have diligently referred to the documentation and examples provided in the forum, but I have not come ...
0
votes
0
answers
8
views
How to HTTPCALL in HPCCECL having authorization , JSON output GET Method as it gives empty string
Can you please suggest how to optimize below code to fetch roxie queries related values properly
IMPORT Std;
WUListQueriesResponse :- Is same as we get in layout .
QueryList := HTTPCALL('https://...
1
vote
1
answer
46
views
In my HPCC Cluster, what does this error in a workunit "Error receiving actinit data for graph: xx" really means?
Note: I am running code in a cluster with 16 slaves, HPCC version 6.4.40
I am running some ECL code that returns this error:
System error: 0: Graph graph2[14], SLAVE #1 [10.313.316.31:20100]: Error ...
0
votes
1
answer
40
views
How to install Python libraries on a university cluster using Ubuntu terminal?
I have been working on a project that requires the installation of certain Python libraries onto a cluster. Initially, I was able to install these libraries on my local cluster by following the steps ...
0
votes
0
answers
24
views
How to write an ECL launch configuration for a university cluster?
I have been working with a local cluster for some time, but now I need to expand my work and use a university cluster. I have written a JSON file for the ECL launch configuration, but I am unsure if ...
0
votes
2
answers
24
views
Delay Transaction on Roxie
Scenario: a client wants to test a scenario where they run a transaction against a Roxie service, and they would like the transaction to take at least 50 seconds before it returns anything. What is a ...
0
votes
0
answers
19
views
0: request to https://play.hpccsystems.com:18010/WsSMC/Activity.json failed, reason: unable to get local issuer certificate
Every time I run an ECL file in VS Code, I am getting this error:
0: request to https://play.hpccsystems.com:18010/WsSMC/Activity.json failed, reason: unable to get local issuer certificate. However, ...
1
vote
1
answer
78
views
ModuleNotFoundError: No module named 'matplotlib' when using plt.imread() in embedded Python code
I am trying to read an image using plt.imread() in embedded Python code within my DATA function. The code is as follows:
IMPORT PYTHON3 AS PYTHON;
IMPORT Std;
IMPORT Image;
myRec := RECORD
STRING ...
1
vote
1
answer
55
views
How to convert hex data to byte data in ECL?
I am working on a code in ECL and trying to convert hex data into byte data. I am using the following line of code for this:
hex_data := Std.Str.ToHexPairs((>DATA<)ds.contents);
d1:=Std.Str....
1
vote
1
answer
112
views
Linking multiple C++ files in ECL
I have C++ code spread across multiple C++ and header files and wanted to know how to link them in ECL. Should I install them as libraries (.cpp and header files) into the local cluster I am running ...
0
votes
1
answer
43
views
Send dataset to a postgress database from HPPSystem - ECL
I need to send an uploaded dataset from a csv inside the ECL Watch of HPCCSystem.
I don't know where the ODBC driver is installed on the server, I downloaded a VM template that contains the HPCCSystem ...
1
vote
2
answers
129
views
How to send POST request in HPCC ECL?
Can someone help me understand how to make a REST POST request in HPCC? I've read through the documentation but I couldn't find any examples. Any help would be appreciated.
0
votes
1
answer
41
views
Remote call to Roxie server from Thor
Assuming that we have the end-point to a Roxie server of interest, I was wondering if it is possible to make a remote call to it from a bwr script on Thor, and get the number of nodes that Roxie ...
0
votes
1
answer
68
views
Error when trying to hit an API using HTTPCALL in ECL
I try to hit the API by using HTTPCALL in ECL and it is giving following error.
I have tried hitting it using Postman, its working fine in it.
Here is the URL for API - 'https://api.waqi.info/feed/...
1
vote
1
answer
98
views
Passing a dataset as an argument for C++ embedding in ECL
I want to pass a dataset as an argument to a C++ embedded function. This code snippet seems to be working when we hardcode the dataset. The dataset is a BLOB type sprayed blk.dat file having ...
1
vote
1
answer
51
views
Issue while setting up local HPCC cluster using WSL
When I start the cluster ( using the command etc/init.d/hpcc-init start). Starting mydali shows a TIMEOUT message. Something like this:
How do I solve this problem?
Thank You.
1
vote
2
answers
93
views
'Cannot read properties of undefined (reading 'eclccPath')' in HPCC Systems VS Code extension
How to resolve this error 'Cannot read properties of undefined (reading 'eclccPath')' in HPCC Systems VS Code extension.
I have installed ecl client tools. While installing I tried with both options '...
1
vote
0
answers
37
views
Updated support schedule for HPCC releases 8.6, 8.8 and 8.10
The schedule defined for the support of the HPCC releases, found at "https://wiki.hpccsystems.com/display/hpcc/Latest+release+and+documentation", seems outdated – 8.8 & 8.10 not present, ...
1
vote
1
answer
183
views
How can I split CSV files into multiple CSV files in runtime based on file size?
I have been trying to split CSV files into 1GB parts and the file size of the product varies such that it won't be helpful to split in a fixed number.
I have managed to find a way to estimate the ...
0
votes
1
answer
65
views
I wanted to return the obtained function output into a variable that I added into the RECORD using TRANSFORM. What does this error in image here mean?
distcomp (REAL4 lat1, REAL4 lon1,REAL4 lat2, REAL4 lon2, REAL4 dist) := FUNCTION
REAL4 lo1 := lon1/57.29577951;
REAL4 lo2 := lon2/57.29577951;
REAL4 la1 := lat1/57.29577951;
REAL4 la2 := lat2/57....
1
vote
4
answers
630
views
Is there a way to pass multiple argument as a single string in bash?
I'm running a program that takes variable numbers of arguments with the same flag. For example
myprogram -args 'var1' 'var2' 'var3' 'var4'
myprogram -args 'var5' 'var6'
I have to launch this program ...
1
vote
1
answer
117
views
HPCC - ECL: Module "name" does not EXPORT an attribute main()
I have been explaining to others what "Module name does not EXPORT an attribute main()" error means and how to fix it.
Every time I answer this question I say it's because MODULE is a ...
0
votes
1
answer
97
views
HPCCSystems - ECL, Does NOTHOR runs on hthor or one node of thor?
Using ECL/HPCC, I'm working with a team on an issue. They're trying to run actions within child queries. During trying different solution a question came up:
Does ECL code executing with a NOTHOR ...
8
votes
2
answers
7k
views
How to use embedded Webassembly in Vite?
I would like to use this great package: https://github.com/hpcc-systems/hpcc-js-wasm
It bundles a Webassembly (graphizlib.wasm) with the Javascript functions to use. I added it as a dependency in ...
0
votes
1
answer
96
views
simtk.openmm.app.internal.charmm.exceptions.CharmmFileError: [Errno 2] No such file or directory: '../toppar/top_all36_prot.rtf'
I am trying to run an OpenMM simulation on HPCC, but I get the error that 'top_all36_prot.rtf' file, which is an rtf file, is not into my directory - while I have copied that file into my directory. ...
0
votes
1
answer
61
views
HPCC Systems - Can't change default port for Dali
I'm installing HPCC Systems in ubuntu 20.04
When starting the application it fails because Dali, tries to use port 7070 which is alredy in use by another application.
I changed the port in the ...
1
vote
0
answers
201
views
Errors running Linpack
I'm trying to run HPL Linpack on my personal laptop. I'm using CentOS 8 on a VM.
Allocated cores : 6
Memory : 12.5 gb
Nodes : 1
When I run with smaller values of N, its running fine, but when I try ...
1
vote
2
answers
245
views
Print "Jobs Finished" only when all the bjobs are completed
I have a python script that submits multiple jobs using bjobs. Below is the code snippet
for jobs in job_list:
i=0
os.system("bsub -J JOB_{} jobs".format(str(i))
i+=1
I want to ...
0
votes
1
answer
77
views
Undeclared variable error when compiling HPC Challenge Benchmark
I want to compile and run HPC Challenge benchmark on my server.
When I want to make hpcc-1.5.0 with mpicc compiler I get this error:
../../../../STREAM/stream.c:178:26: error: expected ‘=’, ‘,’, ‘;’, ‘...
1
vote
0
answers
136
views
Python loop not running in SLURM
I have a python code that uses a function (let's call it integrator_MPI)that is parallelized according to MPI. I am exucuting this code in a HPCC by submitting the job whose relevant lines are :
#!/...
1
vote
1
answer
732
views
Jenkins safely aborting child process
I have a Jenkins job that starts a python program on a cluster via SSH from a windows slave.
The program has a cleanup step that is triggered via a python finally statement for any kind of exception.
...
1
vote
2
answers
111
views
Running a matrix algorithm on how many cores?
I am running a program called dnadist from PHYLIP (http://evolution.genetics.washington.edu/phylip/doc/dnadist.html). This creates a dna distance matrix from the number of sequences you input.
...
1
vote
1
answer
553
views
How to convert SET to DATASET in HPCC / ECL?
I have this SET in ECL
EXPORT TableNames := [
'tbl1',
'tbl2',
'tbl3',
'tbl4'
];
...
0
votes
1
answer
65
views
How can my HPCC docker container be accessed from another laptop?
I am running a HPCC Cluster in docker-compose in my laptop using docker-toolbox as I have Windows7. I can successfully connect to container (http://192.168.99.100:8010)
My docker-compose.yml file is:...
1
vote
1
answer
115
views
Switching from local to thor cluster in ECL IDE
I'm trying to switch to the thor cluster in the ECL IDE, but thor isn't available in the drop down menu. Is it a mistake I made during installation? Does anyone anything about this?
0
votes
1
answer
43
views
Finding whether two geo-coordinates x and y for 2 different cars lie on the same road
my data consists of vehicle_ID, x and y coordinates (for location), the speed of the vehicle, time at which they travel. And we would like to know which cars are taking the same roads?
this is a ...
-2
votes
3
answers
261
views
Loading PetaBytes of data at scale
I need to load petabytes of text data into a storage (RAM/SSD) within a second.
Below are some of the question to solve the above problem.
1) Is it practically/theoretically possible to load ...
2
votes
1
answer
138
views
What is the ECL equivalent of NaN or IsNumeric?
I want to check if some value (i.e: 42 or 41.0005) is a number in ECL (HPCC).
I have in mind a function using Regular Expressions like this:
EXPORT IsNumeric(STRING UniqueID) := FUNCTION
...
4
votes
2
answers
299
views
Controlling ECL loop macro using results of a query
We are trying to write a loop macro with a break function. Basically we would like to repeat a certain action until X (derived from a Dataset) is true. The compiler doesn't seem to like this approach ...
1
vote
1
answer
127
views
HPCC-ECL Logical Operators - Why is OR not short-circuiting?
The documentation indicates that the OR logical operator should short-circuit:
If the probability of occurrence is known,
you should order them from the most likely to occur to the least likely ...
-1
votes
1
answer
269
views
I want to convert this code from SAS to ECL(HPCC)
Can Somebody please explain me the meaning &, && , % in this SAS code.
I want to convert this code to ECL(HPCC). Thanks
PROC SQL;
create table &RD (compress=binary) as
select a.*,b....
0
votes
1
answer
476
views
HPCC/HDFS Connector
Does anyone know about HPCC/HDFS connector.we are using both HPCC and HADOOP.There is one utility(HPCC/HDFS connector) developed by HPCC which allows HPCC cluster to acess HDFS data
i have installed ...
1
vote
0
answers
229
views
Data corruption for large data manipulation
I have some very weird data corruption trouble recently.
Basically what I do is:
transfer some large data (50files, each around 8GB) from one server to hpcc(high performance computing) using "scp"
...
1
vote
2
answers
7k
views
Data distortion after using "scp" for transfer
Recently I transferred a set of data from one server to hpcc(high-performance computing)
Command is like:
scp /folder1/*.fastq.gz xxx@hpcc:/home/
scp /folder2/*.fastq.gz xxx@hpcc:/home/
scp /folder3/...