Questions tagged [hpcc]

High Performance Computing Cluster (HPCC)

Filter by
Sorted by
Tagged with
-1 votes
1 answer
43 views

Is it always necessary to define a record structure while using ECL?

Every time I make use of ECL, I have to define a record structure for any file in use (CSV, XML, JSON). However, is there a way of making use of these files on ECL without defining the record ...
nix's user avatar
  • 3
0 votes
1 answer
59 views

Can we make use of AWS S3 connectivity with HPCC Cluster without administrative access to the cluster?

So I wanted to look on how one can develop AWS S3 connectivity with a HPCC Cluster. Tools like s3cmd and aws cli work well if I am accessing files from my S3 bucket onto my localhost cluster by ...
decoded051's user avatar
0 votes
1 answer
30 views

MongoDB plugin for ECL language

Trying to load all json docs from a mongodb server in a dataset using the mongodb plugin for the ECL language. IMPORT MongoDB; server := 'cluster0.smthing.mongodb.net'; user := 'uname'; pwd := '...
Mukund's user avatar
  • 1
0 votes
2 answers
64 views

Turn json string to data set

I'm able to get a json object back from Python, but it's a string. What's the best way to turn it into a data set in ECL. I feel I would have to save it to a file, then read it as json, but I'm ...
George Triarhos's user avatar
0 votes
1 answer
61 views

Is there a way to return a 3D list from embedded Python back to ECL?

I have a 2D array in Python which I want to return back to ELC, for further usage. The recordset is of the following format layout:=RECORD INTEGER RED; INTEGER GREEN; INTEGER BLUE; ...
Shashank B's user avatar
0 votes
1 answer
43 views

Is there a way visualise BLOB data in ECL?

I have an image in Python embedded in ECL(through the EMBED() function). The image is in the form of a numpy array. How do I return it back to ECL and are there any methods to visualize the image? I ...
Shashank B's user avatar
1 vote
2 answers
52 views

How to send POST request using sendgrip api in HPCC ECL?

Could someone kindly assist me in making a POST request call using the SendGrid API in HPCC ECL? I have diligently referred to the documentation and examples provided in the forum, but I have not come ...
Yashi Mishra's user avatar
0 votes
0 answers
8 views

How to HTTPCALL in HPCCECL having authorization , JSON output GET Method as it gives empty string

Can you please suggest how to optimize below code to fetch roxie queries related values properly IMPORT Std; WUListQueriesResponse :- Is same as we get in layout . QueryList := HTTPCALL('https://...
harsh desai's user avatar
1 vote
1 answer
46 views

In my HPCC Cluster, what does this error in a workunit "Error receiving actinit data for graph: xx" really means?

Note: I am running code in a cluster with 16 slaves, HPCC version 6.4.40 I am running some ECL code that returns this error: System error: 0: Graph graph2[14], SLAVE #1 [10.313.316.31:20100]: Error ...
Oscar Foley's user avatar
  • 6,867
0 votes
1 answer
40 views

How to install Python libraries on a university cluster using Ubuntu terminal?

I have been working on a project that requires the installation of certain Python libraries onto a cluster. Initially, I was able to install these libraries on my local cluster by following the steps ...
Dumbledore's user avatar
0 votes
0 answers
24 views

How to write an ECL launch configuration for a university cluster?

I have been working with a local cluster for some time, but now I need to expand my work and use a university cluster. I have written a JSON file for the ECL launch configuration, but I am unsure if ...
Dumbledore's user avatar
0 votes
2 answers
24 views

Delay Transaction on Roxie

Scenario: a client wants to test a scenario where they run a transaction against a Roxie service, and they would like the transaction to take at least 50 seconds before it returns anything. What is a ...
flyer's user avatar
  • 13
0 votes
0 answers
19 views

0: request to https://play.hpccsystems.com:18010/WsSMC/Activity.json failed, reason: unable to get local issuer certificate

Every time I run an ECL file in VS Code, I am getting this error: 0: request to https://play.hpccsystems.com:18010/WsSMC/Activity.json failed, reason: unable to get local issuer certificate. However, ...
Saksham Sahgal's user avatar
1 vote
1 answer
78 views

ModuleNotFoundError: No module named 'matplotlib' when using plt.imread() in embedded Python code

I am trying to read an image using plt.imread() in embedded Python code within my DATA function. The code is as follows: IMPORT PYTHON3 AS PYTHON; IMPORT Std; IMPORT Image; myRec := RECORD STRING ...
Dumbledore's user avatar
1 vote
1 answer
55 views

How to convert hex data to byte data in ECL?

I am working on a code in ECL and trying to convert hex data into byte data. I am using the following line of code for this: hex_data := Std.Str.ToHexPairs((>DATA<)ds.contents); d1:=Std.Str....
Dumbledore's user avatar
1 vote
1 answer
112 views

Linking multiple C++ files in ECL

I have C++ code spread across multiple C++ and header files and wanted to know how to link them in ECL. Should I install them as libraries (.cpp and header files) into the local cluster I am running ...
ANAGHA ANAND's user avatar
0 votes
1 answer
43 views

Send dataset to a postgress database from HPPSystem - ECL

I need to send an uploaded dataset from a csv inside the ECL Watch of HPCCSystem. I don't know where the ODBC driver is installed on the server, I downloaded a VM template that contains the HPCCSystem ...
Andrés Nuin's user avatar
1 vote
2 answers
129 views

How to send POST request in HPCC ECL?

Can someone help me understand how to make a REST POST request in HPCC? I've read through the documentation but I couldn't find any examples. Any help would be appreciated.
Vipin Katara's user avatar
0 votes
1 answer
41 views

Remote call to Roxie server from Thor

Assuming that we have the end-point to a Roxie server of interest, I was wondering if it is possible to make a remote call to it from a bwr script on Thor, and get the number of nodes that Roxie ...
flyer's user avatar
  • 13
0 votes
1 answer
68 views

Error when trying to hit an API using HTTPCALL in ECL

I try to hit the API by using HTTPCALL in ECL and it is giving following error. I have tried hitting it using Postman, its working fine in it. Here is the URL for API - 'https://api.waqi.info/feed/...
Vipin Katara's user avatar
1 vote
1 answer
98 views

Passing a dataset as an argument for C++ embedding in ECL

I want to pass a dataset as an argument to a C++ embedded function. This code snippet seems to be working when we hardcode the dataset. The dataset is a BLOB type sprayed blk.dat file having ...
ANAGHA ANAND's user avatar
1 vote
1 answer
51 views

Issue while setting up local HPCC cluster using WSL

When I start the cluster ( using the command etc/init.d/hpcc-init start). Starting mydali shows a TIMEOUT message. Something like this: How do I solve this problem? Thank You.
chandan kasamsetty's user avatar
1 vote
2 answers
93 views

'Cannot read properties of undefined (reading 'eclccPath')' in HPCC Systems VS Code extension

How to resolve this error 'Cannot read properties of undefined (reading 'eclccPath')' in HPCC Systems VS Code extension. I have installed ecl client tools. While installing I tried with both options '...
Jyothi Shetty's user avatar
1 vote
0 answers
37 views

Updated support schedule for HPCC releases 8.6, 8.8 and 8.10

The schedule defined for the support of the HPCC releases, found at "https://wiki.hpccsystems.com/display/hpcc/Latest+release+and+documentation", seems outdated – 8.8 & 8.10 not present, ...
Sergio Fernandes's user avatar
1 vote
1 answer
183 views

How can I split CSV files into multiple CSV files in runtime based on file size?

I have been trying to split CSV files into 1GB parts and the file size of the product varies such that it won't be helpful to split in a fixed number. I have managed to find a way to estimate the ...
Emerson Grisi's user avatar
0 votes
1 answer
65 views

I wanted to return the obtained function output into a variable that I added into the RECORD using TRANSFORM. What does this error in image here mean?

distcomp (REAL4 lat1, REAL4 lon1,REAL4 lat2, REAL4 lon2, REAL4 dist) := FUNCTION REAL4 lo1 := lon1/57.29577951; REAL4 lo2 := lon2/57.29577951; REAL4 la1 := lat1/57.29577951; REAL4 la2 := lat2/57....
Sai Lahari's user avatar
1 vote
4 answers
630 views

Is there a way to pass multiple argument as a single string in bash?

I'm running a program that takes variable numbers of arguments with the same flag. For example myprogram -args 'var1' 'var2' 'var3' 'var4' myprogram -args 'var5' 'var6' I have to launch this program ...
je2018's user avatar
  • 125
1 vote
1 answer
117 views

HPCC - ECL: Module "name" does not EXPORT an attribute main()

I have been explaining to others what "Module name does not EXPORT an attribute main()" error means and how to fix it. Every time I answer this question I say it's because MODULE is a ...
Alchemist's user avatar
0 votes
1 answer
97 views

HPCCSystems - ECL, Does NOTHOR runs on hthor or one node of thor?

Using ECL/HPCC, I'm working with a team on an issue. They're trying to run actions within child queries. During trying different solution a question came up: Does ECL code executing with a NOTHOR ...
Alchemist's user avatar
8 votes
2 answers
7k views

How to use embedded Webassembly in Vite?

I would like to use this great package: https://github.com/hpcc-systems/hpcc-js-wasm It bundles a Webassembly (graphizlib.wasm) with the Javascript functions to use. I added it as a dependency in ...
Wizard of Kneup's user avatar
0 votes
1 answer
96 views

simtk.openmm.app.internal.charmm.exceptions.CharmmFileError: [Errno 2] No such file or directory: '../toppar/top_all36_prot.rtf'

I am trying to run an OpenMM simulation on HPCC, but I get the error that 'top_all36_prot.rtf' file, which is an rtf file, is not into my directory - while I have copied that file into my directory. ...
amirMD's user avatar
  • 19
0 votes
1 answer
61 views

HPCC Systems - Can't change default port for Dali

I'm installing HPCC Systems in ubuntu 20.04 When starting the application it fails because Dali, tries to use port 7070 which is alredy in use by another application. I changed the port in the ...
Ric's user avatar
  • 65
1 vote
0 answers
201 views

Errors running Linpack

I'm trying to run HPL Linpack on my personal laptop. I'm using CentOS 8 on a VM. Allocated cores : 6 Memory : 12.5 gb Nodes : 1 When I run with smaller values of N, its running fine, but when I try ...
Dr. S K Singhal's user avatar
1 vote
2 answers
245 views

Print "Jobs Finished" only when all the bjobs are completed

I have a python script that submits multiple jobs using bjobs. Below is the code snippet for jobs in job_list: i=0 os.system("bsub -J JOB_{} jobs".format(str(i)) i+=1 I want to ...
Astro's user avatar
  • 11
0 votes
1 answer
77 views

Undeclared variable error when compiling HPC Challenge Benchmark

I want to compile and run HPC Challenge benchmark on my server. When I want to make hpcc-1.5.0 with mpicc compiler I get this error: ../../../../STREAM/stream.c:178:26: error: expected ‘=’, ‘,’, ‘;’, ‘...
Arman's user avatar
  • 1,019
1 vote
0 answers
136 views

Python loop not running in SLURM

I have a python code that uses a function (let's call it integrator_MPI)that is parallelized according to MPI. I am exucuting this code in a HPCC by submitting the job whose relevant lines are : #!/...
delta_omega's user avatar
1 vote
1 answer
732 views

Jenkins safely aborting child process

I have a Jenkins job that starts a python program on a cluster via SSH from a windows slave. The program has a cleanup step that is triggered via a python finally statement for any kind of exception. ...
fberg's user avatar
  • 21
1 vote
2 answers
111 views

Running a matrix algorithm on how many cores?

I am running a program called dnadist from PHYLIP (http://evolution.genetics.washington.edu/phylip/doc/dnadist.html). This creates a dna distance matrix from the number of sequences you input. ...
Gina's user avatar
  • 71
1 vote
1 answer
553 views

How to convert SET to DATASET in HPCC / ECL?

I have this SET in ECL EXPORT TableNames := [ 'tbl1', 'tbl2', 'tbl3', 'tbl4' ]; ...
Oscar Foley's user avatar
  • 6,867
0 votes
1 answer
65 views

How can my HPCC docker container be accessed from another laptop?

I am running a HPCC Cluster in docker-compose in my laptop using docker-toolbox as I have Windows7. I can successfully connect to container (http://192.168.99.100:8010) My docker-compose.yml file is:...
Oscar Foley's user avatar
  • 6,867
1 vote
1 answer
115 views

Switching from local to thor cluster in ECL IDE

I'm trying to switch to the thor cluster in the ECL IDE, but thor isn't available in the drop down menu. Is it a mistake I made during installation? Does anyone anything about this?
Tito's user avatar
  • 102
0 votes
1 answer
43 views

Finding whether two geo-coordinates x and y for 2 different cars lie on the same road

my data consists of vehicle_ID, x and y coordinates (for location), the speed of the vehicle, time at which they travel. And we would like to know which cars are taking the same roads? this is a ...
red's user avatar
  • 23
-2 votes
3 answers
261 views

Loading PetaBytes of data at scale

I need to load petabytes of text data into a storage (RAM/SSD) within a second. Below are some of the question to solve the above problem. 1) Is it practically/theoretically possible to load ...
srinivasan Hariharan's user avatar
2 votes
1 answer
138 views

What is the ECL equivalent of NaN or IsNumeric?

I want to check if some value (i.e: 42 or 41.0005) is a number in ECL (HPCC). I have in mind a function using Regular Expressions like this: EXPORT IsNumeric(STRING UniqueID) := FUNCTION ...
Oscar Foley's user avatar
  • 6,867
4 votes
2 answers
299 views

Controlling ECL loop macro using results of a query

We are trying to write a loop macro with a break function. Basically we would like to repeat a certain action until X (derived from a Dataset) is true. The compiler doesn't seem to like this approach ...
DataMacGyver's user avatar
1 vote
1 answer
127 views

HPCC-ECL Logical Operators - Why is OR not short-circuiting?

The documentation indicates that the OR logical operator should short-circuit: If the probability of occurrence is known, you should order them from the most likely to occur to the least likely ...
David Sette's user avatar
-1 votes
1 answer
269 views

I want to convert this code from SAS to ECL(HPCC)

Can Somebody please explain me the meaning &, && , % in this SAS code. I want to convert this code to ECL(HPCC). Thanks PROC SQL; create table &RD (compress=binary) as select a.*,b....
Siddharth Kumar's user avatar
0 votes
1 answer
476 views

HPCC/HDFS Connector

Does anyone know about HPCC/HDFS connector.we are using both HPCC and HADOOP.There is one utility(HPCC/HDFS connector) developed by HPCC which allows HPCC cluster to acess HDFS data i have installed ...
Chhaya Vishwakarma's user avatar
1 vote
0 answers
229 views

Data corruption for large data manipulation

I have some very weird data corruption trouble recently. Basically what I do is: transfer some large data (50files, each around 8GB) from one server to hpcc(high performance computing) using "scp" ...
LookIntoEast's user avatar
  • 8,228
1 vote
2 answers
7k views

Data distortion after using "scp" for transfer

Recently I transferred a set of data from one server to hpcc(high-performance computing) Command is like: scp /folder1/*.fastq.gz xxx@hpcc:/home/ scp /folder2/*.fastq.gz xxx@hpcc:/home/ scp /folder3/...
LookIntoEast's user avatar
  • 8,228