Questions tagged [hadoop-yarn]

YARN (Yet Another Resource Negotiator) is a key component of second generation Apache Hadoop infrastructure. DO NOT USE THIS for the JavaScript/Node.js Yarn package manager (use [yarnpkg] instead)! Originally described by Apache as a redesigned resource manager, YARN is now characterized as a large-scale, distributed operating system for big data applications including next generation MapReduce (MR2).

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

How to run yarn develop for strapi when there is problem with memory?

So I have an issue running yarn develop in my VS Code. The issue is this: {code: 'ER_OUT_OF_SORTMEMORY', errno: 1038, sqlMessage: 'Out of sort memory, consider increasing server sort buffer size', ...
Gracella Q Sumarlin's user avatar
0 votes
0 answers
35 views

setting up project with Yarn 4 pnp, webpack and typescript

I am trying to set up my project with Yarn and webpack and typescript. You can see the code below. The problem I have is that when I import another .ts file, it gives me an error and can't resolve the ...
Saeed Am's user avatar
0 votes
0 answers
22 views

Invalid option name ("--modules-folder=/layers/google.nodejs.yarn/yarn_modules/node_modules") while deploying nextjs app to google app engine

I followed those steps: distDir: 'build' in next.config.js modified package.json { "name": "SHOP", "version": "1.0.8", "description": "...
Rina's user avatar
  • 9
0 votes
0 answers
102 views

yarn install Dockerfile gives message: failed to solve: process "/bin/sh -c yarn" did not complete successfully: exit code: 1

When try to execute my frontend project (react-vite) that works in my local machine , with yarn in Docker , Its a really long confusing message I try running yarn install with others args but nothing ...
Pedro Calado Moura's user avatar
0 votes
0 answers
55 views

Failed to Load SWC Binary Error in Next.js Development Server

I encountered an issue while running my Next.js project using the yarn run dev command. The error message is as follows in the image Error Details: Next.js Version: 14.0.3 Local Server: http://...
Awais Ahmad's user avatar
1 vote
0 answers
16 views

Expo monorepo with react-navigation

I have tried to create a monorepo containing two Expo apps using React Navigation with different SDK levels and I forced it to use nohoist for all "workspaces": { "packages": ...
Archimedes Trajano's user avatar
0 votes
0 answers
64 views

Spark-submit yarn-client mode hangs even though spark task completed (pyspark 3.4.1)

Recently set up a dockerized env along with CDP for submitting yarn-client mode spark jobs on a kereberized hadoop cluster and seeing inconsistent behavior with application lifecycle. Scenario: ...
StrangerThinks's user avatar
0 votes
0 answers
63 views

YARN + yarn resource manager stores a ton of znodes related to running/old applications in zookeeper

We have HDP production Hadoop cluster , include 2 active / standby resource managers services Some details from our cluster HDP version - 2.6.5 OS Linux machines version - 7.9 Number of node ...
jessica's user avatar
  • 2,438
1 vote
0 answers
43 views

Low CPU Usage and Memory Errors in PySpark Job on Dataproc Cluster

Question I'm running a PySpark job on a Dataproc cluster, and I'm facing two issues: the CPU usage is consistently low (5%-10%), and when I attempt to parse the entire dataset at once, I encounter ...
Omar Cotugno's user avatar
0 votes
0 answers
38 views

Error when starting Yarn Nodemanager: [ERROR util.SysInfoWindows: ExitCodeException exitCode=1]

I am trying to setup Hadoop on my laptop (just to learn it and do some basics, and if I have the time a couple advanced things). I am on Windows 10, using Intellij, I also installed Hadoop 3.2.1 ...
Brahim Khalil Abid's user avatar
0 votes
2 answers
267 views

How to solve "Error while loading rule 'react/display-name': `[[GeneratorState]]` is not present on `O`" error in eslint

In my case I am using eslint on nextjs with eslint-config-next. I use bun as a package manager but I found instances of this issues on npm and yarn as well. I didn't add any new package but after ...
Alain1405's user avatar
  • 2,489
0 votes
0 answers
39 views

issue running hadoop mapreduce wordcount

I am running Hadoop version 3.2.4 in windows and want to perform a WordCount operation on the file located in hadoop/share/hadoop/mapreduce/share/mapreduce-examples-3.2.4.jar. However, it failed, and ...
ryan's user avatar
  • 21
0 votes
0 answers
15 views

Installed yarn globally, but nothing works

I completely messed up my ability to do any coding on my computer a few months ago and I was so frustrated that I walked away for awhile and now I have no recollection of how I even got myself into ...
Jacqueline's user avatar
0 votes
1 answer
43 views

Zero datanodes running in Hadoop 2.6.5 AWS

I am trying to input/upload the file to the hdfs hadoop but it's giving me this error When I run this command I get this : - hdfs dfs -put A.txt /input/ 23/11/08 22:19:25 WARN hdfs.DFSClient: ...
Rahul's user avatar
  • 11
0 votes
1 answer
34 views

Trouble Show output using hadoop word count

I'm new to using Hadoop, and I want to execute Hadoop syntax using WordCount to count words. However, why is it that when I try to display the output, it doesn't appear? I would appreciate an ...
ryan's user avatar
  • 21
1 vote
1 answer
85 views

RAILS 6 Build Failed

I have this issue after building in heroku after fixing the babel issue, I still get this error. Done in 13.56s. autoprefixer: /tmp/build_0623a275/app/assets/stylesheets/application.scss:...
Jaymarc San Luis's user avatar
-1 votes
1 answer
35 views

Is there any way to override some released functions in Flink’s /lib folder?

My flink applications is deployed by yarn and I include all the necessary flink dependencies in Flink’s /lib folder. Now I want to add some unique features to the jar 'flink-table-planner', but I can ...
John Lee's user avatar
0 votes
0 answers
11 views

YARN Application Stuck in ACCEPTED State with Cluster Resources Empty

I'm experiencing an issue with my Hadoop YARN application where it remains in the ACCEPTED state and does not transition to RUNNING. After submitting my application, it seems to be added to the ...
zhehelima's user avatar
0 votes
0 answers
36 views

Hadoop Yarn Default queue ‘s Resources are all 0(both vCores and Memory) all the time so application submitted can't turn from ACCEPTED to RUNNING

I have three servers, each with 2GB RAM and 2 CPU cores. I had previously configured a Hadoop cluster (version 2.8.0) and could run MapReduce jobs using Java. Now, I'm in the process of setting up ...
Zifeng Tan's user avatar
0 votes
1 answer
151 views

Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader

I get the following error when trying to run yarn dev to launch a discord bot. I am using typescript. I have already tried using nvm to update the node js version. I even tried using lower versions of ...
Hoko A's user avatar
  • 57
-1 votes
0 answers
49 views

Require in ESM not supported

I have a problem. I integrated the storybook in our project with Next JS, Docker, yarn v1.22.19. I know about it, it is a very old story book, but I don't think that's the problem. After that I used ...
Anar Guliev's user avatar
0 votes
0 answers
21 views

error Command failed with signal "SIGSEGV" in Nuxtjs

enter image description here Error appear when in run: yarn dev / yarn build. I run it successfully local, but when I uploaded it to run on the server, an error occurred. The error description appears ...
D. Sam's user avatar
  • 1
0 votes
0 answers
12 views

Min Share and Weight which has greater priority in case of getting resources

Let say we have 2 queues in fairscheduler.xml. First one with minShare 150 and weight 1 and the second one with minShare 1 and weight 10. Now let say we have 200 cores and equivalent memory. Which one ...
Soumya Bhattacharjee's user avatar
0 votes
0 answers
34 views

What versions of JAR files should I download for my hadoop and spark cluster? [duplicate]

I have a 3 node hadoop cluster on 3 Ubuntu EC2 instances, where I have Hadoop version 3.3.6, Apache Spark version 3.5.0 and hive version 3.1.3. And Im using postgres as my external metastore. Im able ...
user22824524's user avatar
0 votes
0 answers
23 views

Dependabot suggestion to edit yarn.lock results in Rails server refusing to work, saying to run `yarn install --check-files`

I have a Rails application, and Dependabot says: When I modify the yarn.lock file as per that suggestion, and try to run rails s, I get this: warning Integrity check: Lock files don't match error ...
Keith Bennett's user avatar
0 votes
0 answers
31 views

CPU core allocation in a DataProc cluster in GCP

I am just trying to understand how CPU cores are allocated/utilised in a DataProc cluster spined up in GCP. When we create a cluster in DataProc, there is a master node. Besides that we can configure ...
user3103957's user avatar
0 votes
0 answers
42 views

How to run package dependency with yarn in the console? Like npx tsc init for npm package.json with typescript dependency?

I know node_modules folder does not exist when using yarn and instead there is a .yarn/cache folder where the dependencies go. I have tried using yarn dlx tsc init and npx tsc init thinking they might ...
uzair's user avatar
  • 1
0 votes
1 answer
39 views

AWS EMR YARN does not allocate all requested executors

The situation: 1x Primary node: 4-cores 8GiB memory 2x core nodes: 16-cores -64GiB memory I tried to request 6 executors, each running 5 cores, so that my application would utilise 30 cores total. ...
William's user avatar
0 votes
1 answer
47 views

Why could I only request the index.htm when I had run a yarn serve at local. I could not request other file,

I'm learning about GitLab ci. There is my pipeline code. enter image description here There was the result. I have already ensured all the variables are right. enter image description here There is my ...
cola1917's user avatar
0 votes
0 answers
16 views

Internal Error: Cannot find module '.cache/node/corepack/yarn/1.22.19/bin/yarn.js'

I am getting below error in the Jenkins pipeline. Any idea on how to fix this? Internal Error: Cannot find module '.cache/node/corepack/yarn/1.22.19/bin/yarn.js' at Function.Module.\_resolveFilename (...
user14398734's user avatar
0 votes
0 answers
19 views

how to set .yarnrc config with nexus repository "401 Unauthorized"

My package.json file is configured as follows. The 'npm install' command works without any issues {     ...     "dependencies": {         "my-package": "https://nexus....
안원균's user avatar
1 vote
0 answers
25 views

Is there any way to set spark applicationId format

I am running spark submit in standalone mode, i am getting applicationId in the form of app-20231009145337-0067 (app-datetime-number) but same in spark submit YARN mode applicationId is in timestamp ...
cpawali's user avatar
  • 276
0 votes
0 answers
122 views

Yarn NodeManager unexpected SIGTERM

In an Hadoop 2.6.2 cluster with one ResourceManager (HDFS functional), on Debian 5.10.191-1 (2023-08-16) x86_64 GNU/Linux0 there are 3 DataNodes/NodeManagers. Scenario: When I launch a basic MapReduce ...
Laurent's user avatar
  • 13
0 votes
0 answers
10 views

how can i use job.addCacheArchive to load cache file from the output of another mapreduce?

Im trying to load cache file from the output of another mapreduce by java while writing mapreduce I tried FileOutputFormat.setOutputPath(job1, new Path(out, "out1")); ... Job job2 = Job....
KEIFTH YANG's user avatar
0 votes
0 answers
18 views

Aggregate Spark standalone executor logs

I am trying to test spark-submit standalone mode and running below example task spark-submit \ --class org.apache.spark.examples.SparkPi \ --master spark://MBP-49F32N-CSP.local:7077 \ --...
cpawali's user avatar
  • 276
0 votes
0 answers
99 views

no matter how puppeteer is installed, runtime gives: `errno: -13 "/.config/puppeteerrc"`

I've written a couple puppeteer scripts (small ones). In the past, for the first one I wrote, I run into this occasionally: error /var/tmp/vproj/pvid/node_modules/puppeteer: Command failed. Exit code: ...
Jim's user avatar
  • 1,518
0 votes
0 answers
19 views

Execute pyspark job in driver node

I have a 3 node cluster and working in client mode. Spark version is 3.0 Yarn as resource manager I want my spark application to run on application master JVM. That is in uber mode. How can i achieve ...
db2db2's user avatar
  • 1
0 votes
0 answers
13 views

Local IDE submits Spark tasks to remote Yarn, remote Yarn cannot access local Driver

My code configuration and netstat of the local Driver port are shown below. It is possible to submit tasks to Yarn, but I found that during the task execution process, I need to remotely connect to my ...
datagic's user avatar
  • 25
0 votes
0 answers
13 views

Symptom An error occurs when yarn is used to manage project package files

I had no problem with [email protected], but when I just updated yarn to the latest version ([email protected]), I ran into this problem and was hoping to ask for help. I searched many articles about yarn ...
sunny's user avatar
  • 11
0 votes
1 answer
58 views

yarn create app with a specific version of a app

Is there a way to use the following: yarn create strapi-app my-project --quickstart but use a specific version of strapi like version v4.13.6 instead of the latest which at the time of writing is v4....
user2371684's user avatar
  • 1,497
0 votes
0 answers
51 views

AdonisJS Error: require() of ES Module yarn

I have troubles width Adonis today and I hope you'll help me. So I have node 18.18.0, yarn 1.22.19. I upgraded all my packages with yarn upgrade -L. When I launch the following command inside my ...
SKamRa's user avatar
  • 1
0 votes
1 answer
144 views

Issues with npm start and Installing Packages: Warning about babel-preset-react-app Dependency and Audit Fix Freezes VSCode

Whenever I run npm start I run into this message: One of your dependencies, babel-preset-react-app, is importing the "@babel/plugin-proposal-private-property-in-object" package without ...
Lareine Han's user avatar
0 votes
1 answer
46 views

hadooop 3.3.6 resource manager not working

i'm using a ubuntu 22.04 pc with djk 17, java 11, hadoop 3.3.6 ```bash cd hadoop/hadoop-3.3.6/sbin ./start-all.sh ``` output: ```plaintext WARNING: Attempting to start all Apache Hadoop daemons as ...
Alwin Varghese's user avatar
0 votes
0 answers
15 views

How to decommission a yarn node gracefully?

I want to decommission a yarn node. But some tasks are still running on the node. I want the node dose not accept tasks anymore.And after the tasks finished, the node start to decommission. How should ...
klb's user avatar
  • 1
0 votes
1 answer
39 views

Is spark-3.0.0-bin-hadoop the same as spark-yarn? [closed]

I have been trying to follow a tutorial in order to install Spark on Windows : https://sparkbyexamples.com/spark/apache-spark-installation-on-windows/ because I would like to use the library ...
Code Klaudia's user avatar
0 votes
0 answers
14 views

Yarn upgrade vscode lsp version to above 3.17

i am running a c++ project that integrates with vscode and am facing issues currently implementing the type inlay hints for the language server i implement. I kinda boiled (aka: cannot find any reason ...
X39's user avatar
  • 810
0 votes
0 answers
26 views

How to optimize github action with mounted volume for yarn application

We use k8s github-runner for multiples yarn applications. Using the caching, the runner needs to download a tarball and extract which is quite long to process. We want to accelerate the caching using ...
Dimitri Kopriwa's user avatar
-1 votes
1 answer
45 views

Spark Cluster has heartbeat timeouts and Yarn Scheduler

I have checked other related posts on Stackoverflow but none seems to relate directly with mine, as the spark version they referenced are over 5 years ago and the spark version I am using is Spark 3.3....
Oyindamola Victor's user avatar
1 vote
0 answers
516 views

Could not auto-determine entry point from rollupOptions or html files and there are no explicit optimizeDeps

In my team project, at frontend part, we use vite, react, yarn, typescript. I'll build & deploy docker with frontend. I've successfully built a docker image with frontend, but I can't deploy it. ...
폰강아지's user avatar
0 votes
0 answers
27 views

Error while Configuring Yarn on a Slurm Cluster with Hadoop HDFS

I'm encountering an issue when configuring Yarn on a Slurm cluster alongside Hadoop HDFS. I have a Slurm batch script that launches nodes and dynamically edits core-site and yarn-site configuration ...
PKS's user avatar
  • 21

1
2 3 4 5
79