2

I am trying to install the open source hadoop or building the HDP from source to be installed by ambari. I can see that it is possible to build the java packages for each component with the documentation available in apache repos, but how can i use those to build rpm/deb packages that are provided by hortonworks for HDP distribution to be installed by ambari.

1 Answer 1

1

@ShivamKhandelwal Building Ambari From Source is a challenge but one that can be accomplished with some persistence. In this post I have disclosed the commands I used recently to build Ambari 2.7.5 in centos:

Ambari 2.7.5 installation failure on CentOS 7

"Building HDP From Source" is very big task as it requires building each component separately, creating your own public/private repo which contains all the component repos or rpms for each operating system flavor. This is a monumental task which was previously managed by many employees and component contributors at Hortonworks.

When you install Ambari from HDP, it comes out of the box with their repos including their HDP stack (HDFS,Yarn,MR,Hive, etc). When you install Ambari From Source, there is no stack. The only solution is to Build Your Own Stack which is something I am expert at doing.

I am currently building a DDP stack as an example to share with the public. I started this project by reverse engineering a HDF Management Pack which includes stack structure (files/folders) to role out NiFi, Kafka, Zookeeper, and more. I have customized it to be my own stack with my own services and components (NiFi, Hue, Elasticsearch, etc).

My goal with DDP is to eventually make my own repos for the Components and Services I want, with the versions I want to install in my cluster. Next I will copy some HDP Components like HDFS,YARN,HIVE from the HDP stack directly into my DDP stack using the last free public HDP Stack (HDP 3.1.5).

3
  • 1
    Thank you for the answer, my question was related to the later part of your answer. I was able to compile and deploy the ambari myself and wanted some solution to do the compilation of the hadoop binaries. I am completely aligned with the steps that you mentioned to generate the rpm packages. I think i will have to do that myself, since this is something no one has tried yet. Jun 10, 2020 at 8:16
  • Yes, I am deep on this path myself. I have a pretty stable stack now based originally off HDF and now putting HDP pieces in. I also have some third party tools (hue and elasticsearch). I got HDFS working, but still working on hadoop (yarn,mapreduce), hive, and some others...Can you accept the solution? Check out: makeopensourcegreatagain.com/rpms and: github.com/steven-dfheinz/dfhz_ddp_mpack Jun 10, 2020 at 10:31
  • 1
    Hi @steven-matison, really appreciate your works! Cloudera closes all access to all version of CDP and HDP since Feb 2020, do you have another HDP version repositories?
    – Nur Faizin
    Feb 10, 2021 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.