Spark Application with Kafka Consumer and Hbase
1. Maven Project pom.xml. Import all required dependencies, and package all dependencies, resources into a single jar. This is the best option else you will […]
Spark
Kafka, Spark, Hbase, Hadoop, Java
Step 1: Install and Setup Hadoop Cluster Follow the link here Step 2: Install and Setup Hbase Follow the link here Step 3: Install and […]
Log4j2 and Spark 2.3.1
LOG4J has been very popular logging library in Java world for years. LOG4J2 is even better. In Aug 2015 Log4j development team officially announced end of life for […]
Install Spark 2.3.x on YARN with Hadoop 3.x
This is the preferred method for Spark Installation, as Yarn is started as ResourceManager in hadoop, this reduces the transaction time required when installing Spark […]