Sqoop Eval Command With Example This post describes how to use the Sqoop ‘eval’ tool. It allows users to execute user-defined queries against respective database servers and preview the result in the console. So, the user can expect the resultant table data to import. Using eval, we can evaluate any… Continue Reading Sqoop Eval Command With Example

Sqoop Codegen Command With Example This post describes the importance of ‘codegen’ tool. From the viewpoint of object-oriented application, every database table has one DAO class that contains ‘getter’ and ‘setter’ methods It generates DAO class in Java, based on the Table Schema structure. The Java definition is instantiated as… Continue Reading Sqoop Codegen Command With Example

Apache Sqoop Job Command With Example This post describes how to create and maintain the Sqoop jobs. Sqoop job creates and saves the import and export commands. It specifies parameters to identify and recall the saved job. This re-calling or re-executing is used in the incremental import, which can import… Continue Reading Sqoop Job Command With Example

Sqoop Export to MySQL database from HDFS Export data back from the HDFS to the RDBMS database. The target table must exist in the target database. The files which are given as input to the Sqoop contain records, which are called rows in table. Those are read and parsed into… Continue Reading Sqoop Export to MySQL database from HDFS

Sqoop Import From MySQL Database data to HDFS The ‘Import tool’ imports individual tables from RDBMS to HDFS. Each row in a table is treated as a record in HDFS. All records are stored as text data in the text files or as binary data in Avro and Sequence files.… Continue Reading Sqoop Import From MySQL Database data to HDFS

Pre Requirements 1) A machine with Ubuntu 14.04 LTS operating system. 2) Apache Hadoop pre installed 3) MySQL Database pre installed 4) Apache Sqoop 1.4.6 Software (Download Here) NOTE Apache Sqoop comes with Hadoop compatible version. Check with your Hadoop version and download sqoop. Sqoop 1.4.6 Installation on Ubuntu Installation… Continue Reading Sqoop Installation

Apache Flink Web UI Usage Step 1 – Open a terminal and change the directory to /usr/local/flink/bin

Step 2 – Start flink

Step 3 – Open a browser and type the following URL

Flink Word Count Java Example The following code shows the WordCount implementation from the Quickstart which processes some text lines with two operators (FlatMap and Reduce), prints the resulting words and counts to std-out. Step 1 – Add JARs (Libraries) Add the following jars to your java project build path.… Continue Reading Flink Word Count Java Example

Apache Flume SeqGen Agent Configuration Now we will see, how to fetch data from Sequence generator. Step 1 – Change the directory to /usr/local/hadoop/sbin

Step 2 – Start all hadoop daemons.

Step 3 – The JPS (Java Virtual Machine Process Status Tool) tool is limited to reporting information… Continue Reading Flume SeqGen Agent Configuration