Apache Flume Moving Tomcat Logs to HDFS Now we will see, how you can move apache tomcat logs into the HDFS. Step 1 – Change the directory to /usr/local/hadoop/sbin

Step 2 – Start all hadoop daemons.

Step 3 – The JPS (Java Virtual Machine Process Status Tool) tool… Continue Reading Apache Flume Moving Tomcat Logs to HDFS

Flume Netcat Agent Configuration. Now we will see, how you can generate events and subsequently log them into the console. Step 1 – Change the directory to /usr/local/hadoop/sbin

Step 2 – Start all hadoop daemons.

Step 3 – The JPS (Java Virtual Machine Process Status Tool) tool is… Continue Reading Flume Netcat Agent Configuration

Apache Flume Collecting Twitter Data We will create an application and get the tweets from it using the experimental twitter source provided by Apache Flume. We will use the memory channel to buffer these tweets and HDFS sink to push these tweets into the HDFS. Step 1 – Create an… Continue Reading Flume Collecting Twitter Data

ZooKeeper Java API ZooKeeper has an official API binding for Java and C. The ZooKeeper community provides unofficial API for most of the languages (.NET, python, etc.). Using ZooKeeper API, an application can connect, interact, manipulate data, coordinate, and finally disconnect from a ZooKeeper ensemble. ZooKeeper API has a rich… Continue Reading ZooKeeper Java API

ZooKeeper Command Line Interface (CLI) usage ZooKeeper Command Line Interface (CLI) is used to interact with the ZooKeeper ensemble for development purpose. It is useful for debugging and working around with different options. To perform ZooKeeper CLI operations, first turn on your ZooKeeper server (“bin/zkServer.sh start”) and then, ZooKeeper client… Continue Reading ZooKeeper Command Line Interface (CLI) usage

Pre Requirements 1) A machine with Ubuntu 14.04 LTS operating system 2) Apache Hadoop 2.6.4 pre installed 3) Apache ZooKeeper 3.4.6 software (Download Here) ZooKeeper 3.4.6 Single Server setup on Ubuntu Installation Steps Step 1 – Update. Open a terminal (CTRL + ALT + T) and type the following sudo… Continue Reading ZooKeeper Single Server

Creating View and Indexes This post describes how to create, manage a view and indexes in HCatalog. Database views are created using the CREATE VIEW statement. Views can be created from a single table, multiple tables, or another view. To create a view, a user must have appropriate system privileges… Continue Reading Creating View and Indexes