Performance Analysis of Twitter Data Extraction Using Apache Flume

Abstract

There has been a tremendous rise in social media usage in recent years. This has led to huge volumes of data available online for analysis. Various organisations use this data to study consumer patterns and their behaviour. Sentiment Analysis allows data to be analysed and divided into positive, negative or neutral sentiments. Healthcare Centres can use Sentiment Analysis to study patient behaviours and serve them better in regards to treatment, diet and medicine. In this paper, we have extracted Tweets through Twitter Agent in Apache Flume using different block sizes, stored them in HDFS and studied their effect on Data Generation in terms of speed.