Data Preprocessing With K Nearest Neighbor, Normalization In Big Data

Abstract

The proposed MapReduce framework works more effectively in big data. The big data concern
large-volume, complex, growing data sets with multiple, autonomous sources. This may lead
problems such as noisy data, missing data and redundant data because the big data is collected
from multiple systems or sources which affects the decision making for data mining with big data
in this work the k nearest neighbor algorithm is used to calculate the missing values. Assume
each feature of training data sets has a distinct dimension in some scope, and take an
observation value for its feature coordinates in that dimension, then acquire some set of points
from space. Then assume the identical of two points would be the distance between them in this
space depends on some metric. This method is used to gain proper dataset and it increases data
quality for further processes.