Data Quality Assessment & Enhancement for classifier’s Performance Improvement

Authors: Shubhangi sharma, Maya yadav

Published in: IJRRETAS, Volume 2, Issue 8, pp. 1-10 (01 Aug 2016)

Impact Factor: 2.3 (2016)

Certificate: View Certificate

Abstract

The data mining is a tool which includes various mathematical definitions and methodologies by which the data is corrected, manipulated, and identified according to their patterns. Additionally that is frequently used for various data classification, pattern recognition and other decision making tasks. The learning of data mining techniques are basically depends on the datasets or the training samples. According to the training samples the learners can be classified as supervised and unsupervised models. Additionally when the supervised learning is used the data sets are played more essential roles to guide the learning algorithm. But due to incomplete, random and inconsistent data can affect the performance of supervised classifiers. Therefore the proposed study work is concentrated over improving the data set quality using the pre-analysis of data. In order to find most optimum data analysis algorithm various techniques and methodologies are studied. But most of them have limitations such as limited class processing ability and a number of data processing steps. Thus a new data is proposed in this study using the linear regression analysis for analysing the data. The proposed algorithm works in three major steps first input data noise estimation using regression analysis, second noisy data separation and the finally the data correction. In these steps the data is analysed and unfit patterns or noisy patterns those are affecting the performance is detected and corrected using the outlier estimation technique. The implementation of the proposed algorithm is provided using the MATLAB environment. After implementation the performance of proposed and available technique is evaluated and compared over different datasets of machine learning. The comparative performance of the system demonstrates effectiveness of the proposed algorithm. According to the obtained results the proposed algorithm improves the classification accuracy between 5-25% additionally reduces the resource consumption approximately 5%. Therefore the proposed technique is adoptable due to high accuracy and less time consumption..

Introduction

different perspectives and summarizing it into useful information. Data mining is used to extract information from a dataset and transform it into an understandable structure for further algorithmic use. The real world applications such as financial data analysis, retail industry, biological data analysis, telecommunication industry and many more are utilizing the techniques of automated data analysis. According to the applications and the environment of learning the data mining methods may be categorized as either supervised or unsupervised. In unsupervised methods, no target variable (class labels) identified. The data mining algorithm responsible to search patterns among the entire feature sets. The unsupervised data mining method is also termed as clustering algorithms.

Conclusion

Data mining offers a way by which the hidden patterns form the raw input data are discovered. This process need to develop some algorithm that analyse the data and formulate them in a mathematical model. The formulated model helps to recognize the similar pattern data. But the noisy data can affect the formulation of data model and the decision making capability of the developed classifier model. Proposed study addresses the problem of noisy and incomplete datasets and their performance issues. Therefore a new technique is developing which preevaluate the data and their patterns for obtaining the incompleteness and noisy candidates of the datasets.