A Review of Big Data Challenges and Techniques

Abstract

Big data is the most important trend that is defining the new emerging analytical tools. Big data has various applications in different areas like traffic control, weather forecasting, fraud detection, security, education and health care. Extraction of knowledge from massive amount of data sets has become a challenging task. Big data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, store and analyze it within a tolerable elapsed time. Due to widespread usage of many computing devices such as smart phones, laptops, wearable computing devices; the data processing over the internet has exceeded more than the modern computers can handle. Due to this high growth rate, the term Big Data is envisaged. However, the fast growth rate of such large data generates numerous challenges, such as data inconsistency and incompleteness, scalability, timeliness, and security. The question that arises now is how to develop a high performance platform to efficiently analyze big data and how to design an appropriate mining algorithm to find the useful things from big data. This paper begins with a brief introduction to the big data technology and its importance and also focuses on various challenges and issues that need to be emphasized. The tools used in big data technology are also discussed in detail.