Open Access Open Access  Restricted Access Subscription or Fee Access

Big Challenges in Big Data Research

Devesh Kumar Srivastava

Abstract


Data-driven decision-making is now being accepted widely, and there is rising excitement for the notion of ``Big Data.’’ With more than one Exabyte of data being created in circadian way, astronomically immense data personifies a major en-sample shift in today's mission critical enterprises. In this paper we discussed the new demanding of Big data which bring research work to data scientists. Big Data introduce statistical challenges including scalability, unique computational and storage bottleneck, incidental endogeneity, noise augmentation, fake correlation, and measurement errors. These distinguished demands require new computational and statistical paradigm.  Big Data hold great assurance for discovering overnice patterns and heterogeneities that are impossible with minute- scale data. We also provide various new aspects on the Big Data analysis and computation.


Keywords


Big Data, Data Storage, Scalability, Large-Scale Optimization, Massively Parallel Data Processing, Hadoop Distributed File System (HDFS)

Full Text:

PDF

References


White, Tom (10 May 2012). Hadoop: The Definitive Guide. O'Reilly Media. p. 3. ISBN 978-1-4493-3877-0.

Vance, Ashley (22 April 2010). "Start-Up Goes After Big Data With Hadoop Helper". New York Times Blog.

WEF (World Economic Forum), & Vital Wave Consulting. (2012). Big Data, Big Impact: New Possibilities for International Development. World Economic Forum. August 24, 2012, from http:// www.veforum. org // reports /big-data-big-impact-new-possibilities-international -development

Oracle and FSN, "Mastering Big Data: CFO Strategies to Transform Insight into Opportunity", December 2012

Jacobs, A. (6 July 2009). "The Pathologies of Big Data". ACM Queue.

J Magoulas, Roger; Lorica, Ben (February 2009). "Introduction to Big Data". Release 2.0 (Sebastopol CA: O’Reilly Media) (11).

UN GLobal Pulse (2012). Big Data for Development: Opportunities and Challenges (White p. by Letouzé, E.). New York: United Nations. Retrieved from http://www.unglobalpulse.org/projects /BigDataforDevelopment

Laney, Douglas. "3D Data Management: Controlling Data Volume, Velocity and Variety". Gartner. Retrieved 6 February 2001.

Beyer, Mark. "Gartner Says Solving 'Big Data' Challenge Involves More Than Just Managing Volumes of Data". Gartner. Archived from the original on 10 July 2011. Retrieved 13 July 2011.

Laney, Douglas. "The Importance of 'Big Data': A Definition". Gartner. Retrieved 21 June 2012.

D. Agrawal, S. Das, and A. E. Abbadi Big data and cloud computing: New wine or just new bottles? PVLDB, 3(2):1647–1648, 2010

Delort P., Big data Paris 2013 http://www.andsi.fr/tag/dsi-big-data

Jeffrey Dean and Sanjay Ghemawat , “MapReduce: Simplified Data Processing on Large Clusters” 2004 http://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf

Big Data for Development: From Information- to Knowledge Societies", Martin Hilbert (2013), SSRN Scholarly Paper No. ID 2205145). Rochester, NY: Social Science Research Network; http://papers.ssrn.com/ abstract= 2205145

Webster, John. "MapReduce: Simplified Data Processing on Large Clusters", "Search Storage", 2004. Retrieved on 25 March 2013.

Dave Beulke, Big Data Impacts Data Management: The 5 Vs of Big Data “, http://davebeulke.com/big-data-impacts-data-manage ment-the-five-vs-of-big-data, November 2011..

Lu, Haiping; Plataniotis, K.N.; Venetsanopoulos, A.N. (2011). "A Survey of Multilinear Subspace Learning for Tensor Data". Pattern Recognition44 (7): 1540–1551. doi:10.1016/j.patcog.2011.01.004.

Obama Administration Unveils "Big Data" Initiative: Announces $200 Million In New R&D Investments". The White House.

[ Cartell R. Scalable sql and nosql data stores. SIGMOD Record 39(4):12–27, 2010.

Big Data @ CSAIL". Bigdata.csail.mit.edu. 2013-02-22. Retrieved 2013-03-05.

Graham M. (2012). "Big data and the end of theory?".The Guardian.

Anderson, C. (2008, June 23). The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. Wired Magazine, (Science: Discoveries). http://www.wired.com/science/discoveries/magazine/16-07/pb_theory

Danah Boyd (2010-04-29). "Privacy and Publicity in the Context of Big Data". WWW 2010 conference. Retrieved 2011-04-18.

Jones, MB; Schildhauer, MP; Reichman, OJ; Bowers, S (2006). "The New Bioinformatics: Integrating Ecological Data from the Gene to the Biosphere" (PDF). Annual Review of Ecology, Evolution, and Systematics 37 (1): 519–544. :10.1146/annurev.ecolsys. 37.091305.110031.

The Search for Analysts to Make Sense of Big Data. Yuki Noguchi. National Public Radio, Nov. 30, 2011. http://www.npr.org/2011/11/30/142893065/ the-search-for-analysts-to-make-sense-of-big-data

The Age of Big Data. Steve Lohr. New York Times, Feb 11, 2012. http://www.nytimes.com/2012/02/12/sunday-review/big-datas-impact-in-the-world.html


Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.