Educational Thoughts

To read previous related articles. Hadoop Eco System Installation - PART 1  Hadoop Introduction & Terminologies - PART 2 Data Formats Data Formats :  Way of representing and storing the raw data in the secondary storage devices.  Each of the data file format have got its own list of pros and cons depending upon the business context and usecase. Plain Text Files ( CSV, TSV, XML or JSON files ), binary files , rich file formats like Avro, ORC and Parquet To know more on data format click here:  https://techmagie.wordpress.com/category/big-data/data-formats/ Data Ingestion &  Data Streaming Ingest data in Batches or Stream data in real time. Both of the processes allows to collect, load, transfer, integrate and process data from wide range of data sources.  Data Ingestion Data ingestion is the process of obtaining and importing data for immediate use or storage in a database. To ingest something or to t...

Hey Guys! This article might be useful for the beginners those who wish to learn Big Data frameworks and choose their career in the stream of "Big Data Analytics, Big Data Scientist and Big Data Administrator". It is good that most of the tools and frameworks are open source.  Let us start with the definition of Big Data Big data is the domain which deals with voluminous of data both in veracity and velocity in-terms of  Mega, Giga, Tera, Peta, Exa and Zetta bytes . For example social media analysis (Recommenders), Online Transactional Data Processing (OLTP), e-commerce data (eBay, Amazon, FlipKart), ERP, Online Analytical Processing (OLAP), Financial Data, XML, JSON Objects, Customer Relationship Management ( CRM),  Sparse Data (Non IT Devices like sensors), larger volumes of image, video, audio data (NETFLIX AND AMAZON PRIME) etc. To start with hand-on practices there are three methods are suggested. 1. The fi...