Big Data: What Is Spark - An Easy Explanation For Absolutely Anyone

Like Hadoop, Spark is open-source and under the wing of the 

Apache  Software Foundation. Essentially, open-source 

means the code can be freely used by anyone. Beyond

 that, it can also be altered by anyone to produce custom 

versions aimed at particular problems, or industries.

Volunteer developers, as well as those working at 

companies which produce custom versions, constantly 

refine and update the core software adding more 

features and efficiencies. In fact Spark was the most 

active project at Apache last year. It was also the most 

active of all of the open  source Big Dataapplications, with 

over 500 contributors from more than 200 

organizations.

Spark is seen by techies in the industry as a more 

advanced product than Hadoop - it is newer, and designed 

to work by processing data in chunks "in memory". 

This means it transfers data from the physical, 

magnetic hard discs into far-faster electronic memory where 

processing can be carried out far more quickly - up to 

100 times faster in some operations.

Spark has proven very popular and is used by many 

large companies for huge, multi-petabyte data storage 

and analysis. This has partly been because of its speed. 

Last year, Spark set a world record by completing a 

benchmark test involving sorting 100 terabytes of data 

in 23 minutes - the previous world record of 71 

minutes being held by Hadoop.

Additionally, Spark has proven itself to be highly suited to 

Machine Learning applications. Machine Learning is one of

 the fastest growing and most exciting areas of 

computer science, where computers are being taught to 

spot patterns in data, and adapt their behaviour based 

on automated modelling and analysis of whatever task 

they are trying to perform.

Unlike Hadoop, Spark does not come with its own file system - 

instead it can be integrated with many file systems including 

Hadoop's HDFS, MongoDB and Amazon's S3 system.

Another element of the framework is Spark Streaming, which 

allows applications to be developed which perform analytics 

on streaming, real-time data - such as automatically 

analyzing video 

Source :- https://www.bernardmarr.com/default.asp?contentID=1079