Apache Spark

Apache Spark is a lightning-fast software engine designed for large-scale data processing. It supports in-memory computing and cyclic data flow, and allows to achieve up to 100x memory speed boosts if compared to other technologies, and up to 10x on disk, while remaining easy in use and maintenance. Spark is well-designed for the development of Java, Python and R-based applications due to its 80+ high-level operators, rich stack of seamlessly combined libraries (SQL, machine learning, GraphX, etc.) and can run everywhere - standalone, in the cloud, and on Hadoop.