Global Sources
EE Times-India
EE Times-India > EDA/IP

Open-source platform eases smart app development

Posted: 17 Jun 2015     Print Version  Bookmark and Share

Keywords:Spark  data scientist  data engineer  analytics  open source 

IBM will put Apache's Spark platform on the commercial open-source operating system (OS) map, a move that will help train more than one million data scientists and engineers. In addition, IBM will also build Spark into its analytics and commerce platform, said Beth Smith, general manager of Analytics Platform at IBM, at the Spark Summit 2015 (June 15-17, San Francisco).

"IBM is fully committed to Spark as a foundational technology platform for driving analytics across every business in a fundamental way," Smith told EE Times in advance of her keynote speech at the Spark Summit. "Spark will advance our user's data strategies, driving business transformation and differentiation."

Apache Spark

Just as IBM put open-source Linux on the professional OS map likewise it is announcing a major corporate wide commitment to Apahe Spark—the in-memory platform for cluster computing apps like machine learning. (Source: Apache)

The most surprising aspect of IBM's announcement, according to Mike Gualtieri, principal analyst at Forrester Research Inc., is IBM's level of commitment to such a new open-source project—what IBM calls "the most important new open source project in a decade." At the Spark Summit, IBM promised to commit 3,500 of its own researchers and developers to Spark-related projects at over a dozen laboratories worldwide.


Smith: Spark will advance our user's data strategies. (Source: Apache)

IBM will also offer Spark as a service on its Bluemix resource—an open-source platform that allows application developers to load their data, model it and derive predictive results for use in their app. But most important of all, according to Gualtieri, is that IBM is donating its internally developed SystemML machine learning technology to enrich the growing Spark ecosystem and promises to collaborate with Databricks to further improve Spark's machine learning capabilities in the future.

What is Spark?

Apache Spark is an open-source cluster computing framework (originating at University of California at Berkeley in 2009). It simplifies the process of developing "smart" distributed applications. By managing in-memory computing resources, it provides primitives that can boost performance by 100 times for applications like machine learning. Spark keeps all often-used data in-memory, rather than on mass storage devices, allowing it to be quickly and repeatedly accessed, which is why it is appropriate for smart apps such as machine learning. The Apache Software Foundation claims Spark is its most active project, with over 465 contributors in 2014 alone.

IBM is also planning to open a Spark Technology Center in San Francisco to assist the data-science and -developer community. It is also extending its partnerships with the University of California at Berkeley's AMPLab, DataCampz, MetiStream, Galvanize and the Big Data University (originally the Massive Open Online Course or MOOC).

- R. Colin Johnson
  EE Times U.S.

Comment on "Open-source platform eases smart app..."
*  You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.


Go to top             Connect on Facebook      Follow us on Twitter      Follow us on Orkut

Back to Top