Global Sources
EE Times-India
Stay in touch with EE Times India
EE Times-India > Processors/DSPs

Fujitsu uncovers column-oriented data-processing engine

Posted: 02 Mar 2015     Print Version  Bookmark and Share

Keywords:Fujitsu Laboratories  data-processing engine  PostgreSQL  database  CPU 

Fujitsu Laboratories Ltd has created a column-oriented data-storage and processing engine that boasts fast analysis of large volumes of data in a database system. The technology runs on a PostgreSQL open-source database, without being dependent on memory capacity, and instantly updates column-oriented data in response to changes in row-oriented data, and processes column-oriented data quickly, detailed the company.

According to the company, the engine quickly analyses indexes, which are provided by most database systems, and can be used by developers without special consideration to whether the storage method is row-oriented or column-oriented. With a parallel-processing engine especially suited for processing column-oriented data, analyses run on a single CPU core are conducted four times faster than before, and one server equipped with 15 CPU cores can run analyses at least 50 times faster.

Even on smaller computer systems with little memory, this technology enables real-time data analysis reflecting the latest data.

The engine enables high-speed analyses of the indexes that typical database systems provide, and can be used without special consideration for whether the data is stored as row-oriented or column-oriented. On the DBT-3 benchmark Query1 for reading, filtering and aggregating, the parallel-processing analysis engine, which has been optimised for column-oriented data. On a single server with 15 CPU cores, performance is at least 50 times faster, according to the company.

Large-volume column-oriented data storage

To efficiently manage large volumes of column-oriented data that cannot fit into memory, data domains are managed in "extents," large increments (roughly 260,000 records), in which data domains are secured or deleted, and in which free domains are reclaimed. While managing large increments and simultaneously running analyses can result in long wait times, Fujitsu has adopted a solution in the form of MultiVersion Concurrency Control (MVCC), which allows analyses to run at the same time that data domains are managed.

1 • 2 Next Page Last Page

Comment on "Fujitsu uncovers column-oriented dat..."
*  You can enter [0] more charecters.
*Verify code:


Visit Asia Webinars to learn about the latest in technology and get practical design tips.


Go to top             Connect on Facebook      Follow us on Twitter      Follow us on Orkut

Back to Top