Data Defined Storage
||This article appears to be written like an advertisement. (August 2013)|
New advancements in technology, such as growing image and video resolution, social media, mobile internet and bio-sensitive medical devices, are contributing to the increasing interconnectivity of the world through the Internet of Things. These advances drive increasing volumes of data, causing traditional models for managing data to be stretched to their scalability limits  This phenomenon has driven the storage industry and data management industry to take a new approach to scalable data storage, known as Data Defined Storage, or abbreviated DDS. This mechanism for storing, retaining and accessing data is based on content, meaning and value. Data Defined Storage addresses the requirements of industries characterized by escalating data volumes, increasing regulatory requirements and data driven innovation.
Data Defined Storage is a new approach to managing, protecting, and realizing value from data by uniting application, information and storage tiers into an integrated data centric management architecture. This is achieved through a process of unification, where users, applications and devices gain access to a repository of captured metadata that empowers organizations to access, query and manipulate the critical components of the data to transform it into information, while providing a flexible and scalable platform for storage of the underlying data. The technology abstracts the data entirely from the storage, allowing full transparent access to users for a high performance, grid based, scalable approach to storage and data management.
Data Defined Storage focuses on metadata with an emphasis on the content, meaning and value of information over the media, type and location of data. This translates into key innovations underlying the solution and are necessary to meet the emerging requirements associated with the growing data volumes. Data Centric Management enables organizations to take a single, unified approach to managing data across large, distributed locations with content and arbitrary metadata tag search and big data analytics integration. The technology pillars include:
- Media Independent Data Storage: Data Defined Storage removes media centric data storage boundaries within and across solid-state drive, hard disk drive, cloud storage and tape storage platforms, enables linear scale out functionality through a grid based Map Reduce architecture and provides transparent data access across globally distributed repositories for high volume storage performance.
- Data Security & Identity Management: Data Defined Storage allows organizations to gain end-to-end identity management down to the individual user and device level providing consumerization of IT, mobility and enhanced data security and information governance.
- Distributed Metadata Repository: Data Defined Storage enables organizations to virtualize aggregate file systems into a single global namespace, and access metadata information to extract value leading to informed business decisions and analytics.
The first implementation of Data Defined Storage was pioneered by Tarmin, in its GridBank Data Management Platform. This technology was developed after Tarmin founders Shahbaz Ali and Steve Simpson experienced the challenges associated with storing, capturing and processing massive volumes of financial transactions first hand at MasterCard. GridBank was designed to meet the need of Data Designed Storage, by taking a data centric approach to storage and data management.
The Data Defined Storage market is still in its early days, there is wide acknowledgement amongst key storage and data management industry players that the future of the market is in emphasizing the value of data through scalable distributed metadata management techniques Additionally, Data Defined Storage has received strong support from leading analyst firms including ESG and IDC. Pioneering customers have begun deploying Data Defined Storage solutions in industries such as oil and gas data discovery, healthcare, financial services and managed service providers.
Data Defined Storage takes the approach of unifying object storage with open protocol access for file system virtualization, such as CIFS, NFS, FTP as well as REST APIs, integrating an information governance policy management and data mover engine and consolidating unstructured metadata into a distributed repository.When implemented in combination with block storage technologies, such as IBM XIV, Data Defined Storage unifies block, file and object technologies to enable high performance, massively scalable data storage.
- Scale out storage
- Object storage
- Content-addressable storage
- Block storage
- Information governance
- Enterprise search
- Metadata management
- Big data analytics
- "Object Storage: The Future Building Block for Storage Systems". IBM Hiafa Research Laboratories.
- Peters, Mark. "Unlocking the Power of Data with Data-Defined Storage". ESG. Retrieved June 2013.
- Goyal, Ambuj. "Edge2013 General Session Keynote Speech". IBM Edge.
- Miller, Dan (12 July 2013). "Tarmin and IBM help Premier Oil manage rapidly growing unstructured data". PR Newswire.
- Miller, Dan (17 December 2012). "Leading U.K. MSP brightsolid sees a shining future with Tarmin". PR Newswire.
- "IBM XIV Storage System Gen3 Architecture, Implementation, and Usage". IBM.
- "Object Storage: The Future Building Block for Storage Systems". IBM Haifa Research Laboratories.