Kategorie: Services

Behind Scality’s management interface is a sophisticated key/value store with very rich metadata-handling capabilities.

Scality is built on three key concepts

RING — the core of Scality technology is Scality’s patented RING technology. RING intelligently manages the What?, When?, Where?, Why?, and How?, of storage and retrieval.
• Inspired by peer-to-peer technology, RING is hardened to provide carrier-grade service availability and data reliability.
• It is composed of a number of nodes, typically off-the-shelf servers.
• Each node on the RING is responsible for its own piece of the overall storage puzzle.
• Every node is functionally equivalent. A node can disappear or you can add a new node and the system will rebalance automatically without intervention.
• There is no need for a master database or directory.
• Every node constantly monitors a limited number of its peers, and automatically rebalances replication and load to make the system completely self-healing

Connectors — an connector is the piece of software that communicates with your application. Scality supports multiple accessors— HTTP REST, RS2, Zimbra, Open-Xchange, Dovecot and more—and you can develop your own purpose-built accessor with Scality’s API tools.

BizIOD — an IO daemon developed by Scality that communicates directly with specific storage hardware. BizIOD supports most existing storage hardware technologies, either directly attached – SAS, SCSI, SATA, or network attached – iSCSI, Fiber Channel, NFS v3 and NFS v4, Dispersed Storage and more.

How Scality works

Any store or retrieve request for an object goes through an “accessor”. The accessor is close to your application, and it decides to how to handle the request. For example, depending on specific data attributes, the accessor may assign a storage class—the data needs to survive one, two or more disk failures. The accessor makes intelligent decisions about how it handles requests for larger files, such as a movie, and smaller objects like an email. It may decide to stream the former, for example.

Accessors contact one of the nodes on the RING—it doesn’t matter which node because that node will recommend the node to contact next. The protocol is such that the algorithm will converge in one hop for a ring of 10 nodes, two hops for a RING of 100 nodes, and three hops for 1000 nodes. Each node can typically handle 10-to-50 TB of storage. Within three hops, one node can address a massive 50 petabyte of storage within ≤ 20 ms latency on a typical gigabit LAN.

Once the correct node has been reached, the object will be passed to BizIOD, which places it on the storage hardware. The storage hardware can be memory (for caching), solid-state disk (for very high throughput); it can be directly-attached storage or network-attached storage. Several BizIOD can co-exist on the same RING node.

• The RING nodes also manage which objects should be off-loaded to Tier Two storage, according to a sophisticated algorithm that takes into account read/write patterns for that object, as well as information about the object type.
• RING implements a proven compression/de-duplication algorithm when data is pushed to Tier Two. Data is still accessible through the accessor, which means the process is completely transparent for the application.
• The RING system can be set to never delete information and supports undelete. Typically, to add throughput (IOPS) you simply add Tier One RING nodes.
• To add capacity you add Tier Two storage. Tier Two storage can be implemented with another RING sub-system with a different hardware configuration, a NAS-based system, or with technologies such as Dispersed Storage which deliver sub 1,000 USD/TB price points with very high reliability.

Additional functionality

• RING — RING manages the number of replicas requested for each object. There is no concept of master, every replica is functionally equivalent. When a replica disappears, the RING system automatically detects it and recreates it without external intervention
• Accessor — when there are multiple copies of a given object, the accessor automatically load balances, reading across all replicas and reconciling versions transparently
• BizIOD — there is one BizIOD running per physical piece of storage hardware. If a given disk is causing problems, it only impacts that BizIOD daemon, not the entire RING node

Source: http://www.scality.com