HA Data Backend Layer
NodeFabric delivers hyperconverged database and storage solution for highly available, self-healing and load-balanced cloud services. It can run on any virtual or physical infrastructure: Amazon EC2 cloud, Openstack and VMware private clouds or directly on bare metal.

We are happy to announce initial NodeFabric Host Images and introductory articles public release – which should help anyone to get started with NodeFabric HA Data Backend Layer:

Part I:     NodeFabric Introduction
Part II:    NodeFabric Architecture
Part III:  Getting Started with NodeFabric in Amazon EC2 Cloud

Background

Modern cloud applications are often built with scalable frontend and business layer designs based on load-balancing – however achieving the same on data storage layer is a hard problem. It is relatively easy to provide single instance database or network file system mount for that, but it gets much more complicated if data backend has to be horizontally scalable, distributed, highly-available and self-healing.

Common options here would have been either to use cloud providers specific implementations of “cloud database” with possibly relaxed ACID requirements and network file-system services - with provider lock-in and blackbox magic - OR to build a DIY Pacemaker-based clusters for your own DB and NFS servers. The latter requires fencing and multicast networking for proper operation, not to mention that Pacemaker clusters can be complicated to setup and maintain.

All of these approaches are not really portable between different cloud environments, for example, between different cloud service providers or between private and public clouds. The DIY approach is typically using a single master active-passive clustering design with limited scale-out and self-healing capabilities.

Say hello to NodeFabric!

In our day-to-day work we really wanted to have active-active data backend clusters that could be easily deployed and ported across different cloud providers, hypervisors and bare-metal deployments. We use Docker a lot – so we dreamed about distributed data backends for Docker apps. We could not find anything suitable.

So what did we do?

We packaged MariaDB-Galera database and Ceph distributed storage components as Docker containers and included these into CentOS 7 based Nodefabric Docker Host Image build - together with Hashicorp’s Consul cluster state and coordination layer.

The result:

Pre-built VM and bare-metal “Boot-and-Go” NodeFabric Host Images – for launching auto-joinable, self-contained and efficiently co-locatable data backend clusters. Currently MariaDB-Galera SQL database cluster and Ceph are supported with more backends in the pipeline (RabbitMQ, etc).

How to use it?

Here is an example how NodeFabric could be used as a full vertical Solution Stack for including your own Docker app containers together with the clustered Data Backend Layer:

You need to boot up 3 or 5 cluster nodes using NodeFabric Host Image - either VMs or bare-metal hosts - and to provide cluster hostmap in a form of Hashicorp ATLAS service token or manually. Hostmap is required for auto-joining the NodeFabric Core Layer and starting the included MariaDB-Galera and Ceph storage services.

NodeFabric Core Layer uses quorum-based approach for clustering - it means that 3-node cluster can handle single node loss and 5-node cluster can afford losing two nodes in total. Automated cluster recovery after full outage/shutdown is also supported and does not require manual intervention.

So creating data backend clusters with NodeFabric comes down to 3 simple steps:

• Step 1: supplying remote registrator service token (for auto-hostmap) or entering base cluster hostmap manually - in standard hosts file format, mapping IPs to hostnames;
• Step 2: booting up cluster nodes and join the base cluster (ie NodeFabric Core Layer);
• Step 3: starting and bootstrapping “hosted” data backend clusters: MariaDB-Galera, Ceph et cetera .

That’s it!

Main features and benefits

• zero­-configuration fabric deployment ­- just “Boot-­and-­Go” -­ using prebuilt NodeFabric Host Image
  and optional remote auto-join service;
• self-contained and runs everywhere - AWS, Openstack, VMWare, KVM, bare-metal etc.
• very low infrastructure capabilities requirements for clustering - NodeFabric does not require multicast networking or node fencing;
• ability to efficiently co-locate different managed clusters;
• built-in service discovery and health monitoring;
• built-in load-balanced inter-service communication channels over localhost-like endpoints;

Status

NodeFabric is currently in Tech-Preview status - base features being there, with MariaDB-Galera and CephFS services included and usable. More features are planned, together with support for other built-in backend clusters - so please let us know which backend clusters you would like to see supported on NodeFabric - by emailing to: info@opennodecloud.com

Commercial services for NodeFabric deployment, support and custom development are available through OpenNode LLC - for request please write to: support@opennodecloud.com

NodeFabric has Amazon AMI, Openstack, KVM, VMWare, Parallels Desktop and VirtualBox images available.

Goto NodeFabric AWS Marketplace product page

AMIs by region:

 | Region                    | AMI ID       |
 |---------------------------|--------------|
 | US East (N. Virginia)     | ami-1daaf778 |
 | US West (Oregon)          | ami-1045a623 |
 | US West (N. California)   | ami-ddce0d99 |
 | EU (Frankfurt)            | ami-9cd0dc81 |
 | EU (Ireland)              | ami-79635c0e |
 | Asia Pacific (Singapore)  | ami-8cdccfde |
 | Asia Pacific (Sydney)     | ami-b1afe58b |
 | Asia Pacific (Tokyo)      | ami-f0315cf0 |
 | South America (Sao Paulo) | ami-5112834c |

Downloadable Openstack, KVM, VMWare, Parallels and VirtualBox images are available from sourceforge.net: https://sourceforge.net/projects/opennode/files/NodeFabric/

Getting Started

http://nodefabric.readthedocs.org

Part II:    NodeFabric Architecture
Part III:  Getting Started with NodeFabric in Amazon EC2 Cloud