Maxta presents software defined storage for vSphere challenging traditional SAN/NAS. Software eats storage!
November 12, 2013 3 Comments
Software defined storage (SDS) is hot. Today Maxta came out of stealth mode and presented the world their software defined storage solution for VMware vSphere which looks like a perfect mix of VMware Virtual SAN and Nutanix capabilities. No SAN or NAS needed but resilient local storage for caching (consumergrade SDD) and persistent storage (HDD). Through its software-only solution, Maxta turns standard servers into a converged compute and storage solution
While the Maxta Storage Platform has been running for over a year in production now at an undisclosed number of customer sites, untill today the company did not do a lot of marketing. The current version of Maxta Storage Platform or MxSP is 2.2.3.
This skepticism about SDS was part of the reason why Maxta has kept under the radar since its 2009 start. Maxta decided not to announce the company until it had technology validation, and market validation.
The software is installed as a virtual machine running on each hypervisor (vSphere 5.0 Update 1 and higher). This virtual appliance aggregates local storage and presents it over NFS to the hypervisor. Hyper-V and KVM support is currently in Limited Availability)
Because it is a very cost-effective solution and is VM-centric (cost based on raw storage capacity) Maxta Storage Platform can be used for remote offices/branch offices instead of traditional storage solutions like Dell EqualLogic, Lefthand, HP StorageWorks 1000 Modular Smart Array, VMware Virtual Storage Appliance, VMware Virtual SAN.
Other use cases are VDI, Disaster Recovery and Test/dev.
Quite a few software defined storage solutions are based on ‘converged storage and compute’ or ‘datacenter in a box’. It means that storage for virtual machines is located in the same enclosure as compute (cpu+memory). The raw local storage in multiple servers is aggregated, replicated and made available to the server nodes in a resilient way. Storage is presented and managed either in kernel or by a software appliance running on the host. This architecture has many advantages over traditional SAN/NAS solutions.
VMware Virtual SAN or VSAN is a software defined storage solution currently in open beta. It gets a lot of attention from VMware customers as it reduces costs while enhancing performance & agility and simplifying management of storage compared to using traditional SAN/NAS solotions.
Nutanix has a packaged software and proprietry hardware solution which gets a lot of attention as well. Other players delivering Datacenter in a Box solutions are SimpliVity and Scale Computing. See my earlier blog in which I compare VMware VSAN with Nutanix.
At November 12 Maxta showed the world their software defined storage solution which is a hypervisor agnostic, highly resilient, scalable, VM-centric storage platform.
This post will introduce Maxta as a company and will provide all the technical details about their solution. Subjects discussed are:
- The features
- The architecture
- Storage features
For a three minutes introduction showing benefits of Maxta see the video below.
Inside-bigdata.com has a video presentation by Maxta of 21 minutes. Yoram Novick, CEO of Maxta, presents the solution.
Industry veterans realizing the vision of a software defined data center
Maxta is located in Sunnyvale, California in the United States. They have 20 employees at the moment. The company has been in a stealth mode as they say themselves since 2009 when the company was founded by CEO Yoram Novick. Venture capital firm Andreessen Horowitz invested $10 million into the company. The venture capitalist write a blog on Maxta titled ‘Software Eats Storage’
Yoram Novick studied Computer Science in Israel and started his career as a researcher for IBM Israel working on storage solutions. He then founded Topio which software allows customers to replicate, recover, and protect data over any distance regardless of the underlying server or storage infrastructure. Topio was acquired by NetApp in 2006. Novick became General Manager of NetApp Data Replication Business Unit and later founded Maxta.
Amar Rao is vice president of Business Development and OEM Sales at Maxta. He worked 18 years for IBM and later went to EMC, Kashya and EVault befoire joining Maxta.
One of the engineers who develops MxSP is Raghu Shastry. He worked for VMware on ESX storage stack and Distributed storage systems. He also worked on the core IO engine for Virsto products, enabling IO acceleration, efficient thin provisioning, high performance snapshots/clones and storage-tiering for virtual machines. VMware acquired Virsto in Feb 2013
At November 12 2013 Maxta really came out of the stealth mode. The layout of the website has been changed, whitepapers can be downloaded and the marketing machine has started.
Time to reveal the technical details of the solution.
Maxta Storage Platform
Maxta Storage Platform or MxSP is a software defined storage solution which converges storage and compute into a single industry, off the shelf server. As storage is close to the CPU, latency is less then when using a storage area network or NAS. There is no need for expensive storage arrays which can only scale by adding new boxes with a relative high capital investment. Operational expenses are reduced compared to traditional shared storage because of less floor space, less power consumption, less cooling, easier management etc.
Features of Maxta MxSP
- Hypervisor agnostic. Currently supports vSphere 5.0 U1 or higher. Hyper-V & KVM can be tested.
- Supports any server hardware listed on the hypervisor hardware compatibility list
- Combining Server-Side Caching using SSD with Server-Side Persistent Storage using HDD
- Management of storage and compute integrated in single console
- Support of consumer grade SSD
- Consistency groups to provide consistent snapshots or clone of an application spanning multiple virtual machines
- VM-level snapshots and zero-copy clones
- Thin provisioning
- Inline deduplication
- REST API for remote management
- Optimized writes by staging random write in buffer and then write sequentially. This eliminates the IO blender issue in which sequential writes from VMs become random when multiple VMs write to the same disk.
- Uses checksums to guarantee data integrity
- Supports maximum virtual disk size of VMware ESXi 5.5. So 62 TB VMDK is supported when vSphere 5.5 is used. VSAN is currently limited to 2 TB VMDK.
Maxta currently has an undisclosed number of paying customers some of which are using the software for over a year in production now. For example a Dutch governmental organization is using the software. Case studies available are of Driscoll’s which is a leading multinational company based in Watsonville, California. Another case study is VSS Monitoring, a Danaher company (a Fortune 200 company) which is a world leader in network packet brokers (NPB).
Maxta MxSP is used in four different use cases by those current customers: VDI, Disaster Recovery, Branch offices and test & development.
In the use case for DR Maxta is used in the secondary site. The customers using Maxta have for example an EMC/HP/NetApp storage array in the primary site and use vSphere Replication to replicate to their secondary site.
I have not tested MxSP myself yet. However The Register, a well informed UK website on IT, got the software and installed MxSP in their lab. The author was impressed by the simplicity of the installation. Read the complete article here. A quote from the article:
Maxta’s play is pure simplicity, and if a day’s worth of hammering on it in the lab is any indication, it Just Works
The installation prerequisites are:
Maxta MxSP architecture
MxSP is installed as a virtual appliance on each node participating in a Maxta storage cluster. The guest operating system used is CentOS, a Linux distribution.
The main components of MxSP are:
- Maxta Distributed File System (MDFS)
- Maxta Management Server
- NFSv3 server
Maxta Distributed File System: An instance of MDFS will be running in each of the servers that are part of virtualization cluster. MDFS enables the creation of the global namespace delivering high availability and scalability. MDFS synchronizes the operations between the instances and eliminates the split-brain condition where each server believes that the other server is nonoperational.
Maxta Management Server: The management server is deployed as a web server on one of the nodes running an instance of MxSP. In the event of a server failure where the management server was deployed, the management server automatically fails over to another node running an instance of MxSP. The management server leverage RESTful APIs to provide administration and configuration of MxSP. These APIs provide the basis of delivering the integration of user interface into the virtualization UI.
NFSv3 server: Maxta Storage Platform implements an NFSv3 server. The assimilated storage across all the servers in the virtualization cluster running the MxSP instance will be presented as a single Maxta storage pool to the hypervisor. The Maxta storage pool is presented to the hypervisor leveraging NFS protocol. The Maxta storage poolcan co-exist with storage from existing storage arrays eliminating the need to rip and replace existing storage infrastructure.
Maxta MxSP supports two kind of server nodes. A Converged Compute/Storage server and a Compute Only server. A Converged Compute/Storage server has a minimum of 1 HDD. It is highly recommended to have at least 1 SSD as well for performance reasons. . Any VMware PCIe SSD, SSD, HDD & controller is supported as long as it is on the HCL of the hypervisor. A Compute Only server does not need any storage devices (it leverages the Maxta storage but doesn’t contribute storage to it).
In a cluster, you must have at least 2 Converged Compute/Storage servers (to avoid single point of failure). The rest of the servers can be any combination of Converged Compute/Storage servers and Compute Only servers.
The SSD supports both write back cache and read cache.
The maximum number of SSD/HDD in a server and the maximum capacity of storage in a server is the maximum that the virtualization hypervisor (ESX in VMware environments) supports.
SSD is not required. However it is highly recommended to have SSDs in the server configuration for performance optimization (cache). The current recommendation is to have at least 1 SSD in each of the servers that are contributing storage to the Maxta datastore. The minimum disk drives in a system is also 1.
Virtual disk files are presented using NFSv3 to the hypervisor. NFS is a filebased disksystem instead of iSCSi and FiberChannel which are block based.
NFS has advantages over blockbased storage. Also some caveats. For example Microsoft does not support running Exchange Server mailbox roles on virtualized servers which are using Network attached storage like NFS. Microsoft makes one exception: fileshares presented by SMB 3.0 connected by Hyper-V are supported.
Running Exchange mailbox servers from NFS will perfectly work. Only if you need Microsoft support for technical issues they might not offer support because of the unsupported configuration. Vendors like Tintri who are also using NFS to present storage have a written statement saying they will provide best effort support in case Microsoft does not want to provide support.
MxSP supports NRAID10. This means for each block of data a replica is written on another server running MxSP. This means that half of the raw capacity of storage is lost because of this replica. So 10 TB of raw capacity delivers 5 TB of useable storage.
MxSP leverages multiple tiers (SSDs and spinning disk) of storage but in a different way. MxSP leverages both SSDs and spinning disk. MxSP leverages SSDs for both read and write-back caching. The writes are first written to the SSD in a sequential manner and replicated synchronously to an SSD on a second server for data availability. The writes are later de-staged on a periodic basis to the magnetic drives in a sequential manner. I/O performance and SSD endurance are maximized by always performing writes to both SSD and magnetic drives in a sequential manner. The metadata and frequently accessed data (“hot data”) are cached on SSD to enable read acceleration. Leveraging SSD for metadata caching and read caching improves the performance of random read operations. While the hypervisor is leveraging RAM for frequently accessed data and metadata as a higher caching tier in front of SSD, SSD caching enables significant performance improvement since it is substantially larger than RAM supporting larger working sets and it is capable of providing write-back caching.
Replication of virtual machines can be performed using vSphere Replication, Veeam Backup & Replication or any other replication tool. Maxta MxSP does not provide an out of the box, native replication feature yet. There is no VMware Site Recovery Manager Storage Replication Adapter (SRA) supplied at the moment. Maxta has this on the roadmap.
MsXP delivers VM level storage abstraction rather than block or file level abstraction. With Maxta administrators manage VMs, not storage. It eliminates the need for provisioning and managing low level storage constructs such as LUNs and volumes dramatically simplifying IT
MxSP enables the creation of time/performance/space efficient snapshots and clones. Even without the hardware acceleration the time it takes to create a clone is in the order of milliseconds. This technology becomes very important for legacy implementation of clones or full copy. Performance of virtual machines is not effect even when hundreds of snapshots are made.
Zero-clones allows to make an exact copy of a virtual machine without consuming twice as much as data.
VAAI as well as VASA are not yet supported but are on the roadmap.
To protect data from being corrupted for each block of data written a checksum is generated. This checksum (or signature) is written on another device. On every read the data block is compared to the checksum. If there is a mismatch, MxSP reads the block from one of the other copies (either at the same server or another server) and always provide the customer good data. This feature address both hardware issues as well as potential hypervisor issues.
Support of industry standard server hardware
Maxta MxSP can be installed on any server configuration supported by the hypervisor. So there is no separate hardware compatibility list as VMware has for VSAN.
Maxta is hypervisor agnostic. The GA product currently supports VMware vSphere 5.0 update 1 and later. However customers can test Hyper-V and KVM according to Maxta. Maxta currently does not see a lot of demand from its customers for those other two hypervisors. The advantage of being hypervisor agnostic is that once VMware customers decide to start using Hyper-V or KVM the investment in the hardware and Licensing is not lost. VSAN, a seperate per socket licensed product, is only available for vSphere as it is embedded in the kernel.
Maxta has a VMware vCenter Server plugin which allows to manage storage and the VMware virtual infrastructure from a single pane of glass. So there is no need to have two consoles open for management.
MxSP provides co-location. This mean that the MxSP always tries to keep the virtual disk belonging to the virtual machine on the same server as the virtual machine is running.
Support of consumer grade SSD
Maxta uses SSD as a performance layer. Data is cached in SSD to increase write and read performance. SSD offers much improved performance over hard disks but they have a caveat. The lifespan of a SSD is limited and depends on how much time it is written to it.
Two classes of SSD are available: enterprise grade MLC SSDs have a lifespan of about 30000 erase/write cycles, while consumer grade MLC SSD have a life span of 5000 to 10000 erase/write cycles.
Maxta supports consumer grade SSD drives. This is an pretty unique feature. Most other storage vendors only support more expensive enterprise SSD. Examples of other storage vendor using consumer grade SSD are Pure Storage, Dell Compellent and Tintri. This blogpost clearly describes why they use consumer grade SSD. Pure Storage is using FlashCare to extend the lifespan of the SSD. Another storage vendor using consumer grade SSD is Skyera.
Maxta writes data sequentially to the SSD instead of random. Random writes have a much lesser performance and have an negative effect on the lifetime of the disk.
A node can be either a compute node or a data node. A compute node is a node without local storage participating in the Maxta distributed volume.
Maxta fully does VM level snapshots. There is no performance degradation like when VMFS snapshots are commited. Snapshots are ready in milliseconds.
MxSP performs in-line compression of data with no performance penalty.
A MxSP cluster needs to have at least two data nodes. To prevent split brain when a node fails a third witness server needs to be available. This can even be a laptop. The maximum size of a Maxta cluster is the maximum of the vSphere cluster.
Maxta MxSP does not need nodes with the same disk configuration. Nodes can have dissimilar disk configurations.
The image below shows the specs for the server nodes. It has been translated from Japanese so sorry for some weird translation.
The MxSP software will be available in a single edition delivering all the features mentioned in this blogpost. Pricing is based on the raw capacity. So customers can use any number of nodes they like, with any number of CPU sockets. The license is based on the sum of raw disk capacity on all nodes where MxSP is installed.
Maxta uses several bands (tiers) with different price per TB. The more storage capacity installed, the lower the price per TB.
Any VMware vSphere edition is supported so even Essentials. vSphere 5.0, 5.1 and 5.5 are supported. That is a big advantage over Virtual SAN for example which only supports vSphere 5.5. Most VMware customers are running a previous version and are only able to use VSAN after upgrade to 5.5.
Maxta provides an evaluation license, the software can be downloaded from the website on request.
Maxta did not disclose the listprice of the software. Customers currently using the software say it is much more cost efficient than traditional storage solutions, according to Maxta.
Maxta’s default support offers a 24×7 4hr response time. Support cost is 20% of the list price of the software per year. If a customer purchases a 3 year support upfront then it is 50% of the list price of the software.
Maxta MxSP looks like a mix of the best ingredients of VMware VSAN and Nutanix. It provides software defined storage using any industry standard server using any combination of consumer grade, cheap SSD and HDD disks and is hypervisor agnostic. It can be used by customers running vSphere 5.0 Update 1 and higher. Hyper-V and KVM is available as well.
Some links to other blogs about Maxta:
Chris Wahl: Maxta Debuts Software-Defined Storage for Virtualization
Willem ter Harmsel Introduction to Maxta Storage
IT Trends & Analysis Maxta Breaks The Code For Integrating Storage And Compute
The official pressrelease of Maxta: Thinking Outside the Storage Box: Maxta Debuts First Software-Only Enterprise Storage Solution for the Virtual Data Center
SSG-Now Maxta Storage Platfrom
Architecting.it Virtualisation – Solving the Storage Problem
juku.it VSAs Everywhere! (and two words about Maxta)
The Register has tested the software: Maxta chucks vSAN out of stealth and into El Reg review suite. MxSP promises to be as simple as you are