How Gartner’s 2015 Magic Quadrant for Solid State Storage Arrays has NetApp Wrong

Intro

NetApp was not placed high in Gartner’s 2015 Magic Quadrant for Solid-state Storage Arrays. Your’s truly did some research to see if this was warranted or not. The result is this article.

This article reviews the 2015 Gartner Magic Quadrant for SSAs (Solid-state Storage Arrays), and notable changes to the ratings of key vendors in this market. There is a critical look to see if NetApp experienced improved results since 2014, and why (or why not). It concludes with a description of additional concerns that 46and2bits.com has with the Magic Quadrant for SSAs (concerns that are not specific to NetApp).

  • Note to the reader: The author of this article is a Competitive Analyst at NetApp. Statements and opinions made do not reflect those of NetApp Corp.

Background

What is the Gartner Magic Quadrant for Solid-state Storage Arrays?

According to Wikipedia.org, Gartner (founded in 1979) “is an American information technology research and advisory firm providing technology related insight … Research provided by Gartner is targeted at CIOs and senior IT leaders […]. Gartner clients include large corporations, government agencies, technology companies and the investment community”.

The general concept of Gartner’s Magic Quadrant can be found at http://www.gartner.com/technology/research/methodologies/research_mq.jsp. “By applying a graphical treatment and a uniform set of evaluation criteria, a Magic Quadrant helps you quickly ascertain how well technology providers are executing their stated visions and how well they are performing against Gartner’s market view.” The top of this web page is shown below.

2015_07_06_gartner_mq_methodology

Observe that the Magic Quadrant is just one of several “Methodologies” used by Gartner when assessing the major vendors (aka “technology players”) within a market. Other Methodologies include …

  • Critical Capabilities
  • Hype Cycle
  • IT Market Clock
  • Market Guide
  • Vendor Rating
  • ITScore
  • Market Share
  • Market Forecast

Gartner has successfully crafted their art of analysis to abstract key concepts in each market in a way that is as market-independent as possible. Thus, Gartner has a “template” of methodologies that can be re-used for either Cloud Service Providers or even (in our case) Solid State Storage Arrays. Although there are some concerns raised in this article, make no mistake – there is not a single independent analyst firm that is further evolved at quantifying the virtually un-quantifiable than Gartner.

Gartner has analyzed the IT Storage Array Market for many years. Starting in 2011-2012, the introduction of all-flash (all-SSD) primary storage at attainable (albeit still expensive) prices was creating a serious challenge. It wasn’t clear how to rationalize against traditional HDD arrays in a single technology market.

To resolve this, Gartner in 2014 introduced a new technology market dedicated to SSA’s, which also (intentionally or not) declaring that SSA was a new tier (or even silo) of storage. 2015 was the second year of Gartner’s MQ for SSAs.

General Industry Reception of Gartner’s MQ for SSAs

Most IT customers hold Gartner in high regards, and thus their view of Gartner’s MQ for SSAs inherits this respect.

The opinions of most IT Storage vendors for Gartner’s MQ for SSAs is generally favorable. No surprise, the higher rated a vendor is on the MQ for SSAs, the more favorable that vendor will be of the MQ for SSAs. For example, see the Tweet below…

The response of the Press has been fair, with the new MQ is often cited in trade articles. Analysts who are more free to say what is on their mind have been more mixed. In a late June 2015 Tweet, Howard Marks (www.deepstorage.net) said (in response to the Tweet below):

I agree with Howard Mark’s well grounded points in NetworkComputing.com (See http://www.networkcomputing.com/storage/gartner-all-flash-array-magic-quadrant-a-bad-influence/a/d-id/1320626 ), and believe Gartner should give them serious consideration. Some quotes are shown below:

… many vendors are so driven to be included in the analysis that they design products to fit Gartner’s definitions even when they believe that there is, or might be customer demand for something else. […]

Gartner even lists vendors that make “SSD” storage arrays that didn’t qualify for the MQ such as Dell and NetApp, implying that a SSD array like a Dell Compellent is somehow less than a real all-flash array like Pure Storage’s or an EMC VNXF, even though the flash in those systems is packaged in SAA SSDs, just like the flash in a Dell Compellent SC2040.

As an IT architect, this makes no sense to me. When choosing the product (or products) I want to solve my storage performance problem, I don’t care if they have a unique product name or share the name with a hybrid, or even all-disk, system using the same architecture. All I care about is that the storage system provides the performance and features that address the problem I’m trying to solve.

The 2015 MQ for SSAs

The 2014 MQ for SSAs is shown below. At the time, NetApp’s AFF was not fully productized, and the review really only considered EF for NetApp.

During the time between Gartner’s Release of the 2014 and 2015 MQ for SSAs, NetApp officially productized AFF and has established a solid revenue stream for this new offering. With the addition of this fast-growing growing, proven and enterprise ready offering, many expected a major improvement for NetApp in the Gartner MQ for SSAs in 2015. (Unfortunately, due to the timing of the 2015 MQ, further enhancements to AFF with ONTAP 8.3.1 are not reflected)

So what changed in 2015 (See http://www.gartner.com/technology/reprints.do?id=1-2IDHAOJ&ct=150624&st=sb)? On June 23 (2015), Gartner released the new “Magic Quadrant” (MQ) Report for Solid-State Arrays (SSAs):

  • Pure : Remains Highest “Completeness of Vision”, and moved up from 3rd to 2nd in “Ability to Execute”
  • EMC : Remains Highest “Ability to Execute”, and remains second in “Completeness of Vision”
  • HP : Advanced from Challengers Quadrant to Leaders Quadrant
  • IBM : Lowered rating on “Ability to Execute” (moving from second to third), yet remains in the Leaders Quadrant

And what about NetApp?

2015_06_30_gartner_diff_netapp_2014_2015

A 16% improvement???? That is absurd. It is as if the AFF had hardly any impact at all.

Reviewing Gartner’s Positioning of NetApp In the 2015 MQ for SSAs

It is one thing for NetApp to not be the leader in the MQ (and any assertion as such would be subjective). But let us be real – With all due respect for Pure Storage, is Pure as a vendor REALLY TWICE (over 2x) as “complete” as NetApp in the SSA market?

Before this article dives into the MQ statements on NetApp, it is first necessary to have a review of Design Centers. NOTE: This is more about establishing appropriate context for EF than AFF, but necessary as Gartner analyzes at the vendor level.

Design Centers and Product Positioning

Every product has a design center – and an architecture that results. This is a rule for all products, in all industries. However, IT has no shortage of belief that “no tradeoffs” is possible.

I like to illustrate the concept using vehicles, as shown below:

A vehicle has the attributes of performance, efficiency, and predictability (or robustness / durability).

The architecture (design) will determine how strong a product is with respect to each of those attributes. [ While it is possible to make something that is actually bad at everything, this article will not roam there..]

In the extreme case, shown by the three vehicles, there is a complete dedication to optimizing for one attribute at the cost of the other two.

  • The Russian BTR-80 (this one is modified to travel depths of water up to 5 feet for extended distances) is neither fuel-efficient or fast. However, it can handle the total loss of almost any two tires and still sustain virtually the same speed in the same elements.
  • The Rocket car (we could have chosen a top-fuel dragster) can reach astronomical speeds, but it is not fuel efficient, and has limited tolerance to part failures.
  • The lightweight solar-powered car is not extremely fast or durable, but given enough time it can cross any continent without using any fuel (or minimal fuel).

When a statement is made such as “a fast product, potentially the fastest in the industry, is flawed because it has less efficiency” … there is clearly an error in logic. THIS DRIVES ME NUTS!!!! The following dialogue captures this point…

2015_07_06_car_buying_cheaper_lambo

Review of Statements on NetApp in Gartner’s MQ for SSAs

I object to multiple NetApp statements in the Gartner 2015 MQ for SSAs:

Gartner MQ for SSAs Statement on NetApp Comments

The lack of data reduction capabilities limits the appeal of EF-Series in server virtualization, VDI and OLTP consolidation use cases.

and

The EF-Series remains the flagship product, and is focused on workloads that do not need data reduction capabilities.

Yes, there are no data reduction capabilities with EF, and this is what makes the EF special. It has an ultra-light threading model and brawny data layout that gives EF the lowest and most predictable IO latency under any load of any SSA product available. And this performance has been validated through the SPC.

EVERY WORKLOAD would love to have data reduction. The question should be: Is the demand of guaranteed latency worth the price for a given application??? NetApp offers choices with EF rocket performance… and AFF still-remarkable high-performance PLUS wide-ranging storage abstraction and excellent efficiency. This choice should be viewed favorably in Gartner’s SSA technology market.

There is significant overlap in NetApp’s solid-state array product portfolio, with three different products causing confusion among customers about sustainable innovation and long-term viability of each of these products.

and

FlashRay, which was announced in 2014 – but is still in limited availability as a single-controller array with several missing software functionalities – raises questions about the ability of NetApp to be competitive amid rapid innovation from competitors.

What is the confusion ? EF is for density, and for when low latency guarantees trump the value of the storage services abstractions of AFF. FlashRay is not shipping in large numbers and is typically not part of the equation at this time. Because its not HA, and the requirements “Have established notable market presence as demonstrated by the amount of PB” doesn’t apply to FlashRay, why does Gartner even care at this time ?

If Gartner is holding FlashRay as a negative against NetApp, then it would only be fair to also hold DSSD as a negative on EMC. DSSD is in extreme limited availability and incomplete, also announced well over a year ago by EMC.

[NetApp Strength :] Both the EF-Series and FAS series are mature products that have a large installed base, offering existing customers platform continuity and management familiarity. EF is based on Enginio, which is closing in on nearly one MILLION systems deployed over 30 years. FAS has over 100,000 active controllers, and ONTAP supports more bytes than any other Storage OS, period. Having the largest, most proven, enterprise install base should weight heavy on the positive – If this isn’t a “Leader”, what does that say about all of those customers?

How to Fix – So where SHOULD NetApp Be ?

There is close, and then there is not close enough.

I cannot assert exactly where NetApp should be. Once it is within 20%, the debate gets very technical, yet also very subjective. For example, is NetApp’s on-site support in the Falkland Islands a 8 vs a 7 for vendor ZYX? So making this easy… if NetApp is within the range shown below, it would at “feel” more right, and let me move on to not having to research research.

Why? Because AFF is..

  • One of the limited systems that is unified (NAS and SAN)
  • Enterprise Ready
  • Proven, benchmarked performance with the SPC
  • Scalable in multiple directions, including supporting more raw and usable capacity in a cluster than any other Gartner recognized SSA product
  • Integration of replication methods (no external VM appliances or hardware)
  • Extensive history of application integration
  • Virtualized and Cloud-integrated
  • A platform that has fully abstracted storage management, not a file-system box, based on a language and interface that has been established for years.
  • I have about a thousand more bullets here… you get the point…

… and because there is no “confusion” in NetApp’s flash strategy as the MQ for SSAs suggests.

… and because Gartner should understand that what makes EF special and unique cannot be a diminished as a product challenge, and NetApp’s portfolio which includes EF performance and AFF all-round versatility and storage abstraction cannot possibly be seen as “less than one half complete” as any other vendor.

Other Observations on Gartner’s MQ for SSA

As an analyst, I respect the work that Gartner does, and has no false illusion of somehow being capable of producing a superior report. Just the same, the other non-NetApp concerns over Gartner’s MQ for SSA are:

  1. Why are some Hybrid Arrays on it ?
  2. Gartner is not considering that many MQ for SSAs have tradeoffs due to their features – Tradeoffs that previously never existed…

Why are some Hybrid Arrays on the Gartner 2015 MQ for SSAs ?

Here is the Inclusion and Exclusion Criteria from the Garter All-Flash MQ:

To be included in the Magic Quadrant for SSAs, a vendor must:

  • Offer a self-contained, solid-state-only system that has a dedicated model name and model number (see Note 1).
  • Have a solid-state-only system. It must be initially sold with 100% solid-state technology and cannot be reconfigured, expanded or upgraded at any point with any form of HDD within expansion trays via any vendor’s special upgrade, specific customer customization or vendor product exclusion process into a hybrid or general-purpose SSD and HDD storage array.
  • Sell its product as a stand-alone product, without the requirement to bundle it with other vendors’ storage products in order to be implemented in production.
  • ..[The bullets RE geographic availability, support, Gartner interviews, etc, are not shown]

The SSAs evaluated in this research include scale-up, scale-out and unified storage architectures. [….]

Thus.. By Gartner’s own Definition of an SSA, EMC XtremIO is a Hybrid

Ready for some controversy?

calvin_argument

Like Nimble Storage CS Arrays, XtremIO should not be on the Gartner MQ for SSAs. This is according to Gartner’s own Inclusion rules.

It is well documented that each X-Brick contains two controllers, one DAE (Disk Array Enclosure), battery backup unit(s), and (possibly, depending on scale) Infiniband switches. According to http://blog.sina.com.cn/s/blog_53b574980101ckii.html, each XtremIO controller contains:

  • 2 * 8-core 2.1GHz Intel Xeon CPUs
  • 256G RAM
  • Customized linux, XIOS runs in user space on top of the Linux
  • 2 200G SSDs, one for boot partition, one for journal dump
  • 2 900G HDDs, store data path IO traces and logs

Look at evaluation criteria, and it is clear. EMC XtremIO uses HDDs, thus is it not an all solid state storage appliance.

Describing XtremIO as a Hybrid is NOT FUD

This is not a statement that XtremIO is therefore faster, or slower – Or any more or less capable.

Using HDDs as transaction logging devices is more than about just saving a few dollars. Those HDDs are probably faster with that streaming write IO profile, and these devices might have been a bottleneck if they were SSDs. I actually applaud EMC for using HDDs in this application if they helped, but just the same, EMC was a little “????” for not making sure Gartner knew this.

Suggestions for Gartner

Take XtremIO out of the Garner SSA technology market, or change the inclusion rules.

Gartner is not considering that many SSAs have tradeoffs due to their features – Tradeoffs that previously never existed…

WHAT ABOUT THE OPTION TO SELECTIVELY THICK PROVISION? How can enterprise storage actually support secure multi tenancy without the ability to thick provision and securely reserve capacity when needed ?

Sorry for the caps.. but this is a topic I’m passionate about. It is just another one of those configuration options that AFF has (thick or thin) that many others do not (thin only!).

Summary

Gartner continues to innovate with the Gartner MQ for SSA and keep with the times, and I’d like to thank Gartner for their efforts, and look forward to future reports.

Unfortunately, I believe Gartner has miscalculated on where to place NetApp. While it is nobody’s business other than Gartner’s to state exactly where NetApp should be in the MQ for SSAs, NetApp is so far away from the ballpark that the report as a whole misses its full potential.

Finally, some rules appear to have been used without balance or consistency. This is partially the reason for NetApp’s positioning in the 2015 MQ, and also a challenge as a hybrid system is present in the MQ. With so much hard work already done, these are easy fixes.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s