Gartner’s view on database landscape (MQ)

Courtesy of NuoDB I was drawn to read Gartner’s updated magic quadrant report.

It’s sidenote includes an observation I have to agree with:

Through 2018, a wave of consolidation will affect the operational DBMS market’s smaller vendors, through mergers, acquisitions and business failures.

NuoDB, who I mentioned in my previous post, did well as a visionary, whilst Clustrix remained in the Niche quadrant. It’s important to remember that even making it into a Magic Quadrant means you’ve passed a startup hurdle: you need over $20M in revenue, and 100 customers, amongst other criteria.

I did see one database vendor I hadn’t come across before but which warrants investigation under certain use cases.

My earlier post was talking about how common it is for older businesses to be over-invested in Oracle or SQL Server without a NewSQL alternative being investigated. I was not looking at NoSQL or graph databases, which require refactoring the application, but ACID-compliant SQL-based alternatives.

That’s TmaxSoft software called Tibero, a South Korean firm. What I like from surface-level reading their content is:the compatibility with SQL Server (reducing migration complexity is a key thing to consider), and easy upgrades between versions (key for quality of life).

Another I hadn’t seen for some years is InterSystems, who have done well to get into the Leaders quadrant. They’re offering an interesting hybrid, and Gartner point out their high customer satisfaction and problem-free usage.

Also, NuoDB still stand out to me with their horizontally-scalable design.

So if it’s not already in the plans, I’d suggest IT leaders actively assess their database landscape; identifying workloads which are unnecessarily being placed onto tier 1 database infrastructure. This Gartner report also covers NoSQL and Graph options if you are refactoring or developing new applications.

Posted in geek | 9 Comments

The NewSQL alternatives

At Rackspace I have a number of business customers who are committed to a Microsoft architectural stack. Some of these are older businesses, not startups born with cloud tech choices nor with access to cloud-era talent. Their stacks include SQL Server exclusively, and it has been part of their world forever. Recently as I’ve been talking with their IT leadership I’ve been particularly curious to ask about their work around cost architecture, and this often leads to asking if their view of workload on their database.

I thought it’d be interesting to reflect on these traditional businesses, and some of their challenges.

For example, there’s a common situation where say a third of a customer’s infrastructure compute is consumed by SQL, with a traditional architecture where the application assumes high-availability from the layers beneath it. Their app has little message queuing or service-orientation. The consequence of their architecture is a requirement for very solid infrastructure.

I’ve been talking to enterprise IT folk about the complications and process of migrating services to the cloud for about four years. Whilst all of us want to see service-oriented IT more widely-deployed, the reality is that this modernisation takes time.

So whilst a business’ IT might be a while away from being a distributed system, it seems sensible to at look at the compute offload, cost reduction and scalability offered by NewSQL.

Migrating to NuoDB or Clustrix can be a much simpler proposition than refactoring to use NoSQL. Existing MS SQL Server databases can be ported, with complexity depending on the usual factors (extensions, size of code base, coupling). It is worth a proof of concept to assess the change effort and benefits.

Posted in observations | Leave a comment

How cloud-oriented is the app?

RightScale lets you easily deploy an application to a cloud provider like AWS or Rackspace. We do this by abstracting your server infrastructure design to a layer above that cloud provider. Consequently, I often have conversations about whether an existing application is a suitable candidate for the cloud. The less common question, but of great significance, is whether an application can really exploit the capabilities of the cloud.

I recently developed a little framework which helps to explain some of the dimensions which effect how well an application can go beyond a forklifted application towards one which is architected for the cloud.

What’s a cloud-oriented app?

It’s fairly well understood amongst web-based businesses who use the cloud, that you have to design for failure. You have to assume any element of your infrastructure or application stack could fail. Perhaps Netflix made this most famous with their chaos monkeys who deliberately destroy service elements, so Netflix could test what happened and make sure the system was resilient. At reInvent last year, it was thematic in key presentations from AWS. This is no secret.

Applications which are designed with a service-oriented architecture should gracefully degrade if a service on which it depends are no longer available. A simple example is if an application presents a menu choice, and that menu is derived from a database, but that database is not there any longer – it’s failed for whatever reason – the application needs to keep working anyhow.

In traditional enterprise data centres, the design assumption applications can make is that the hardware environment will be available 99.5% of the time or more. Core systems will expect 99.9%, and so on. The application at the top assumes all its layers underneath are working, and generally does not fail gracefully.

Cloud applications dismiss that availability assumption, and not because the cloud is necessarily less reliable, but because it makes for a more resilient and scalable application in any case, and because the cloud is a distributed system.

Cloud apps can scale horizontally (more servers), instead of vertically (CPU). Doing this requires the constituent parts are able to scale that way.

What are some non-cloudy ways?

So, some applications expect to find on a local file system, or store sessions in a low-throughput database. If the app is doing that, it’s not very cloud-oriented. That’s the “data location: actual server” column of this diagram.

I itemised a few technical elements which can be done in a cloudy way, or not:

  • user data,
  • session management,
  • infrastructure config and
  • application configuration.

Some web-based applications like Drupal, phpBB and Concrete CMS store their application configuration and user data in ways is not compatible with horizontal scalability. They need some adaption to use GlusterFS, for example, as a distributed file system, and do not have public reference architectures for putting them into a horizontally-scaling environment.

I’ve mapped out those elements against the continuum from least to the most cloud-oriented method (the solutions I’ve listed are not exhaustive).

how cloud oriented is the app?

For horizontal scaling, ideally the application needs to be installed unattended in a scripted fashion. (So, you can’t do the next/next wizard of phpBB, for example). This in turn means the configuration of the application needs to be defined and stored in a database or file which is accessible to the installer script, and then you have questions around version control and persistence of that configuration information.

People and process change management also needs to be considered: if the user edits a configuration on a local file, but doesn’t push that configuration back to the SVN, it will not persist when the server is rebuilt next time.

Some apps go easily to the cloud

Some applications will adapt well to a horizontally-scaling cloud environment, others will not.

Those which do not scale horizontally may still be suitable for deployment to cloud, but in a forklifted fashion. You’d retire the existing server and move the application to an equivalent cloud server, which is most likely cheaper. The application may not like to be turned off at night to save you money. If it can, though, then your financial savings increase markedly. Turning off servers is one key to unlocking cloud economics. Scaling horizontally is another key.

In either case, it’s a lot more intellectually fulfilling and interesting that staying with the status quo.

Posted in geek | 3 Comments

Two worlds of cloud

Increasingly it’s clear to me that there are two worlds of cloud users, the populations of which often do not realise they are world’s apart. The first is inhabited by traditional enterprise IT users. Extending the analogy, it has more terra firma than clouds.

The second has a population of cloud-first companies who use cloud-era technologies. They largely do not use Windows, SQL Server, WebSphere or similar packaged software, and instead their first choice is a SaaS or Open Source equivalent like Debian, MariaDB, Couchbase, django or a plethora of technologies and languages.

The world of enterprise IT has a density of applications, systems, hardware and processes which create a huge gravitational force towards status quo. Change is costly and risky. Interestingly, the perspective of enterprise IT folk actually distorts how clearly they can see what cloud is, and can offer. More on that later.

New cloud technologists are often fairly crap at risk management and change control until they’ve either hit some kind of scale (which brings the need for change control and rigour) or failed in some way. There’s often a religious rejection of anything that sounds like waterfall method (like planning), and a sprint to the next scrum instead.

I could have expressed this idea as a continuum with enterprise IT represented by banking at the most extreme on one end, and a seed-funded fledging internet startup with a few clients at the other end. One has great traceability and stability, the other has great spontaneity and agility.

James Staten from Forrester used a nice diagram which expressed how enterprise IT guys see cloud differently to cloud-era folk. The essential difference is that enterprise guys see cloud as the next logical evolution of virtualisation, whereas cloud developers see it as a programmable service. He comments on how this leads to private cloud implementations failing because the enterprise guys make it robust and highly change-controlled, whereas the developers just want an API that abstracts all those details away so they can launch a server in 5 minutes.

Google the paper “Rise of the new cloud admin” if you’re not a client, and read it. It’s absolutely spot-on awesome.

Result is, in the real world, that an enterprise decision maker will look at his sunk investment in a VBlock environment and ask me if we can make it into a cloud. Sure. But that question only gets you to first base.

You can run a private cloud on enterprise-grade hardware, but you don’t have to of course. Let’s assume you do though, because you want more HA on the private cloud, because that’s the norm. Then the usual first step is to make it easier for test and dev workloads — so your application teams can launch a base server really easily to test a new version of an application. Making that happen with a self-service portal isn’t hard either.

Here’s the catch though. Enterprise users are then putting traditional packaged applications into a private cloud, managed by ITIL, subject to change control and normal InfoSec policies. So you get a non-agile result in some dimensions. The dimension you have made cloudy is provisioning the hardware and operating system layers of the virtual machine, using a self-service portal.

Often, the installation of that enterprise application is not automated. If it is, it’s not multi-cloud portable, nor version-controlled so it can be elegantly moved through a lifecycle of test, staging and production.

The difference in the cloud world is that the application would have been defined and installed using configuration scripting. The application gets considered first, the hardware and operating system are almost assumed.

The interesting challenge for enterprises is to simultaneously:

  • look for enterprise applications they can move to the cloud,
  • identify applications they can migrate to cloud-era technologies, particularly considering NoSQL or graph databases, or cloud-friendly horizontally-scaling frameworks like django
  • identify applications or systems where configuration-based infrastructure definition is possible (RightScale uses Puppet or Chef), instead of virtual machine images which are managed under change control.
Posted in observations | Leave a comment