Testing at MACH Speed: C is for Cloud

C is for Cloud, or how everything is the same and different at the same time! Part three of this blog series builds off of our discussion on testing APIs.

Cloud Atlas is the title of David Mitchell’s amazing book of interconnected lives that span both centuries and literal styles. Seriously, if you are a book lover, you need to read it (but after this blog post, please). The book’s Wikipedia entry said the title is a reference to “a changing landscape (a “cloud”) over manifestations of fixed human nature (the atlas).”

I keep going back to that reference when I think about testing applications in the cloud; the changing landscape is our shift from servers being these fixed assets that we kept on-premise to moving to cloud data centers with hundreds, if not thousands, of virtual servers and containers running on a single rack. While this changes the nature of testing, the fixed nature of the atlas comes back to how we still need to test them.

There are some really great books on how to test cloud services, but I’m going to focus on two areas for the rest of this article:

1. A few key differences when testing applications in the cloud vs. testing on-premise
2. How good change management habits will save you time and reduce your team’s frustration when testing applications in the cloud

When moving to the cloud, let’s first get something out of the way; the saying “the more things change, the more they stay the same” is so true when testing applications in the cloud.

I’ve had people argue with me that moving to the cloud means you no longer have to consider things in your testing, like server/system configuration differences and server capacity in your test plan.

The reasoning goes that, after all, you’ll just copy the configuration from one cloud server to another as you spin up another instance in ten minutes because the load is too high. I mean, it’s not like you have to order another AS/400 and wait for it to be added to your data center.

And while both of these things can be done, you still need to test to see if the configurations are applied correctly. Also, the concept of reducing cost by using lower-capacity servers in the test environment has carried over to the cloud. There, it has been joined by shutting down servers after business hours, as most cloud providers offer a pay-as-you-use model. I once ran up a $500 AWS bill by leaving a load test running overnight… I was lucky that our AWS rep owed my boss a favor.

The key difference here is that you will need to map your test environment as well as your production environment as if it were on-premise. As one of my testing mentors once said, “It may just be QA to you, but it’s my team’s production.” So treat it as such for your QA team.

Here are a couple of things to add to your environment map:

• Percentage difference between the QA and Production environments (if any)
• What systems will be connected to the QA environment vs. which systems will only be connected in production; never assume a system that exists in production will also be connected to the QA environment
• What times will the QA environment be available for testing; this is critical if members of the QA team are working in a different time zone
• If you have batch jobs that move data between the systems, confirm what jobs will be running in production vs. those running in QA

My “if you follow only one best practice, this is it” would be to use a change “train” for easier troubleshooting while testing. A change train is a simple document or spreadsheet that the whole team can see and update that will quickly tell you:

• What is the change?
• Where (the environment) was the change made?
• When was the change made?
• Who made the change; really good ones will also tell you how to undo the change if needed.

It’s tempting to just use tools like Slack and Teams channel to call out changes, but I would only do this if you really like searching for this kind of information in the channel threads when trying to find out why a test is failing.

Just because your application has moved to the cloud doesn’t mean all the best practices you’ve built up go away. Instead, look at your cloud environment as something that buys you both time and flexibility that you still have to test.

One more letter to go, which is where you’re going to lose your head… that’s right, the next and final article in this blog series is about testing Headless.