No-one really likes test environments. It's a not-so-disputed fact. Developers don't get access to them often enough, QAs complain (rightfully) that they don't replicate the production environment or hold the wrong data, Operations hasn't got the time to maintain them, as production issues are always prioritized (as they should). In Ops, keeping the SLAs come first.
A typical case can be when an issue is found in the production system, and the test environment holds the development code for the current sprint release already, so it can't be used for debugging the production issues. So at 2:30 AM, you need to find someone to roll back the test environment to the last deployed release.
Or if someone created duplicate entries in the SoR, since the test suite hadn't hooked the clean-up process, So now the automated tests won't run until Ops has the time to restore the database with the baseline data. But they are busy with important Production issues. So you just have to wait...
At some point, someone wrote temporary code to mock an external service, just so the unit tests would pass and the CI screen would show green. It's not quite clear who is responsible to make sure that the mock is up-to-date, and actually matches the integration the code artifact will actually use once it's deployed to production. But everyone is happy as long as the CI screen shows green!
The platform engineers have no means to modify the mock integrations in the test environment, and just have to assume that the developers are keeping the service up-to-date with the production APIs, when they are changed by the provider.
Mocks can be quite useful for the developer. You send a request, and get a response back right away, and your unit tests will pass. But the one thing you can't test with mocks is how the application performs when the integration point doesn't respond right away.What if the response time is twice as long as normal? Or five times as long? Or even more? How does your application handle this non-ideal situation, and how do you test for it?
Can you work with test data which comes directly from the production environment, maybe with a wash of the data to remove personal information? Then you are lucky! In many regulated business you are not allowed to move data from a production system into the test system. This means no nightly snapshots of the database loaded into the test environment. Can your organization handle a quality assurance scenario where test data cannot be snapped from the production system?
It seemed seem cheap, just $0.02 for a transaction against the payment provider's sandbox system. Then you run a casual load test with 10M transactions, just to make sure the site will withstand the expected peak load during the sport event, where your company has invested in targeted ads. A few days later, an administrator from the finance department comes to ask you about this $200k invoice they got...
You're a global enterprise, with a single assessment center, where new and hopeful candidates are put to the test. It's a very selective job opportunity, so you fly in candidates from different parts of the continent. When they arrive it becomes clear that your assessment environment is not ready yet, as the Operations team, which is responsible for loading the correct data into the environment has been busy with a critical issue in the production environment. They have been working around the clock the whole weekend, setting the mundane task of restoring assessment environments aside - as production SLAs come first. .
Well, basically yes. But as with all technological concepts, it's a matter of how you implement it. Service Virtualization is not a turn-key concept which will solve all your Test Data Management problems by just implementing it, but it's rather a very competent resource in your toolbox - and it can solve issues, like the ones previously mentioned.
In the next article, we will go through the Key Concepts of Service Virtualization - how it works, which services you can virtualize, and more.