# Synthetic Monitoring
source: https://developer.mastercard.com/blog/synthetic-monitoring/index.md

Authors: Noel O'Sullivan. You've been working on your cool new service for months and now it's launch-time. You've told your friends, your colleagues and you've tried to create some buzz on the blogosphere. You've done your behavioral and scale testing and polished your UX, you've added logs and analytics to make sure users are engaged. You're excited, but paranoid - will your service work in the real world? For the first couple of months you nurse your new baby, logging in proactively and regularly to make sure it's as wonderful as it should be. Your friendly users log in too and report any problems to you. After a few months, things settle down, your priorities move on, your "real" users start using the service, and numbers fall back to a more organic level. This is the time when complacency creeps in. The last thing you want is to find out there's a breakage in your service by seeing an unhappy mention in your Twitter-feed. If that happens, the damage to your reputation is already done and you've already lost customers. So how do you get ahead of this? Synthetic Monitoring is part of the answer. Synthetic Monitoring is a set of disciplines that involves simulating real user-like (Synthetic) interactions against your production services and measuring the results (Monitoring). Rather than waiting for organic user interactions to give us a passive understanding of how the services are performing, we actively generate a representative set of interactions to assure the key services are operating correctly and with adequate performance. Why "Canary in the Coalmine"? This English idiom comes from the days when coal-miners would take a caged bird (usually a canary) with them into the mine. If there was a gas-leak, the bird would suffer/die before the miners, giving them time to escape. No birds need to be killed in Synthetic Monitoring - but we do get an early warning if things are going wrong. Getting started Don't focus on the tools This may seem counter-intuitive, but focussing too early on the tooling can lead you to lock-in your thinking (or even lock you into a vendor-solution). Instead make sure you have a clear vision of what you want to achieve, what you want to measure, and what constitutes success. Real user experience If your service is a REST API, this is relatively straightforward - pick your most important services and create test clients to exercise them. If it's a web-interface, things are more complicated as the "real user experience" involves much more than just pulling down the HTML \& Javascript that make up your website. You need to make sure that the page renders correctly at different sizes, that buttons are not hidden, that click actions actually work and that the expected behaviour functions. You need to execute in a real browser to make sure the DOM \& Javascript components render a usable page. Hopefully, you've already created Selenium or similar tests in your regression and performance testing suite; if so, it may be possible to promote a set of these to act as your Synthetic Monitors. Meaningful interaction What's your most important user flow? If you're running a commerce site, probably the single most important flow is enabling users to find a product, add it to a cart and complete a checkout. If this isn't functional and responsive, you lose sales - so start there and measure each step of the flow. Get outside your domain If you have a corporate network, don't measure from within; your customers don't access your service from within your network, so it's meaningless to measure from there. Either look at an external Software-as-a-Service provider or run your monitors in external virtual infrastructure. Be geographically relevant Where are your users based? There's less value in testing from servers in New York if your users are in New South Wales. It may also be unrealistic to run tests from a data-centre with massive bandwidth if your user community typically has lower-bandwidth. Look at rate limiting traffic to give a better measure of real-world user experience. The boring bits To make monitors useful, they need to be able to complete meaningful flows. That means getting a dummy account set up, suppressing the re-captcha, and anything else that would block the flow of the interaction. This can be tedious to arrange, but is critical. Monitor Frequency, Alerting \& Reporting Make monitors frequent enough to catch outages and problems before your users see them. Make sure you have alerts set up so you get the message when things are going badly. You or your team need to be able to react to problems quickly. Have accountable people receiving alerts so they can take quick action to restore service. Create reports against your capture step-times and observe them over time. Are things getting slower or faster? Are there times during the week when unexplained slowdowns occur? These give key insights into your live site - don't ignore them. Getting Technical The basics Writing your test scripts will be your first task. Take the time to learn Selenium as nearly all synthetic monitoring solutions base their syntax (or solution) on it. Pick a language binding you're comfortable with. Selenium has bindings for many languages and you may want to pick your solution based on which ones you're productive with. On the other hand, SM scripts are usually pretty straightforward, so if there's a compelling SM-service you want that meets the goals above, you may want to consider learning "just enough" Python/Node.js/Ruby - or whatever the vendor's language is - to get on-board. Designing testable websites All web scripting involve seeking an element on each page and performing actions on it. This means you'll have to get familiar with XPath or else make sure your pages are populated with unique 'id' attributes. Making pages with easily locatable, consistent tags is a great way of maintaining your sanity in general. When you're writing testing/monitoring scripts against them it's mandatory. Use name/id tags liberally and be careful not to change them as you upgrade your site. Avoiding vendor lock-in If you're planning on maintaining a larger scripting suite, consider a language abstraction for your scripts. It can be reasonably easy to write a lightweight framework to simplify your scripts even further and avoid locking yourself into a single vendor. By writing a simple wrapper to interpret code like the below, you can target it to multiple vendor solutions (or run it locally in development using Selenium directly). This keeps your scripting suite separate from the execution platform and means you can move from one to another while just changing the framework code. JSON { "name": "first-script", "country": "au", "description": \[ "Standard checkout for a book retailer" \], "execution": { "locations": \[ "sydney" \], "interval": 360 }, "steps": \[ { "action": "getPage", "value": "https://moesbooks.com/harry-potter", "comment": "Get started. Let's buy us some Harry Potter." }, { "action": "findAndClick", "locator": { "xpath": "//div\[@id='product-information'\]//div\[@class='buy-button'\]/a" }, "name": "Buy" }, { "action": "findAndClick", "locator": { "linkText": "View Shopping Cart" }, "name": "ViewShoppingCart" }, { "action": "findAndClick", "locator": { "xpath": "//div\[@class='checkout-buttons'\]/a\[@class='checkout-button'\]" }, "name": "Checkout" }, { "action": "findAndClick", "locator": { "xpath": "//a\[@class='review-order-button'\]" }, "scroll": true }, { "action": "findAndClick", "locator": { "id": "Confirm" }, "scroll": true } \] }