Automating Testing with Python & Google Analytics

It happens, more than it should and it’s annoying…..web developers inadvertently break tracking while developing websites.

The problem is – the earliest sign you have that things have gone south is 24 hours later the revenue or conversions look unnaturally out of trend. By this point, you’ve lost a bunch of data, Year on Year trend is screwed and you’re adding annotations to Google Analytics to explain what happened in the hope that someone looks at the annotation next year when doing their Year on Year comparisons.

If you’re working on a site that has ongoing development, this type of problem can seem to happen almost once a month.

So what can be done?

Alerts that send an email if the trend changes significantly can help but this only works when a significant amount of data has already been lost.

Using the Selenium Module for Python it is relatively easy to create an automated test for testing a transaction or conversion.

Selenium makes it easy for you to automate browser activity i.e. find this field, enter this value and click the submit button. In short, it can be used to replicate exactly what you would do when testing an ecommerce transaction which I will briefly detail below:

Psuedo Steps that can be translated into Python using Selenium

  1. Add product to cart
  2. Navigate to checkout
  3. Enter delivery details
  4. Enter payment details
  5. Confirm and submit transaction
  6. Wait 10 mins
  7. Login to Google Analytics
  8. Navigate to ecommerce transactions for today
  9. Check for ecommerce transaction
  10. Send email if transaction does not appear in Google Analytics

The script can then be set to run as a scheduled task on windows or a Linux server.

The only problem with this method of automated testing is that by running the tests you create inflated transactions and revenue in Google Analytics. It’s possible to reduce this somewhat by applying a coupon specifically for testing that discounts the transaction revenue to zero but you will still have inflated transactions recorded in Google Analytics; this may or may not be a problem depending on how many transactions your site has in a day or how often you run the test.

Of course, one solution is to ensure all development work is done on a staging site and that before any changes are published to the live site the tracking has been tested thoroughly. While this is good in theory it seems most developers make changes to live sites and if they do work on a staging site they do not test any tracking.

Interestingly while setting up an automated test script for a client I began to notice that sometimes that python script would fail because pages were taking too long to load, usually caused by Javascript files making third-party calls – these side effect issues are also worth monitoring i.e. if a page is slow when being testing by Python it will also be slow for the user.

A good resource for getting started is this chapter in the free Python resource Automating the boring stuff:

https://automatetheboringstuff.com/chapter11/

If anyone has any clever tips on how you can do this level of automated testing without inflating Google Analytics conversion metrics then please let me know!

 

Author: Kevin Thomson

Head of Analytics & Measurement at Attacat

Leave a Reply

Your email address will not be published. Required fields are marked *