0

Background

Recently I did a major Drupal upgrade on a site which required a lot of code changes in my custom code and also some work with the caching layer in Drupal. After deploying the site to the stage environment everything seemed to take much longer than before. The stage environment adds a lot of extra complexity with varnish and memcache and other sites running in the same environment - as well as people from other departments adding more or less random pieces of external javascript code. The task at hand was to find a way of measuring page-load times, but with minimum interference from the environment and other infrastructure. Since we already have CI in place which runs some behat tests this seemed to be the best place to add performance testing.

Implementation

This approach is done in front of a Drupal background but I think it can be generally used in any project with behat and more widely in any project with CI or any automated testing. I took some inspiration from https://www.lambdatest.com/blog/how-to-measure-page-load-times-with-selenium/

We have a setup where we have Drone CI and run behat tests as part of the pipeline.

I have created a behat context PerformanceTimingContext for this. To use it in a composer based project:

composer require miiimooo/behat-tools:4.x

Then enable it in your behat configuration:

default:
  suites:
    default:
      contexts:
...
        - miiimooo\BehatTools\Context\PerformanceTimingContext

One caveat is that the performance timing is only available with browser tests (e.g. those that use the @javascript tag in behat)

A simple test looks like this:

# javascript.feature
# @codingStandardsIgnoreFile
Feature: Javascript

  @javascript
  Scenario: Ensure we have browser testing support
    Given I am not logged in
    And I am on "/"
    Then I should see the text "Home"

When running this there is now new output attached to each scenario:

  │
  │  Backend: 2147 / Frontend: 541
  │  
  │
  └─ @AfterScenario @javascript 

Since this is running as part of CI there is a history of performance and whenever there is a claim that the site is really slow or someone has improved something there is a place to look what has actually changed.

Caveats

The CI runs not just one project but many and performance can vary depending on how many other tasks are running.

In reality the numbers vary quite a bit from build to build. In Drone CI it's possible to re-run a previous commit as a new build (Restart build).

Todos

One major area of improvement is presentation. There is a slack notification plugin for Drone CI that is used for reporting failure/success in a dedicated channel. It would be useful to have the average performance of the whole test there as well, together with standard deviation.

  • Hi! Is there a question in here? It sounds like you're looking for some feedback on the strategy, but it's hard for me to see which part. Could you please give us some idea of what part of this testing strategy you're specifically asking about? – Bruce Becker Jul 3 at 8:58
  • @BruceBecker yeah good point - I didn't put any question. I wanted to open a discussion. Maybe someone else is trying to achieve something similar but has a different approach. Maybe there is something that already exists that you're using to achieve this. I haven't used the Performance Timings API. Maybe there are other ways. The referenced article was the first thing I found when I started searching. The implementation was so painless (so far) that I wonder why it hasn't been done before (probably has though) – miiimooo Jul 3 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.