Many hosting providers automatically monitor your homepage and will notify you when the page fails to load. This is a good first step, but even if the page loads successfully, it might display incorrectly, whether it’s missing styles, menus, content, or images.
This is where visual testing is key. Visual testing compares a screenshot of the current web page with a reference screenshot to catch display issues. There are several paid services that can run these comparisons and notify you when changes occur—but what if you wanted to Do It Yourself?
How to DIY Automated Visual Testing
Our solution consists of four parts:
- Playwright test
- Server scripting
- cron
- Slack App Interaction
Playwright is the core of this implementation, as it handles the actual screenshot and comparison. Playwright has multiple SDKs including Node.js, Python, and Java. We will be using Node.js. Here is an example Playwright test:
import { test, expect } from '@playwright/test';
test.describe('Website Stability Tests', () => {
test(`home page visual test`, async ({ page }) => {
await page.goto(TEST_URL);
await expect(page).toHaveScreenshot({
maxDiffPixels: 500,
mask: [page.location(MASK_SELECTOR)]
});
});
});
Note how we can allow for some slight variation with maxDiffPixels, and we can also use the mask option to skip elements we don’t want to include in the comparison, such as elements with a background video.
Then we need to run this test, and notify Slack if it fails. Here is an example script:
const { exec } = require('child_process');
const { WebClient } = require('@slack/web-api');
const slack = new WebClient(SLACK_TOKEN);
exec(`npx playwright test --reporter=json`, null, (error, stdout, stderr) => {
const report = JSON.parse(stdout);
const status = report.suites[0].specs[0].tests[0].result.status;
if (status !== 'failed') {
return;
}
await slack.chat.postMessage({
channel: CHANNEL_ID,
text: `Test Failed`,
blocks: [
{
type: 'actions',
elements: [
{
type: 'button',
text: 'Update Reference',
action_id: 'update_reference',
style: 'primary'
}
]
}
],
});
});
We can run this script periodically, e.g. once a day, with cron, so that we are notified of site changes.
After we get this notice in Slack and check the site, if the update is a legitimate update, we can click the Update Reference button. This will send a request to the Request URL we configured in the Slack App. Here we can run playwright with the “–update-snapshots” flag to update the reference screenshot, so that we don’t get notified again.
Conclusion
Monitoring your websites with automated screenshots is a great way to ensure you catch issues quickly. Hopefully, this will give you some ideas about how to set up monitoring for your own sites.
