Screenshot Testing
Screenshot testing is a form of automated user interface testing.
The user interface is controlled by a programmable browser (in case of web applications), programmable emulator or programmable device (in cases of mobile applications). It works by simulating real user-interactions as realistically as possible, and in the end taking a screenshot which can be compared to a screenshot that is known to be good. If the screenshots match, the test succeeds.
1 Pros
1.1 Easy to introduce
Screenshot tests can be used with any kind of code. It doesn’t matter how testable it is, since no direct interactions with the code are necessary.
1.2 Realistic Interactions
There’s probably nothing closer to having a real human test the SUT than using screenshot testing, as (most) humans interact with the typical application using the graphical interface.
2 Cons
2.1 Hard to maintain
Since pixel-differences can cause a test to fail (depending on the algorithm used to compare the good screenshots with the screenshots generated during a test-run) the tests tend to fail a lot when they shouldn’t.
2.1.1 How to fix
There’s more advanced algorithms that have offsets of differences, can determine whether a change actually looks different to the user. Some of these advanced algorithms use Machine Learning for better results.
2.2 Slow to execute
Since it is a takes a lot more processing power to run a full browser instance or emulator, and have it connect to the application instance than it is to simply execute some functions as would happen in for example unit-tests, screenshot tests tend to be slow (as end-to-end tests usually are).