Synthetic vs Real User Monitoring : What to use when?
Synthetic vs Real User Monitoring : What to use when?
Site speed tracking tools can be divided into two categories - Synthetic Testing and Real User Monitoring. Both of these are different kinds of monitoring and help achieve different goals. So, the question isn't which one of these is better but which one of these is more suited to your requirement. This requires you to know what exactly you want to know from the tool so that you can choose one over the other or both:
| Synthetic Testing | Real User Monitoring |
| Helps identify changes or regression in site speed. | Helps know the actual speed experienced by users. |
| Can detect even small changes in site speed. | Small changes in speed go undetected due to large number of external factors at play. |
| An observed behavior can be easily reproduced because of the controlled environment. | An observed behavior is difficult to reproduce due to multiple external factors at play at any given time. |
| Need to define and setup the use cases to test. | Does not require defining any scenarios - it covers every scenario that real user performs. |
| May not reveal some issues because it cannot test all the possible combinations of pages, devices, geo-locations, network conditions and load profile. | Reveals issues that actual users experience. |
| Does not require any changes within the site to setup. | Requires insertion of Javascript tracking snippet to enable tracking. |
| Provides additional metrics (like Speed Index) and diagnostic information (waterfall charts) to drill-down issues. | Does not provide diagnostic information to ensure it remains lightweight and doesn't affect user's speed experience. |
| Can be used in pre-production environments to test prior to release. | Cannot be leveraged unless the website is in production. |
| Helps making sure changes to website do not cause regression in speed. | Helps uncover usage pattern (geography, devices, load, network conditions) to identify key areas for optimization. |
So, if you are trying to determine where to focus for site speed optimizations - you should look at your website's RUM numbers. By knowing how your real users experience your site, you can determine where should you first look for site speed optimizations. And, as you optimize your site, you shall setup synthetic monitoring to make sure you are able to see the gains in site speed. Once optimized, you shall continue to monitor your site's speed with synthetic monitoring to make sure the gains are not lost during future changes.
