The web is full of third-party scripts. Sites use them for ads, analytics, retargeting, and more. But this isn’t always the whole story. Scripts are unpredictable, they execute code, but you don’t know what this code actually does.
With Taboola’s advertising video player solution, we struggle with 3rd party scripts daily.
Working with different advertisers has exposed us to a variety of malicious behavior: sound violations, auto scroll and change page DOM are just some of them.
In this post we’ll take a closer look at how we detect sound violations.
Steer clear of 3rd party script risks
A sound violation is a state in which the video plays sound without user interaction. Several advertisers do this, to ensure that the user will notice their ads. We struggled with this often and received lots of complaints from publishers.
Sound violations should be prevented by the video player, but often aren’t. The main challenge is to be the first to detect them.
The passive approach is to wait for the publisher to complain. A different approach is to leave your computer and speaker on, and hope to hear sound from video. We chose the proactive approach – be the first to discover your “weak spots”.
We decided to develop a framework to test and detect malicious advertisers’ behavior. Our first goal was to detect sound violations before the publisher.
Why Selenium is not your best friend
Our first framework was based on Selenium, the most popular automation tool in the industry, which is also used in Taboola.
Yet, Selenium did not suit our needs. Let me explain why…
Selenium was running tests on Chrome 46, while Chrome 60 was the latest version.
Also, Selenium was not intuitive – each test was a struggle.
Finally, we had issues with installing Selenium on both Mac and Windows.
All you need is Node
Our conclusion was that Selenium did not suit our needs. So, we decided to characterize and develop a testing framework on our own, and chose to base our solution on Node.js.
Why Node? With the help of some Node libraries, any browser can be controlled and debug. We use chrome-remote-interface, child_process, sendmail, and more. All installed using npm and regularly maintained.
Be proactive – DIY
The first mission was to inspect and debug the most popular browsers, CDP (Chrome DevTools Protocol) is the best solution for debugging.
CDP is an API for both Chrome and Safari. It’s installed using NPM, intuitive and easy to maintain. CDP allows you to control every browser aspect – DOM inspecting, network monitoring, load extensions, are examples of CDP’s abilities. It enables you to share user experience, and improve it as much as possible.
The second phase was deciding on the testing environment. Remote iOS debugging is not available on Windows, so we decided on Mac Mini as our debugging machine. We needed to install Chrome, Xcode, and a real Android mobile device. By executing command line scripts, we could control browsers on all platforms.
The last step was remote launching our testing environment. We set Mac Mini to act as a TeamCity agent. We created a project on TeamCity, which runs every day and operates the agent.
In the end, this is what the framework looks like:
Sound violation fighters
Now comes the best part, we were ready to develop a test to detect sound violations.
CDP does not provide info about sound playing in tab. Luckily enough, sound can be detected by Chrome extensions. Developing an extension, which listens to changes in a page’s audible state, is very easy. The extension will trigger an event when the ad’s sound is changed from mute to unmute.
Here is a simple example of an extension that detects sound playing tabs:
Once we were done with the test, we were able to run ~1000 web pages everyday. Playing different videos from a variety of advertisers, sampling for sound violation. When the sampling is finished, team members receive a full report about exposed sound violations.
The sound of silence
We are able to stay one step ahead of malicious 3rd party scripts, and overcome issues before our publishers find out about them.
We have since developed more tests like this, and we can track more data about advertisers’ behaviors.
As a result of being proactive, we have been able to catch almost every sound violation occurring in our video player. In the beginning we caught around 10 ads in every sample. Now, we have decreased that to 1 or none per sample.
Today we can also watch network activity and trace large video files, and we try to decrease page load time and CPU usage.
How can everyone benefit from a proactive framework?
You can use it for E2E and feature tests. You have all major browsers connected, with the ability to test them. Write some simple tests, throw in some case studies, and you have full coverage of your code using only Node.
Currently we use it for E2E and sanity tests on Chrome, iOS and Android – it has reduced QA test time for new versions by half.
Being proactive and developing a customized framework has made our work life much easier. We improved our user experience and relationships with partners. We are able to develop a bug-free (as much as possible, we are human after all) advertising video player.
Being proactive has placed us as one of the best video players in the industry.