Writing features as added chunks into an ever growing one bulk of code is unorganized and messy. Overtime, the tasks of testing new behaviors becomes harder and harder. Why is that?
Chunky Code is hard to Navigate
When working with developers on a new feature, we have to identify what we need to test. As the code grows, the challenge of finding the parts that were modified and should be tested is increasing. If a feature becomes problematic or irrelevant, reverting it becomes more difficult, since we need to go back to every line of code we changed. This way we endanger production environment and are affecting our end-users. Last but not least, it is irritating to impossible to manage when dealing with “legacy” code, that needs to be reverse engineered to find how to control.
Coming to compare a buggy behavior with its intended fix or testing a new feature, we need to have two abilities – reproducing the issue and verifying the fix. It becomes even more challenging when the change in the code is based on an existing feature that was overridden or overwritten.
It may require to deploy two versions of the product, an old and a new one, to have a clean comparison of the tests.
Recalculating Route – Feature Flags to the Rescue
To avoid building a lot of product versions for QA environment, we need a way to switch features on and off. Why is that so important?
In such a way, the change in behavior, can be attributed to the feature being switched on. This way we can be sure the fix was correctly implemented.
A great way to deal with this issue is feature flags. Let me explain why…
It lets you breakdown your product into a set of controlled features you can simply switch on and off. You can set parameters between possible values in a simple input fashion, giving the user the ability to enable or disable new features. It gives us a way to be more flexible and easily tweak our behaviors.
With feature flags, we can provide a sense of isolation to each product feature, and an ability to “divide and conquer” each feature. In a sense, it segments the code into seemingly structured individual components.
A good rule of thumb is to set up a new flag for each feature, which will allow us to turn it on or off. This helps the QA (or any other user) to turn the feature on and off appropriately when coming to perform the necessary tests in order to verify the feature before publishing it. It also provides us a safe way to AB test the feature as an individual change.
After testing if the feature is compatible with the required behavior, the flag can now be cancelled and the input value we tested can become the default setting. Another great advantage is that now we can easily remove the feature without risking production environment, by simply turning the feature off, if we found it does not work as expected or harm our users.
One of the greatest profits from using feature flags is developing features based on customer requests. Building the feature with a configurable flag allows us to set the feature with one behavior for this customer, and will provide us with a safe way to set it up differently for other customers. We can also easily turn it off for the customer, if they were not satisfied with the final result.
A great example is our flag for auto-playing video ads.
As Taboola’s video content platform, we want the publisher to have the ability to decide if they want the video to start playing as soon as the video ad unit comes into view by the user, or if playing is delayed, waiting until the user clicks it. Enabling autoplay can benefit the publisher by getting higher viewability rate, but we want the publisher to have the ability to decide on the user’s video experience.
“Fun With Flags” saved us from ourselves
New features are being developed all the time, each feature with its own unique control flag. Its makes our tests easier, but how you can control so many flags?
Overtime, we have developed over 1,000 flags. We can’t expect our developers or QA testers to remember all of them by heart.
We spend a lot of time naming our flags. The name may be a long phrase, describing exactly what the feature does, or an abbreviation of the description. Most importantly, flags should be well documented.
In order to make our lives even easier, we needed a way to be able to search for a specific flag without constantly asking the developers or looking for old tickets.
Using a flags bank, or “Fun with Flags” as we call it, can come in handy. We developed a web page listing all our flags by product version. Our system is an automated data bank tool that scans the code and maps and aggregates flags together for our video platform.
For each flag, we provide its full name, description, value type and default value. Now you don’t have to remember any flag, you can simply type any keyword and find the relevant flag you need to use.
This flag bank lets us search for flags for certain features. We can easily find a flag we want to retest, or a flag we want to play with in order to see different aspects of the product. We can see how it behaves on different platforms and if the feature “plays” well with other features. We can monitor different combinations of flags and see how they behave together on QA and production environments, and choose the winning ones.
We have a lot of benefits from our “Fun with Flags” system. We can check the frequency of a feature use, from live data, to monitor when a flag was last changed or used. If a flag wasn’t changed in a long time, maybe it should be deprecated, since it is no longer relevant, and the value can be set as a standard hard coded value.
Another benefit is when mentoring new QA teammates. Enabling them swim alone in the deep water with the sharks, like the big boys, is quicker. Letting them use the “Fun with Flags” system on their own reduces the time they waist, by asking questions or waiting for help from more experienced teammates. It is much simpler and faster if they learn the ropes of video configurations on their own using “Fun with Flags”.
The Grass is Now Greener with our Fun with Flags
The struggle of controlling features is the never ending battle of QA. We always want to be able to reproduce different product behaviors on various locations. Being able to control behaviors was not easy in the past, when all the settings were hard coded.We started using feature flags to set features’ enablement and separation with a simple flick of a switch.
Before our “Fun with Flags” system was developed, QA testers had to chase and wait for each feature developer to modify the code each time to compare a bug to its fix. That could take from hours, when the developer was busy, to days, when the developer is on vacation.
After we started using feature flags, times were shortened considerably, but could still take a long time For each feature, we should approach the developer, or someone else from the same team, and ask them which flag we should configure.
When “Fun with Flags” system was developed, we QA testers were able to lookup the needed flags with ease, within seconds. It allowed us to work independently and individually, without bothering developers or wasting precious time.
“Fun with Flags” saved us from chasing our tails, going in circles, and we now run smoothly in a straight line ahead, always looking forward.
