HOW WE MADE MANUAL TESTING OF 590 TIMEZONES IN 2 DAYS POSSIBLE
As a tester I often get to test functionalities in which the approach to the subject plays a key role. In a previous article, I have already written about how to approach manual testing on mobile devices generally. But here I would like to talk about one of the most interesting functionalities I have had an opportunity to test. Namely the testing of advertisements in different time zones.
The assumption of the functionality was that a banner placed on an advertisement on a given day and time could be displayed anywhere in the world, making it possible to promote movie premieres or sales promotions. The creator of the advertisement would be able to set the day and time strictly in the “User time zone” but would also be able to enter the time zone manually.
Why 590 timezones?
Initially, the programmers wanted to use UTC to divide it into and cover 24 time zones. Then all the creator would have to do, is simply enter the country and the program would automatically add the time zone.
However, after initial tests, it turned out that this solution has one main disadvantage, namely, there is no provision for summer and wintertime, which change in virtually every country.
After consultations, they decided that the best solution would be to use the moment.js library, which is based on times in Wikipedia. With this solution, the user could enter any destination region into the field regardless of the current time. The most up-to-date solution was divided into 590 regions (TZ database).
What devices did we have to check?
This functionality was supposed to work on both desktop and Android devices from 5.1.1 version to the newest (it’s now 10.0) and for iOS from 9.3.5 to the newest (13). There were 13 versions of both systems to cover and 590 possibilities to check. I and a second tester had a maximum of just two days to plan and conduct the tests. So we had to come up with a manual testing approach that would give us sufficient test coverage but in this very limited time.
How to approach manual testing when you have 590 Time zones and 2 days
Finding common ground
We started by listing all 590 zones and finding common features, i.e. the same UTC. For example, times in Europe / Astrakhan, Europe / Samara, Europe / Saratov, Asia / Dubai, Asia / Muscat, Asia / Tbilisi, Asia / Yerevan all have UTC +04: 00 in summer and wintertime so there would only have to be one test for all of these regions on all of the devices. After these merges, according to the UTC, there were 54 zones to check. However, 54 zones is still a lot, so we had to find another elimination factor.
Priority
The next step was to take all of the 54 available zones and check which ones have high priority, which medium and which did not need to be tested at all.
High importance in our case meant that the timezone was important from a business point of view. For example, the zone in which a large part of Europe is located, i.e. +01: 00 in winter and +02: 00 in summer. Medium priority was given to those that were less significant for business or less used, such as the zones in which the Azores or Egypt are located in. But also parts of Australia, for example, because Australia has 5 different time zones and some were more important than others. Zones on smaller islands, practically uninhabited, such as the zone in which Fakaofo is located, i.e. an archipelago from a group of coral islands with just 265 inhabitants, were also considered not important from a business point of view.
After counting and analyzing the highest priority zones, we managed to reduce it to 25 must-haves and 11 nice-to-haves that would also be checked eventually if time allowed. You should also remember that some of these zones would need to be divided into summer and wintertime as well. So let’s keep that in mind.
How to change time zones on devices. Preparing for the manual testing
Each of the devices being tested had to have a different zone with each test. The largest available city was selected from each time zone, and the times were changed based on that.
Changing the time zone on a specific device is not difficult, you just need to know where to look. I will show how to do it on macOS, iPhone, and Android devices (in this example Sony Xperia).
First device: MacBook
To change the time zone and Date & Time we have to press the date in the upper right-hand corner. Then press “Open Date & Time Preferences” and then go through 4 steps:
1. Unlock the possibility of changes
2. Turn off automatic date updating
3. After going to the Time Zone tab, turn off the ability to automatically load locations
4. Choose the location around the city or by clicking on the map
Second device: iPhone
On the iPhone we have to go to Settings -> General -> Date & Time and disable the option “Set Automatically”. Now we can change the time zone by entering the city name and change the date and time.
It is important that when you want to test the application on iOS, you need to first download the applications on the phone (from Xcode). Then change the date/time zone because in the reverse order you will not be able to upload the application in the future date (on Android there is no such dependence).
Third device: Sony Xperia
On a Sony Xperia phone (there are no major differences on other Android devices) we go to Settings. Then we go to Date & Time and disable the automatic date and time zone settings.
Test cases
After preparing the test environment and time zones, we could then start on the test cases. After going through the simplest cases of playing advertisements at a given date and time, we also had to check whether everything was working properly in any given summer and winter time.
A good way to do this was to check the wintertime by setting the date in December or November. Because in October the time changes around the world, but it is worth giving yourself a safety buffer. And then test summertime at the end of May.
It’s also worth checking testing zones at the end of the day. This is to check the disappearance of the banner at 23:59 and its appearance at 00:01 the next day. The disappearance of the banner and appearance in two weeks, only on certain days or vice versa was also checked.
Some of the more interesting issues that came up during the whole manual testing process were:
- when we changed the orientation, the banner did not appear.
- Another thing was that when a given advertisement was fired by the user, the hour was not read in real-time. For example the banner was to appear at 17:00 and the user started displaying the advertisement at 16:59, the banner did not appear.
Summary
Thanks for reading the whole article! As (I hope) I have shown you, manual testing of time zones can be quite simple and interesting. You just have to make sure you clarify the most appropriate approach from the beginning.
In this case, after the feature went into production, we didn’t have to test it any further. If, of course, the tests were supposed to be repeated regularly, it would be necessary to devise a strategy for writing automated tests. A little spoiler alert… We were able to devise the test approach and do the testing itself in just two days! It was a lot of hard work, but the outcomes were totally worth it.
See also
Authors
I have been a tester for over two years. During my testing career, I performed tests on multiple web apps, thousands of interactive ads on Android & iOS devices, and to top it off I also had an episode of being a Project Manager. Currently, I’m learning how to write automated tests in Java and exploring the topic of Machine Learning. In my free time, I’m busy with planning trips, cooking Asian food, and honing my Latino moves.
