The trust between test-dev

In many companies where development and testing are two separate efforts, I’ve noticed some testers trust too much in the developers. Even in companies where development is highly integrated with testing, this could result in missing tests, some of which could be crucial for finding important bugs in the app under development.

One of the clear examples of this is when it comes to CI/CD. Usually, an admin or devops set this up. The involvement of a test expert in this area is particularly important, as they have the skills to plan the right automated tests to validate the builds. I have seen pipelines where there is no testing at all, or a group of insufficient or irrelevant tests.

For some people in the quality team, the developers are their “source of truth”. I’ve fallen into this same error, with other colleagues with more years of experience than me: Whatever they say should be the right way – they know better. Comparing your knowledge to other’s in terms of “more-less” rather than “different” is a recipe for disaster…

Let me tell you the story of a time I trusted too much in an experienced dev: there was an obvious bug in staging environment. I’ve logged it. Someone else logged a duplicated. We were late for release. I said I couldn’t sign up with that bug there.

Butterfly, Insect, Colorful, Moth, Moth Cerura Vinula
From https://pixabay.com/images/id-814568/ – “Oh, hi there! I’m here because the dev lead says it is fine”

The developer assigned to the feature, called the PM and me to his office and both tried to reason me to sign up. I explicitly called out the risk. He said the bug was unlikely to be hit and he didn’t have time to fix it. I trusted him – I was only a junior while he was senior and longer in the company. My instincts told me that the seniority did not make him a fortune teller, but even so I said: “OK, I’ll sign up as long as you get my back if anything happens.” He reassured me and agreed, he said he would take all the responsibility if anything was to happen. Nothing was written down about it.

When users hit the issue in production, both PM and developer immediately blamed me. By writing. Both of them. My manager blamed me. They all said I missed finding a clear bug. When I explained everyone what happened, providing logs of the defect, which was still in open state and the duplicated, where everyone involved was informed. My managed called me to his office. He told me I shouldn’t have signed up and my behavior was not professional as we “shouldn’t point fingers to others”. In other words, I should have taken the blame the others put on me without even trying to clarify it. He didn’t listen to me and it affected my performance review, which was just a week later.

He told me I got two really bad “anonymous” reviews (hard to guess who did them, right?) and the rest, whilst good, were irrelevant.

At the start of that same review period, I found a fix to a very important defect on another team that was costing the company a lot of money and the manager of that team said that if a developer fixed it, they would be immediately promoted. I was the one that provided the fix. It did not seem to matter.

Even the worse of the situations can become a learning, whether it happened to you or someone else. What I learnt from this one:

  • Follow your instincts, if you think something is important stick with it. Avoid trusting blindly other team members based on their experience or how well you get on with them. Know your value and what you can give to the company: I decided to trust a dev based solely on his experience after having figured out an issue that many other experienced devs could not figure out.
  • Getting to an agreement is good, but it is better if you do so by writing, so everyone can remember and review it later on.
  • If you need to release a feature with a defect, that is OK, but plan for the worst and think about a course of action to assure the least disturbance to the users if they hit it.
  • Finally, if you get a manager that does not support you, consider moving away. Leaders and managers should be there to help you grow and learn, to challenge you and inspire you, never to belittle you. You should always feel they make an effort to understand and protect you, at least, that’s what I try to do when I lead/coach others.
Arm, Hand, Desk, Notebook, Pen, Writing, Write, To Do
From: https://pixabay.com/images/id-1284248/ – Plan for the worst and write it all down.

I hope this story I can help you out if you find yourself in a similar situation. Remember that every obstacle can become a learning.

As a final note: having trust is very important, given the right dialogues and plans of action. The opposite (not trusting the developers) is also not ideal. When there is not enough trust, walls are created between teams and things tend to get tested multiple times, but that’s well..another story.

30 days of programming languages – Java vs C#

I am starting to collect a long list of programming languages that I have worked with, but in reality, when I need to switch from one to another, it takes me a bit of time to get used to them again, specially after a while.

A couple of months back, I’ve challenged myself to practice a language each month, by solving a daily problem, to keep myself constantly updated on them. Since challenges seem to be fashionable, you can call this the #30DaysOfProgrammingLanguages challenge.

To do this, I started working with leetcode, which happens to have monthly challenges, so the choice of those problems was already conveniently made for me. These problems don’t take long to complete, and if they do it means I need to practice that concept, which in turns means that it’s a good thing to go over them.

I did April’s challenge with C# and May’s with Java, and here is what I’ve learnt:

Initial assumptions

Before starting my feeling was that both languages are close enough and I was going to need to change very little in between them, maybe some grammar. On the other hand, I thought that one of these coding challenges per month was not going to be enough to review all the details of the languages and to really get them I was going to need to build some app.

Discoveries

Casing differences:

Java prefers classes in uppercase, and methods in lowercase, the opposite as C#.

Length, length(), size(), count:

I am still wondering why do we need different grammar for the same concept. As far as I remember:

  • For C#, arrays use Length, strings Length and Lists and most other classes under Icollection, Count.
  • Similarly, for Java, arrays use length, string length() and the others usually size().

When in doubt, remember the rule of lowercase in Java, upper in C#.

Values and references:

For problems such as Validate Binary Search Tree, sometimes is convenient to send “null” for the first iteration of the methods. In Java, this means passing the object instead of the data type (so you are getting reference instead of value). In C#, it is possible to do this too, using nullable value types (“int?“), but I actually struggled many times with this problem in C# because this was not introduced until C# 8.0 (.NET core 3.0). It is possible to solve it without passing null, it’s just faster and cleaner to use that.

The following are two equivalent function definitions, for C# first, and Java second.

private bool isValid(TreeNode node, int? left, int? right);
private boolean isValid(TreeNode root, Integer lower, Integer higher);

Dictionaries and maps:

This is probably what needs biggest attention. I honestly find dictionaries much easier to work with in C# than in Java. For example, access of an element is done with [], in Java you need the get() method, which is already more typing and more confusing when getting inside other statements that use parentheses.

Iterating through a dictionary (in my experience, if you need to do this, keep in mind that there is probably a better way of doing that exercise than the one you are thinking of), is done by KeyValuePair in C# and entrySet() in Java.

On initializing, you need to use objects for Java while you C# would use data types when possible, which means you need to compare with equals rather than ==. Check below’s example for this last point, first code is written in C# and second the Java equivalent.

Dictionary<Char, Int32> myMap = new Dictionary<Char, Int32>();
 // Initialization omitted. 'a' and 'b' have the same integer.
 myMap['a']==myMap['b']; // -->works
 myMap['a'].equals(myMap['b']); // --> Compiler error: 'int' does not contain a definition for 'equals' and no accessible extension method 'equals' accepting a first argument of type 'int' could be found
 HashMap<Character, Integer> myMap = new HashMap<>();
  // Initialization omitted. 'a' and 'b' have the same integer.
 myMap.get('a') == myMap.get('b'); //-> sometimes would return false when true
 myMap.get('a').equals(myMap.get('b')); //-> works

Note as well how the use of generic are slightly different for both of them. Since most times it might not cause an issue, because the way the heap works, it might cause issues that are hard to reproduce. This was one of the problems that helped me reproduce the issue (in particular when there was a case with a very long string with at least two characters with the same frequency).

Interfaces for the data structures:

Interfaces in C# conventionally start with “I”, while in Java they might have a different names. One more example follows, C# code firstly and Java equivalent after.

IList<T> myList = new List<>();
List<T> myList = new ArrayList<>(); // There is no "List" class

Below is a generic diagram of equivalence of collection classes in Java vs C#, where left and blue represent Java conventions and green and right C#. Note the classes below are for the C# generic collection, but you might find keywords that would match Java’s in C# which are not from the generic collection (you can’t use <T> with them). The use of these are not recommended and I have not added them in the diagram’s C# side.

There are also some concurrent classes that are not included.

Diagram made with Creately. Concrete classes that inherit from Map/Dictionary are purposely not added. Zoom in might be required but worth it.

Let me know if you want another article comparing class by class and zooming in on each group, or if you find any errata on the diagram.

Rediscovering switches:

One great thing about leetcode is that once you have finished the challenge, it shows you a graph comparing your solution’s performance with others. That gives you the opportunity of checking how others did it and improving your skills. By doing this, I realized that I needed to refresh my knowledge about switch statements in c#.

Things I did not get to review

There were also some things I did not happen to find and I wish I would have reviewed:

  • Regular expressions
  • Lambda expressions
  • File manipulation
  • Tuples
  • Concurrency and async programming
  • Iterators over data structures (more in depth)
  • linq

Another issue to mention is that most times you can get away changing the objects that were passed by parameters in the problems, without penalty from leetcode, and this could lead to problems in a bigger code base.

It will be nice to have challenges that highlight these in a git repository or leetcode collection and add some reviewing these concepts.

You can see other differences here.

Conclusions

I actually completed several of these problems per day, and some of the days I would switch back and forth (especially if I find something interesting), but the point is to keep constantly updated with the languages and I was looking for something that I could do daily without getting burnt out. Even so, I did find many things I did not remember of the languages and I refreshed more than the syntax. I think under daily busy situations, one problem a day is still achievable.

If I was to go back to some versions of say, Java, I would encounter different issues than the aforementioned. Keep in mind these are the ones I noticed at the time of writing this article.

Overall, it was a great experience and review of concepts. I am looking forward to keep refreshing other languages and see how it affects me in the long run. So far, I feel I’m slower when writing code because I keep forgetting the right grammar and particulars of each language, hopefully this will help me getting faster while keeping me up to date across them all. I will keep comparing other languages and updating my experience, but that’s well..another story.

Concurrency and parallelism in testing

There are some concepts that people pay extraordinary attention to while developing, but they tend to go under the radar for testing. I believe this is the case for concurrency and parallelism. In this post I’d like to review them and discuss their importance in the testing area.

Running your tests in parallel:

This means execute several tests at the same time in different machines. Sometimes this is run in an external location (for example in “the cloud” – which is basically a group of physical computers placed on several undisclosed locations)

Could you do run tests at the same time in the same machine? Technically, if you have multiple processors, then they might run at the same time for your tests, but it’s not a given. Sometimes, for machines with several processors, you would need to specify their exact use while writing your code. If that’s the case, our code would get too complicated and machine dependent, which wouldn’t be ideal.

Photo by Manuel Geissinger from Pexels – This is a server room, which is basically a room full or computers without screens. When the use of these are allowed externally and interconnected globally, this is called ‘the cloud’

Concurrency: what is it and why would you need it?

You may be thinking: “hold on your lynx, I’ve run tests in parallel in my same machine and they worked as a charm”. To what I reply: how are you so sure that they were actually executing in parallel and it’s not an “illusion” from your machine? Computers tend to be very good at giving the impression that things are done at the same time: you can move the mouse and watch a video and one thing does not interfere the other. But that does not mean that is happening this way, the processor just switches between them so quickly or eyes don’t realise about it.

Concurrency happens when something appears to run at the same time as something else. This is what you could be experiencing if you run tests in “parallel” in your machine.

Even thought this concept might seem far from the testing world, as many applications are developed with this at their core, there are many errors that could be caused by it. Therefore, we should always be aware and try things that could potentially cause them. When some error seems hard to reproduce and almost random, suspect there could be some concurrency issue happening (or parallel, if more machines are involved such as servers).

The importance of independent tests

In order to achieve parallelism, you need to make sure that the tests are not going to interfere each other. Let’s see an example below:

Test 1: “Item can be added to shopping cart” : adds a item to shopping cart. Verify cart has a item.

Test 2: “Item can be removed from shopping cart”: adds a item to the shopping cart and removes it. Verify the cart is empty.

Can you figure out what issues could be caused by these two tests if they were to be executed at the same time? If the shopping cart is shared across the tests, the timing of execution could make one or both tests fail randomly. It is very important that tests provide an unique value and that value is the same under the same circumstances and every time the test is executed.

For better tests, we might check if the value is in the cart or not, rather than if there are more values in the cart. Also, consider what would happen if two tests of the same type execute in parallel (two people pressed the run button). For example, consider the following execution of the two tests before called twice (1.1 and 1.2 are two execution of the same test 1 from above):

Test 1.1: adds item.

Test 1.2: adds item.

Test 1.1: verifies it’s in the cart. Removes it.

Test 1.2: verifies is in the cart (there were two of them, so it is) and removes it.

Unless there are issues on remove (which should have its own test) everything should work as before. However, what about our other test (test 2) executed twice:

Test 2.1: adds item to cart.

Test 2.2: adds item to cart.

Test 2.1: removes item.

Test 2.1: Check if the item was removed. Fails!! (There is still one item there)

Test 2.2: Continues and passes.

From https://pixabay.com/images/id-4974313/ – This seems to be the most wanted shopping item of 2020, and we’ve just added too many of them to the cart.

A solution could be to verify the number of items before removing, but even so, you are not guaranteed that another test would not add another one right after you checked. For this, you might want to add unique values of items per test execution (the items might be stored on a database under a unique ID)

If uniqueness couldn’t be implemented this way, maybe you could have a method to get items from a list based on your IP and execution time, so items would be picked at random and it would be more secure (although this still does not guaranteed independence).

Another way of implementing uniqueness is to use different users per execution (so your shopping cart is unique to your test case), but you might have a restriction on your number of test users and you might need a way of orchestrating them across test executions.

Finally, you could use some sort of orchestration on the test execution to make sure nobody else executes this test at the same time as you. The right solution depends highly on the implementation of your app and your testing platform.

Lastly, you should be careful with methods like “initialise” and “cleanup” to arrange circumstances like this as well. These methods should be used for common repeated operations among tests such as variable initialising, not to perform tasks as to deleting all values from the shopping cart.

Dependencies on testing

Sometimes we might need to deal with dependencies on testing. I’m not talking about: “always execute test 1 before test 2”. That’s probably a really bad idea that you should avoid. But what if you need to communicate two tests? Imagine you are testing a chat platform and need to test the following feature:

Friend 1: opens chat. says hi

Friend 2: should see a new chat with hi. Sends received

Friend 1: sees received.

Photo by Roman Pohorecki from Pexels

In this case, you should orchestrate everything from within a test, and make sure that if you run the same test twice, it wouldn’t interfere with itself. There are many ways in which you could do all of this, but that’s… well… another story…

Writing VR automation with Unium

As part of my talk in Heisenbug, I showed two examples of automation in virtual reality: Unium and Poco(Airtest). I feel the explanations weren’t very deep, so I decided to use this format to expand on them.

1) Preparing the APP to test

In the examples below I’m going to be using a VR app that I created as part of the VR foundations Udacity course. This app was done using Google VR Cardboard(which is currently sort of deprecated, but I believe it’s the best first step into VR development)

You can find the code here. It was build with Unity version 2017.1.0p4 and Google Cardboard version gvr-unity-sdk-1.0.3. I’m specifying this because, sometimes Unity is funny when switching versions and things go deprecated fast in Google VR, so if you use other versions it might not work straight away.

This application is a maze game. The user has to find a key and then a house to unlock the door with it. There are also coins scattered around the maze and the goal is to find them. Inside the house, there would be a board with the number of found keys.

Unit testing the application with test runner we can check some things. For example, we can check that the door ‘lock’ variable is on or off. However, we don’t see the real result as the user would, which could cover issues such as the lock variable does not cause anything to happen in the scene or the sound or particle system are not pleasant or the user cannot reach the objects or they sizes feel wrong…

To test this, you would have to actually go around the maze, get all the coins, get the key, go back to the door and check everything this way. The path is always the same and manual testing could get very time consuming. That’s the reason of looking for solution for automation of user behaviour, as we would see with Unium.

2) Installing and configuring Unium

Although there is a Github project available, I recommend you do this from the asset store in Unity.

To open the store in Unity we click Window -> Asset Store.

Then we search for Unium and we download the asset.

A folder will appear then in the Assets’s folder of the project. Then we add a new empty object in the Scene of the project and we add the script of the folder from our Assets. What this script does is opening a local web server, with port 8342 by default (this can be changed), that we can use to interact with the application, retrieving data from it and sending data to it. If you are curious about how this could work, check out my post about API’s. This creates communication capabilities between the server and our Unity program.

3) Using Unium

Now we can do calls directly in a browser as if we were accessing a website, such as:

http://localhost:8342/q/scene/Main Camera/transform/position={'x':1,'y':1,'z':0}

This moves the camera to the position 1,1,0. To rotate the camera we can use:

http://localhost:8342/q/scene/Main Camera.Transform.eulerAngles={'x':180,'y':0,'z':0}

This would set the camera to a rotation of 180 in the x axe, 0 in the y axe and 0 in the z axe.

Movement of the camera is crucial for VR apps, and we can see three issues already with these functions. The first one is that for VR applications we usually move by clicking objects called “waypoints” or with handset functions. With Unium we could use waypoints as objects and click them as such:

http://localhost:8342/q/scene/Waypoints/Waypoint_7.Waypoint.Click()

Note that I have all waypoints under another object “Waypoints”, the exact waypoint to click is called “Waypoint_7” and then the code that has the click method is called “Waypoint”. The call to click on the door the is easier because we only have one door object and we are not reusing names:

http://localhost:8342/q/scene/DoorObject/Door.OnDoorClick()

If you are not sure of the exact name of the object you can use wildcards and queries.

The second issue is the use of EulerAngles. I was able to see the rotation with quaternions (using “rotation” instead of “eulerAngles” at the end of the call) but I was not able to set it up for some reason (maybe now there is support for it or I was doing something wrong at the time of the testing)

The last issue is the rotation of the camera: users would create a movement when rotating towards a point or in a direction, but the rotation done with the call is done in one setting. This would not really emulate the user behaviour but with Unium we could set up a script and emulate such behaviour or use the web socket capabilities to run a query with a chosen frequency.

Unium has many scripting languages which can be used to automate these calls, for example nUnit (which is the one we would use in C# for unit test). Here is an example of use of code to do this:

dynamic camera = await u.Get("/q/scene/Camera");
await u.Get("/q/scene/Camera.transform.position={\"x\":1, \"y\":1, \"z\":1}")
dynamic pos1 = await u.Get("/q/scene/Camera.transform.position");
await u.Get("/q/scene/Camera.transform.position={\"x\":180, \"y\":1, \"z\":1}")
dynamic pos2 = await u.Get("/q/scene/Camera.transform.position");
// verify that pos1 != pos2

4) Conclusions

Unium is a powerful tool for automation of Unity projects. VR automation is possible with it, even emulating user’s behaviour. However, this sort of automation is not fully black boxed, as you need to add this code into the application and possible use some methods that are existing in the application. Ideally, a fully black boxed automation would simulate the behaviour of the controllers directly.

What’s the difference between automating any game VS automating a VR Game? The main difference is that instead of a movement of a player, you need to automate the camera movement and take into account the handsets (if present).

Sometimes, the transform (walking) is usually done by clicking waypoints and this could be done easily with Unium. Rotation can be emulated by rotating the camera as demonstrated before.

The next step would be to to automate the handsets. However, if we do this directly by calling functions on the controllers of the handset, we might enter the territory of testing that the functions by the controller’s provider are working fine. Instead of this, we might just want to make sure that our functions work fine, so maybe we should call the functions that we have created to control our application when the controller selects and clicks around. But that’s… well… another story…

The HONEST (anti)guide to WFH

With the lock down that many countries are having these days and the need to work remotely, I’ve been noticing an increasing number of posts about how to work from home. I think many of the advises are quite obvious, but some of us tend to gravitate towards a “different” experience when we start doing it. So, here is my satiric guide to how to NOT work from home:

Free-Photos / 9091 images – Example of a perfectly fine home ‘desk’
  1. Turn off your alarm. No commuting means extra time, and the office has flexible starting time, what could possibly go wrong?
  2. Wake up, go to get some breakfast, put your phone to charge and realize you had a meeting 10 minutes ago. Which genius decided to set up a meeting at 9am when everyone usually starts arriving at 9.30m?
  3. Put on a shirt. Don’t worry about the bottom part (it’s an online call and you’ll be sitting down) If you are female improvise some make up without washing your face (no time for that).
  4. Take your laptop and sit in the living room. Everyone is still asleep, surely they wont disturb you.
  5. Try to pretend you’ve been in the call all this time when it’s your time to speak.
  6. Your favorite relative appears in the background and turn the TV on. Turn microphone and camera off while taking the laptop to a more private room.
  7. Come back to the call and apologize for having “connectivity issues”. Turn the camera off and on a few times to prove your point. Finish your talk.
  8. Your relative walks behind you and starts speaking to you without noticing you are in a call. Shut them up.
  9. Realize your microphone was on. Turn off.
  10. The call continues. Realize that the camera is pointing down and you were still wearing your pajamas. Turn it up and hope nobody noticed it. They have.
  11. Finally the meeting is off. Go to speak with your relative who is now very angry and offended because you shut them up and try to explain the situation. They don’t quite understand it.
  12. Go back to your improvised desk. Start checking your slack and email.
  13. Realize is time for lunch. Go to have lunch.
  14. Go back to work. Notice everyone started pinging you while you were having lunch.
  15. Reply to your coworkers, even the ones that are sending you blogs about how to work from home and other jokes.
  16. Go get a snack.
  17. Go back to work. Start reading blogs about how to work from home.
  18. Go to get a drink, the snack made you thirsty.
  19. Go back to work. Remember you were working and close the blogs. Do a bit of work.
  20. Go to the toilet (badly needed now because of the drink you had before).
  21. Go back to work. Realize the day is almost done and you have not done anything useful. Start doing some work.
  22. Your relative comes over. You chat for a bit because you feel guilty about before.
  23. Go back to work. Read the latest news about the global pandemic.
  24. Your neck hurts and back hurt because your desk and chair are not ergonomically set and you have the habit of sitting down in weird positions. Go lie down.
  25. Go back to work but immediately pick up a personal phone call from someone asking how everyone is getting on.
  26. Go back to work, it’s time to turn off the laptop but since you’ve done nothing and you are at home you can make up staying a bit longer…
  27. It’s 9pm. Realize you’ve done nothing really useful today, but tomorrow… tomorrow you’ll be super productive! All you need is to do is to set up some daily milestones and read one more blog about how to work from home.

I hope some of these made you laugh a little and was worth your quarantine’s time. These days I am working on a side project, but that’s…well..another story.

Test architecture in a nutshell

Picture from ulleo / 3990 images – a plant in a nutshell, literally

In my career I’ve gotten a lot of questions about test architecture: How to start testing when nothing has been done for years? How many test projects should there be? How many different type of testing do we need? Who should be in charge of testing in percentages? How to escalate testing? How to deal with big applications? How to deal with existing tests that are incomprehensible? What tools should we use? How many browsers should we test (are there more than two!?)?

First things first: Each application is different, each company is structured in a different way and with different resources and budget and each one has a different maturity state (which does not mean they should aspire to go to the next state, maybe they are right were they should be). There is not a ‘one size fits all’ solution and that’s why I recommend you check it with an expert. But here are some of my previous experiences, in case you find them useful.

On getting started

The first things I ask when I start a conversation about starting testing for a company are:

  1. What process are they following? (Agile, waterfall…)
  2. What technology are you using? (For development, any existing tests, code sharing, code control, task management…)
  3. What’s the relationship between the developers and testers? Do they sit together? Share code? Do common code reviews?
  4. What size is the company?
  5. Do developers write unit tests?
  6. How many people are dedicated (or plan to be) exclusively to tests?

Do they ring a bell? Here are some of the questions I suggested you could ask your interviewer if you are interviewing for a job. Of course, depending on the answers to these questions, I would suggest a different set of tools and practices (or if I am looking for a job, I could tell if I would be able to grow professionally and help the company given the circumstances and what to expect of the position)

Some more that are truly valuable are:

  1. What does your app or company do?
  2. How many clients do you have/plan to have?
  3. What’s the current vs potential growth?
Image from StartupStockPhotos / 121 images – all architecture need a study of the company first

In my experience, when someone asks for help getting started with testing, there are three status the company could be at, and therefore three things might be needed:

  1. Convincing upper management of the need to test: Sometimes you need to be able to explain to high management of the need for testing so you can get started with it. Their main fears are usually time and budget. Addressing those fears are not always easy and depend on each case, but the most important question is “Why do we test?” I leave this one for you for now, but if you are able to explain it, this point would be very easy to deal with.
  2. Convincing other team parties: Other times, they know testing is needed but they don’t know where to start (hint: unit test and TDD) When you don’t have a test team in place yet, I usually suggest the developers start getting into test. Then it would be nice to have someone with test experience to orchestrate these tests, to educate the team about testing and to start creating a test system for end to end and other sort of testing, or even doing manual checks. However, from when you get someone to when you have it all up and running, it might take a while, that’s why it is good that the developers start spending time in unit testing, as ideally they would keep doing this afterwards anyways.
  3. Getting a full framework up and running: In this case, the company have some unit testing and/or manual testing but they need help creating an automation framework and identifying other testing needs. Maybe they are also feeling insecure about the current tests and want to make sure they are not missing anything or want to improve their system (see the next two points).

On keeping it going

Sometimes people are struggling with their current test systems. They might be wondering if they are doing their best as it is, or if the tests they have provide actual value. The questions you need to ask yourself are:

  1. Is there any repetition that could be reduced? (how many current tests and what type of tests do you have? If they run over 15 minutes, this should likely be improved)
  2. Is what we are testing really important? Sometimes we test things that don’t really provide any value but it’s taking time and budget to implement and keep.
  3. Are there any tests that are not up to us? For example, downloading something from a browser, unless your app IS a browser, it’s probably something you should not test.
  4. Are we executing out of date tests? Keeping the system clean is highly recommended, if a feature comes back, you should be able to retrieve those tests too.
  5. Could we use any tools for tracking the issues to understand where we need or don’t need tests?
  6. Do we have the right tests for the right moment? Integrating tests with CI/CD in different branches means you need to decide what to run and check and when.

On escalating

Horizontal scaling means that you add more machines to your testing. For example using the cloud to test in different systems (more web browsers, mobile vs machine, different OS…)

Vertical scaling means that you scale by adding more type of tests (security, accessibility, performance…)

Identifying the type of tests you need and what systems should be covered would be part of an architecture design.

From Free-Photos / 9091 images – Don’t forget to have a conversation with the team before and after designing the test architecture. Everyone should be on board

On moving on

You’ve set up one or more test frameworks. Your team (developers/SDETS…) are managing the new tests which are carefully planned and being executed as part of the CI/CD structure. The analytics are set up and alerting of any problems or improvements… Could you ever be done with testing?

Most people in quality would straight away jump to the easy answer: “you are never done with it”. However, I feel like bringing up the argument: I believe sometimes you are. The same way as you might be done with development too. You will never be 100% happy about it, but sometimes you’ve done your best and maintenance it’s all there is left, at least for that project.

What do you do when you are developing and you are done with that project? You move on to the next. Of course, someone should stay to fix potential issues and provide support, but you wont need such a big of a team or the same sort of skills for this. I argue here, that the same could be said for testing. Development and testing come hand to hand, when there are less changes in development there would be less changes in testing.

Does that mean nobody should look into it anymore? No, that means you need someone with other set of skills to do that. Not worse or better, just different. Some people enjoy being at the beginning of the projects (I love doing research and creating new stuff) but other people prefer maintaining them. And this does not need to be after all it’s escalated, maybe your company is in the place it should be, and investing in more testing might not be necessary, even if you have the budget for it. The problem is, of course, discerning when that is the case, for which I highly recommend talking to a professional.

From JACLOU-DL / 5460 images – look at this cat: this cat is moving on, why can’t you?

On hiring the right test manager/architect

As I mentioned before, having a test expert helping figuring out the maturity of the company and what is needed to improve the quality of the product is very important. What do we call this expert? Test manager and test architect should not be the same position, although, sometimes companies use the descriptions a bit freely.

Test manager: A test manager makes sure the tests are done correctly (being them from test or dev team) and good practices are implemented. The position requires deep knowledge about testing and people skills.

Test architect: The architect designs the frameworks and tools to be used by testing. The position requires deep technical knowledge and experience planning and coding things from the start (also deep knowledge about testing).

Not all companies need both of the positions, and sometimes both jobs would be done by the same person. But being called whatever it’s called, someone has to make sure the system to do the testing is in place and someone has to make sure there are good practices and the right people understand how and what to look for in the quality area.

Therefore, in order to look for the right skills, you need to understand what you need. I have been asked many times for help defining test roles (I talk about the roles I’ve been in here), interviews and even salaries. In fact, there was a time I had to tell my interviewers how to interview me (and, funny story, somehow I ended up not getting the job!) But that’s, well, another story…

Making friends with the impostors inside

Disclaimer: These are only my experiences and this should not at any case be taken as a substitute for any sort of medical advice.

Photo by fotografierende from Pexels

First things first: It is extremely difficult for me to talk about feeling like an impostor. I’ve read about the power of words and how when you say “don’t think about a red hammer”, what do you imagine? A red hammer. So, if anybody is to say: “I feel like an impostor”, whoever is listening is going to imagine you as one. I’ve read about this syndrome before and I couldn’t help myself from thinking: “hmm… if you are feeling like one… maybe you are?” And maybe I am, let me explain…

When people talk about the “impostor’s syndrome” they usually talk about feeling like an impostor. I think it’s more a feeling of not being good enough or feeling different than the others, or both, and that we have multiple of these “impostors” inside.

So, yes, I’ve felt “not good enough” and I’ve felt “not fitting the mold”, and I was probably right every time: I wasn’t good enough for my standards and to myself and I was different from the rest, but everyone is unique in some form.

An impostor on Test

When you are an SDET (Software developer engineer in Test) you definitely are not fitting the mold: Developers tend to shy away from these positions and testers need development skills for them. I remember trying to convince people applying for Microsoft to join the SDET teams instead of SDE teams, because they felt they would be paid less or have less importance.

Photo by mentatdgt from Pexels (not necessarily an impostor)

Microsoft decided to have a separate team for making tools and tasks related to testing (as explained in this book, at least the version I read years ago). This should work out the same way as there are specialized developers for front end or back end. Many other companies followed this approach. But there are so many different companies that work in different structures and call different positions different names, that sometimes concepts get mixed up. And when you are a SDET you get offered all sort of positions, and usually, yes, less salary than a SDE. After many years, Microsoft decided to move everyone to Software Engineers and split the tasks evenly between the teams (which I discussed on previous articles talking about test roles and the future of testing).

On a different company, I’ve worked with SDE’s that have only worked with manual QA’s before, so they could not understand that I was a developer too. These are some of the things I had to hear:

  • “You are lucky, SDETs now are being paid almost as much as SDEs” – I’m soooo relieved that I won’t have to decide between selling my computer science degrees or moving to a position that takes away one letter from my title
  • “I am not doubting you have used it but… you can’t know the same about X technology that someone that uses it daily” – Do you REALLY use it daily? I heard tons of front end engineers talking about back end stuff and nobody is doubting their knowledge based on their lack of daily use
  • “Testing is not part of a developer’s job” – Not sure if you saying this is a bad or a good thing…
  • “We would not mind you to be in our code reviews, but… it would slow us down, better to restrict them to only two people (dev architect and dev lead) reviewing the entire team” – Ignoring the bottle neck you are creating, finding issues during code review would make the process FASTER… and if it is slowing it down you should definitely worry about the quality of the code being written
  • “It’s great that you checked the developers unit tests in your previous company.. But, were they so bad that they needed a tester checking that?” – What they were was actually good team players that split the work evenly and accepted advise
  • “It’s nice of you to be interested on being added to the design meeting, but it is really technical, you will get bored” – You know what? Just take notes afterwards, so I don’t bother you with my “lack of knowledge” (and I save myself from another meeting in which you are discussing technicalities about implementation to assess dominance)
  • “So..your university…was it technical? I mean, did you do technical stuff or was it theoretical but was giving out technical degrees?” – Well, to be honest, I was once walking down the street and I found a degree title with my name on it…(rolling eyes, the average of time for people to finish my degree in my university was twice the expected)
  • “Well, but there are different things in development, for example, design patterns…” – OOh design patterns, really? it sounds so fancy!! Please keep talking… I learnt this stuff on the second year of university: they are just repetitive ways of doing something that people with experience realized about and shared so other people with less experience can brag about… sorry use them too.
From knowyourmeme.com (my face on the last comment)

Of course, at the time, I didn’t think quickly enough these responses. I mostly stared blankly at the person without quite believing what I was hearing. It takes a while to build on responses, but what good would it have made anyways? Sometimes, people just talk because they like hearing their own voices. In Plato’s words “Wise men speak because they have something to say; Fools because they have to say something”.

Sometimes, it does not really matter what you can tell them, they will be stuck on their own beliefs and versions. Sometimes people need to feel, to see, to experience… in order to change their minds. And I believe I did my job with that, since at some point they started coming to me for advise and trusted me much more. But it took a while.

A quick note on degrees: I think having a degree does not necessarily give you better skills. I think degrees are good for having the bases and guidelines. But, no, you don’t need to have a computer science degree to be an SDET, but neither you do for an SDE position (I’ve known plenty of SDE without any sort of certification). However, you have to be really applied and be able to demonstrate skills to do the job; having a degree, experience or some projects you can show are some ways for proving this.

How do I deal with my impostors

Well, as the title of this post states it: I made friends with my impostor’s feelings. I have not overcome them. They are always here, part of me, waiting to get my attention, but I now acknowledge them and learn from them.

Picture by DigiPD from Pixabay (an impostor kitty)

I may not be taking the right steps but here is what I have told my impostor in the past :

  1. Use it on your favor. People that underestimate you are the ones that get most surprised when you get to show your skills, if you feel this is the case, maybe it is a good thing.
  2. Let people be wrong. Haters gonna hate, you don’t need to proof yourself to anybody but you. This is really difficult sometimes, but I learnt to care a bit less about it.
  3. Be kinder to yourself. Most of the times, this is all in your head.
  4. Use it to improve, learn more, grow… I think feeling you are not good enough is a great incentive to get better. Think of the words of Mark Twain: “The secret of getting ahead is getting started.”
  5. Be kinder to others. When someone makes you feel bad you realize you might be making others feel bad too, it can make you more empathic. Use it to reflect about your own acts and also try to understand why the other person is acting like that towards you…maybe they are the ones with insecurities?
  6. Realize that nobody was born knowing everything. Even when they do know about one thing that does not make them automatically know everything else there is to know. A lot of people love flexing. A lot of people have actually not an idea of what they are doing. In the end, everyone is constantly learning, that’s what life is about.
From https://knowyourmeme.com/memes/i-have-no-idea-what-im-doing. I like to imagine this dog with the face of the people that seems incredibly talented or never wrong.

At times, the impostor can get louder, for example when a deadline is coming closer or when public speaking. It does not matter how many times I do this I still can get nervous and anxious about a talk.

A especially bad time for feeling as an impostor is when you get laid off, it does not matter the reason for it, you always feel that if you really were an “A player” it should not happen to you. Also interviewing… this is especially troublesome, as your value might be calculated based in a company’s needs and not your actually qualities: When you need a pencil, a pen will never be as good.

Finally, when you get to join a company the impostor might be calling out the expectations people could have of you.

Those times that the impostors are louder, are when a special effort is required, by listening, understanding and talking to them, but every time it gets a little bit easier. I personally have many other examples when I felt like other impostors, but those are…well…another story.

API Testing

What’s an API?

An API (Application programming interface) is a set of calls that an application does to communicate parts of the application. For example, the user’s view (browser or UI) with some software component (in a remote server or within the user’s computer) that makes the necessary operations for an application to function.

If you are curious about how this looks for a web application, you just need to check the ‘network’ tab on the developer’s tool on your browser. You can see there are many calls happening in the background when trying to reach a website.

Network tab on Chrome for google.com URL

If you click in one of the calls of the left hand side, you will get some information as on the right hand side. The request URL is the address that the call was trying to hit.

For these web calls, there are two main methods: GET (to request some information from the server) and POST (to send some information to the server). You can learn more about other methods here.

The next field is also very important. This is the message that we get from the server when the call is executed. In this case it is 307 (a redirection). If you are curious about what other statuses number mean, you can check this web, if you are a cat person, or if you are more of a dog person this one.

There are two widely used protocols for sending information between devices: SOAP(stands for Simple Object Access Protocol) that sends information with XML format and REST(Representational State Transfer) that sends information with several formats such as json, html, xml, and plain text (see this article for further explanation in the formats).

Tools to test API’s?

Please keep in mind that the tools mentioned below are not the only ones in that you can use for API testing. I’m talking specifically about these ones because they are the ones I’ve used in the past.

In the section after this one, I’ll show an example about how to do an API test.

Swagger:

According to their website, Swagger is an open source and professional tool-set that “Simplifies API development for users, teams, and enterprises”

https://swagger.io/tools/swagger-ui/

I have used swagger UI as a way to easily check API URLs and understand the calls to then add them into my test code, but, I have not tried all the Swagger’s tooling set yet. I think this is a easy way to communicate changes on the API across the team and document it.

Alternatively to this, the developers should document their API calls in some other form, generally some list, as Twitter does here. The problem I have had with this option is that sometimes the documentation might be out of date and then you need to figure out in the development’s code what’s the exact API call. With Swagger, the list of calls should come directly from the code (clarification UPDATE: depending on the API implementation and if the changes are back tracked) , which makes it easier to handle and up to date (clarification UPDATE: you shouldn’t assume anything in testing).

Swagger is supported by SmartBear, same as SoapUI, so for API testing with it, please check below.

Soapui and Postman,

SoapUI is ” a Complete API Test Automation Framework for SOAP, REST and more”. There is an open source version and a professional one featuring more functionality. The API testing part looks like this:

https://www.soapui.org/docs/endpoint-explorer.html

I took the image from their website and it is quite self-explanatory. Besides, there is a lot of documentation to get you started in there.

Postman is a “collaboration platform for API development”. In the development sense, it provides automatic documentation, so there is no problem with developers making change on functionality and forgetting to upload it.

For API testing, it’s very easy to get started. You can create REST, SOAP, and GraphQL queries. It supports multiple authentication protocols (I talk about this later) and certificate management. Please refer to their website for further information.

Wireshark and Fiddler:

These two programs are very useful to analyse network packets. They are powerful programs and a must known for security, network and performance testing and checking the packets at micro level. You can actually see the exact data sent over the network. However, if what you are looking for are tools for API testing, I would probably not go for them but the ones above, because they are more high level and specific for that.

That said, I have used them before in order to test API’s that required specific secure certificates and for debugging issues (specially performance ones). If you are interested in knowing more about Fiddler, I recommend this article. For Wireshark, this one.

https://blog.wireshark.org/

How to do this programmatically?

If you want to add this testing into your automation code, you have some help on with the tools mentioned before. However, there are many ways of doing these type of calls with different programming languages. For example, here is how to do a rest call with python:

import requests
# make a get call
response = requests.get("URL")

# do something with the result
response.status_code # this gives you the status code as mentioned above
response.json() # this gives you a json with the response on it

# make a post call
response = request.post("URL", {json data})

It gets a bit more difficult when you need to add parameters, authentication or parsing some types of data, but there is plenty of documentation about it all. Let’s see an specific example using the API provided by numbersapi.com.

import requests

response = requests.get("http://numbersapi.com/42?json")

print(response.status_code)
print(response.json())

The result when you execute the code above is:

200
{'text': '42 is the result given by the web search engines Google, Wolfram Alpha and Bing when the query "the 
answer to life the universe and everything" is entered as a search.', 'number': 42, 'found': True, 'type': 'tr
ivia'}

With Python, you could play with the json data to easily retrieve and validate the text, or the number, that there is some result…

For more information about what exactly test when testing API, I think this post is wonderfully well explained (they use postman as example).

Why should I care? UI VS API testing

UI (User interface) testing is the best way to simulate the actual behaviour of the users. However, we tend to re-test things in the UI that could be covered already by testing the API (and in some companies this could be done by a different group or team).

Let’s say a developer changes a call for an API. Let this call be the list of movies that someone liked. Now imagine this API call is not modified in some part of the application, the result being the user cannot find its liked movies. What’s happening in the UI test?

We will then get the UI Test couldn’t find an object. This could be due to the API call being wrong or a bug in the automation, an update on the way of the object needs to be retrieved, a button not working properly, the object being hidden somehow…

However, if you have an API test for it, you should be able to understand that the call is not retrieving anything. If you need to verify things such as results of a search, it’s probably best to use API to check the entire list (which could be done with a quick comparison) and let the UI verify that a result appears where it should rather than the result itself. Also, you should be verifying that the API call is correct (and update the test call if it is not).

Level up:

API calls are less likeable to change than UI objects and they generally come in different versions when they do, not to disturb previous releases of the application. This means you might want to add functionality to verify which version is being tested.

It is also interesting to use this to speed up our UI testing. The most common example being the Login method. This method is usually a bottle neck for the rest of the tests, and if it fails, you don’t know what else might be failing or passing and you are blocked until the fix. Whilst it’s super important to have a Login test to make sure that your users can log into the application, performing UI login each time it’s needed for another test, slows down your execution.

Google login screen

What’s the solution? You can use API testing to skip the login bit. Be careful when doing this, it wouldn’t be secure to have an API to do this in production environment. An example could be to set up some unique tokens (see an example about doing this with soapUI here) that are of quick expiration to perform this skip and send it alongside the URL, or have an API call that sets some cookies or session to be logged in.

If you have other repetitive dependent tests, you should consider running API calls for them before continuing with the test that depends on it. This would considerably speed up your test execution and your results would give you more trustful information about the problem.

That said, UI testing is the best way of ensuring everything works as per user behaviour and E2E and integration tests should not be substituted by API tests, use it only as a help and if it is not increasing the complexity and failures of your tests.

Yet another level up: stats

Another interesting thing that you can do thanks to API calls is to find out information about your application and about how users are using it. To analyse and visualise calls in a bigger scale you can use tools such as elastic search and kibana, and even use artificial intelligence to get conclusions from such calls, but that’s… well… another story.

2019 Review and 2020 plan

While everyone has been preparing this year’s resolutions, I wanted to share the ones I had for last year. I was getting the feeling that I did not do very well with them, but after writing them down, it turns out I accomplished a lot and it makes me want to achieve more next year.

Photo by freestocks.org from Pexels

Doing more

One of the things I am often asked by people is: “how do I manage to get to do so many things on my free time?” When I hear this question, my first thought is: “well, you obviously haven’t visited my blog often…”

To me if I can help one single person throughout the contents of my blog or inspire someone with my talks, I would achieve my goal: I would have made the world a bit better. For this reason, I tend to wait for inspiration to write; I might be doing a research for the next post, or working on learning a skill.

Therefore, once you are aware of your end goal, I think the first step to get more things done is to find inspiration. You can get this from reading books, articles, attending to talks, having active friends, joining courses or projects… But, be careful, sometimes this could be counterproductive. Maybe seeing how everyone around you manage their time so well, discourages you for not being able to do as good. Don’t let this bring you down, everyone has their own tempo and you can start at any point in your life.

A trick I use sometimes when watching or listening to some content, it what I call the ‘2x factor’: set it up 2 times the normal speed. I don’t recommend you do this with everything, for me it generally depends on the speaker’s speaking pace. When the speaker is talking slow or about something I already know or don’t find very interesting, I increase the speed of the video. I find generally 1.50x is a good speed for me for most things, 2x might be too fast and whilst I can understand everything, it makes me nervous.

https://www.pexels.com/@neo8iam

If getting inspiration is the first step, the second one, to me, is to get to action. Sounds easy? Get started on anything, even for 5 minutes a day, and you will be achieving more than you are right now. I particularly enjoy the ‘pomodoro technique’ (with rewards), in which you try to work 25 minutes straight followed by a 5 minutes break (in which I grab some drink or snack). I find it very hard to finish the 25 minutes the first two or three times but then I start hating the alarm for taking breaks because it interrupts my flow.

Books

One of my new year resolutions was to track my readings. I’ve noticed I’ve been reading here and there, never really stopping to think about my progress.

First thing I did to firstly, check my progress and secondly inspire myself to read more is to join the goodreads reading challenge. Since I was not sure how much I was currently reading, I set it to be 24 books.

Photo by Pixabay from Pexels

I was shocked to realise that I read 6 books the first month. Ok, let’s be fair, half of those were audiobooks, I love walking around and having them help me get to more information. Because of traveling or other commitments, I didn’t get to 6 books every month, but I also didn’t track them all.

I discovered that sometimes Goodreads didn’t have the book I was reading and sometimes it just removes books from your shelves without noting. I contacted their support (as suggested on that link) and all I was told was to re-add them and make up the dates I read them… This proved to me that it is not a good place to track books so I’ll be happy to hear of other options if you have any.

At the end of the year, taking all of this into account, I achieved almost 60 books in 2019, so I set this as my 2020 goal. However, I want to keep my focus on quality over quantity and I believe this number may also depend on the length and difficulty of the chosen books.

Commitments

In order to motivate myself, I like having some commitments to push myself to do them, for example, participating in meetings and conferences such as the automation guild (it’s an online conference, you might be still on time to join this year’s). The videos for it took me longer than expected, but I learnt a lot because of that. I presented in a total of 5 conferences around the world and spoke in a couple of meetups. If you want to know more of my past and future appearances, you can do so here.

Another way I have of motivating myself is by joining courses. Last year I wanted to finish the VR course I’ve been pushing aside for a while. I’ve been doing some research in VR and that has taken priority over the course, but paying a course monthly and not having time to work on it is really painful. On the other hand, not paying for a course makes me less committed to it. This year, I want to work more on my AI knowledge.

Activities

Another thing you can do to get inspired is to write a list of life achievements or values you want. If you have it nearby and read it often, it could help you focus on what’s more important for you. Also, try activities that get you excited, find other passions.

Some months I was focused on improving Chinese, practising piano and meditating. Besides, on summer I went back to Spain from China, so was recovering from jet-lag for a couple of weeks and then decided to enjoy some time off too.

Another new year resolution was to practice yoga at least 3 times a week and going to the gym at least 3 times too. I’ve been quite regular with this, even when travelling, except the times I’ve been sick.

Photo by Cedric Lim from Pexels

Next year, I would like to plan better the activities and periods of break, the same as I plan for work or conferences. Otherwise, commitments can get in the way and push everything out and compensation later on does not seem to be the best solution.

Traveling

One of my 2019’s resolutions was to travel more, and I think I surpassed my expectations: I travel through China (Beijing, Xian, Zhangjiajie, Tongren, Guilin..), US (San Francisco), Japan (Tokyo), Spain (Valencia), UK (London), The Netherlands (Amsterdam), Germany (Berlin, Dusseldorf..), Switzerland (Zurich) and Russia (Moscow).

I moved to a new city but, because I was meeting many new people throughout the trips and did a lot of sight seeing of the visited places, I did not feel like engaging in many activities there. I did go to a Muse concert (which was, as usually, quite amazing) and walked around a lot. I like doing this when I go to a new city as I feel getting lost is actually the best way of getting to know a place.

Photo by Porapak Apichodilok from Pexels

Whats next

Something else that works for me is to list the things I’ve done in the day at the end of it. Same for the year, as you’ve read. This is a way of being appreciative, which makes me want to achieve more.

Being grateful is also really important. Another daily writing or morning meditation, if you wish. I struggled with this because some days I might not interact with anybody as to be grateful to them. However, you can be grateful to yourself and your achievements and for the opportunities and the good that’s on its way and the good that happens to others.

I’ve counted 12 post that I have half written, so keep checking because there is a lot to come. A couple of them are on VR (I might merge them into one), thoughts to share, AI, API testing… If you are especially interested in one of those, let me know so I give it priority.

I like most of my last year’s resolutions, I want to keep them, and I haven’t added up much to this year: do things to make the world better, block times of rest, have more patience, look at the brighter side of everything, get an AI specialisation, write more on my blog, find a new app for my reading… Would I manage to add all of that? That’s…well…another story.

How to test a time machine

I have recently watched a video from PBS space time that got me thinking: if we were to have a time machine, how would we test it? I’ve seen a lot of “how would you test X” type of questions in interviews, but I don’t think I’ve ever seen this one before (I am not trying to give you ideas for interview questions!)

It’s not rocket science… it’s rocking testing science!

I couldn’t help but compare it firstly to a spaceflight, so I started wondering: how do they test spacecrafts? And what better one than Apollo 11 to start with? If only we had a time machine for getting its source code… Yes, I have been looking at Assembly code trying to make sense of potential testing routines, like this one… and… guess what I found there?

This section of the code is checking the Gimbal lock of the accelerometers! Do you remember the concept from my last post? Maybe I just have a case of Baader-Meinhof, but I do feel Gimbal lock is an important concept to learn, so check it out.

Testing in Assembly was not as ‘easy’ as nowadays (for example, macros does not seem to be a thing that I could find in the Apollo11 programming syntax). Do not expect a page object model or a library with tests or testing functions. Nor common methods for before and after tests. Actually, don’t expect any sort of OOP, to start with.

In my search I could find some files with tests on them, but they are mostly for stressing the hardware by sending signals to the different devices and recovering from bad statuses. Also, spacecrafts might need to check correctness of bits to make sure there are no catastrophic arithmetic errors.

https://upload.wikimedia.org/wikipedia/commons/thumb/3/3c/Ariane_5ES_with_ATV_4_on_its_way_to_ELA-3.jpg/1200px-Ariane_5ES_with_ATV_4_on_its_way_to_ELA-3.jpg
from @wikipedia

Time traveling tests

Imagine we have covered the unit and integration tests for both hardware and most of the software for our machine that could potentially match any currently existing ones. What are the specific cases we should cover for our time machine? The logical cases to think of:

1) The machine should not kill the traveler: We should insert some devices to measure that the cabin of the machine is livable. Also, keep in mind for the rest of the tests, that usually spaceships are first tested with robots inside, rather than humans. (Security test)

2) There must be some ways of safe interrupting the process (risk assessment/testing): In case anything wrong happens, what’s the risk involved for the passenger? (Security test)

3) As the machine goes forward or backwards, the traveler state stays while the surrounding world changes: We could measure this by introducing some other object before that we know it decays and checking how long time it has passed in the machine. (Usability test)

4) There is a way to return: I assume we will want this, so we should test it. (Security test?)

5) Performance tests: how many times it can go backwards and forward in time? (Performance but also, regression test?)

6) What is the minimum time that can travel? (test it) If the time machine requires a rapid negative speed (as the video above suggested as one of the possibilities), I imagine traveling in time as playing billiard or golf. Let me explain: When you try to position a ball on a particular hole, you need to give it the right original impulse (not too much, not too little) but also the right angle depending on the starting point. We are likely to travel space as well as time, so it might be particularly difficult to stop on a particular moment or place. (Which may explain why nobody made it to the time travelers party) (Usability test)

Photo by Tomaz Barcellos from Pexels

7) Test boundaries scenarios: go back to start of the earth (or universe) and forwards to its end. What happens if just before or just after? Does time always exist? Can we travel to a point there is no time? Technically these should be tests we would like to do, but I think they are probably not doable, realistically speaking (can I use this expression in this context?)

8) Try to change a small thing in the past…does it change the future? If a change makes a parallel universe, then we should not ever recover the machine. If we test this, we should look for an event we can easily undo (like maybe turn a light on/off?) (Exploratory test)

9) Time traveler meeting same time traveler. Test paradoxes. (Exploratory test)

11) Test placing a box where the machine is positioning before a trip. Then take the box away, position the machine on that same spot and test traveling to before we positioned the box. Does the box o machine break? (Integration test)

12) Test traveling to when the box is still in place; do the machine or the box break? (Integration test)

13) Try to travel twice to the same exact time and place. (Integration test)

14) Could the machine travel between different universes? (If we have a way of doing so, which might be more likely if we are using wormholes for the trips than negative speed) (Integration test)

From futurism.com (and an interesting article)

15) Is there a maximum/minimum size of the machine? If so, test them. (Boundary test)

16) Can everybody use the machine or only qualified people? (Accessibility test)

Conclusion:

Maybe we have already invented a time machine but kept it secret because it is dangerous. Or maybe we have gone back in time to stop ourselves from inventing it, creating a parallel universe and as a result, our universe is the one in which such machine was never invented (time version of the Fermi paradox)

Whichever the answers to these questions, I hope you’ve found this exercise fun and enjoyed reading this post as much as I enjoyed writing it. Let me know if you can come up with more tests we could do to the time machine (if we were to have one). I will resume my serious posts soon, they will be… well… another story.