As testers, we should all be united in the effort to elevate the role of QA beyond simple black box testing – looking “under the hood” to understand what’s really happening within an application at the code level. You don’t need to be a professional software developer to read code and having this ability is becoming more and more important in attaining and maintaining higher echelon QA or SDET jobs.

Not all organizations understand how much more effective advanced Quality Assurance can be when applying some of these techniques. A QA Manager or Lead often has to advocate for additional access to source code repositories like GitHub or Bitbucket because their Development team may not yet understand how it helps QA deliver a better product.

Reading is fundamental.

Even in the absence of traditional programming education and experience, simply knowing how to read code can help you target your manual QA testing efforts. By examining the business logic portion of the code, one can distill the various permutations of possible inputs, and then ensure each of those pathways are explored during testing.

Inputs usually fall into four different categories – long-term stored, short-term stored per session, direct UI entry by the user, or transformed from these sources using some logic.
The many forms of input:

  • Windows Registry
  • File, either stored locally, or on a remote server
  • Database table, more than likely for modern web applications
  • Web sessions, either server-side, or client-side in the form of cookies
  • User interacting with the controls: think clicking buttons, entering data into fields, making selections from drop-down boxes and so on.

Many web applications created today are designed to be fully stateless to accommodate load-balancing across horizontally scaled1 servers. What this means is that while you may be interacting with a server in New York one minute, and your very next session interaction may take place in Delaware. The server code is static no matter where it runs. As long as the current server accesses the shared back-end data repository (usually in the form of a database), then any server can resume any session anywhere, and maintain continuity despite being different.

Variables aren’t just for Algebra

Variables aren’t just for Algebra

One of the most simplistic, but important, concepts of any programming language is how it handles variables. Variables provide temporary storage for the code. If we need to ‘squirrel away’ values to either store permanently later in the database, or to influence business logic decisions, we need to first store it in a variable. Sometimes, this happens automatically by the binding between a UI control and the variable name.

Before examining the business logic, it’s essential to know: where did the data come from originally and what is it based on? Generally speaking, for procedural code, find the IF block and work backwards through the code. You are looking for the first time the variable was declared (for instance, in C, int myCounter;) or the first time the variable was used. Sometimes, in Object Oriented languages, where or how that variable obtained it’s value can take further investigation. Ask your neighborhood friendly developer for help understanding what your commonly-used objects look like.

Ideally, you want to find every place where that variable can take on a new value. For instance, maybe a user-facing control can modify it, or a file loading, or a database SELECT read.

Code Complexity

There’s a well-established relationship between the complexity of the code and the increased chance of finding a bug. Common sense dictates that simple, easy to understand code will likely have less bugs than more complex code with lots of inflection points. So, seek out those portions of code – that have many lines, that are hard to read, that use many different variables – to determine what path to take.

Know your operator and its order

For your given language, do research on the operators available. Most languages evaluate an expression with a result of true or false.

If (a > 10) then doSomething();

The above statement reads in English as “if the fact that ‘a’ is greater than 10 is true, then go call the function doSomething().

Here’s some quick links for operators in some popular languages:

Pay special attention to your “equals” operators. Some test if the values are the same. Some test if the value AND type are the same (sometimes called a strict comparison). Some test if the object ID is exactly the same as the other object ID.

Also, look for logical operators that require multiple values to be true before a result can be given. These are operators like ‘Logical AND’ as well as ‘Logical OR’.

For example, the Java operator ‘&&’:

&&        Logical and        Returns true if both statements are true        x < 5 && y < 10

If both x is less than 5 and y is less than 10, then the overall statement will be true.

Order of Operation

Order of operation determines in what order a non-grouped expression is evaluated. For instance in Javascript:

console.log(3 + 4 * 5); // 3 + 20
// expected output: 23
console.log(4 * 3 ** 2); // 4 * 9
// expected output: 36

Notice in the first example, the “multiply” (*) operator is evaluated before the “addition” (+) operator.

Notice in the second example, the “power of” (**) operator is evaluated before the “multiply” (*) operator.

Ensure you’re reading the operator precedence and associativity rules properly. Most languages use parentheses to specify a non-standard, or to ensure proper order of evaluation.

We’ve discussed where those values can come from, where those values are stored, so now let’s dissect an IF block.

Control Flow Statements

Procedural code normally executes from top to bottom. Control Flow statements break up this flow by employing decision making. Since the program behaves differently, usually as a result of changing variables, then it makes sense to analyze those various decision making inflection points and wrap test cases around them.

Control Flow Statements can take the shape of:

  • IF-THEN, IF-THEN-ELSE statements
  • SWITCH statements
  • FOR, FOR EACH, WHILE, DO-WHILE

Let’s look at how we analyze a SWITCH statement to target and create test cases:

using System;

namespace SwitchEnum
{
    enum Color { Red, Green, Blue, Brown, Yellow, Pink, Orange }

    class Program
    {
        static void Main(string[] args)
        {
            var color = (Color) (new Random()).Next(0, 7);

            switch (color)
            {
                case Color.Red:
                    Console.WriteLine("The color is red");
                    break;

                case Color.Green:
                    Console.WriteLine("The color is green");
                    break;

                case Color.Blue:
                    Console.WriteLine("The color is blue");  
                    break;

                case Color.Brown:
                    Console.WriteLine("The color is brown");  
                    break;    

                case Color.Yellow:
                    Console.WriteLine("The color is yellow");  
                    break;

                case Color.Pink:
                    Console.WriteLine("The color is pink");  
                    break;

                case Color.Orange:
                    Console.WriteLine("The color is orange");  
                    break;  

                default:
                    Console.WriteLine("The color is unknown.");
                    break;  
            }
        }
    }
}

In this code block, we are changing the execution of the program flow based on the contents of the color variable, when it’s value is Color.Red, then we display a message to the console, “The color is red”.

You can see that there are (7) different values that color can take: Color.Red, Color.Green, Color.Blue, Color.Brown, Color.Yellow, Color.Pink, Color.Orange.

There’s also one other case, and that’s the default case. This case runs whenever none of the other values match. You can think of this as the negative testing path, i.e. what does the program do when the value ISN’T one of the normally expected values?

While this is a trivial example, remember that you should work your way backwards or upwards through the code to find where the color variable is defined or first used. We find that just above the switch (color) block here:

var color = (Color) (new Random()).Next(0, 7);

Notice that both the definition of the color variable is done here, but also the assignment to a Random number between 0 and less than 7.

Scanning the source code answers two questions:

  • Which inputs matter?
  • What happens when those inputs are asserted a certain way?

By spelling out which inputs have code associated with them helps you understand how to vary your input to ensure all cases are covered. If there are no control flow statements tied to the variable or field, then that input is essentially a “doesn’t matter” field. Complete your normal negative tests against them.

Analyzed for Action

Sometimes only a part of a field is analyzed for action. Let’s take a look at the following example. This example application uses an office identifier to select which location of a business is being referred to. This office ID has the following format:

CITY_NAME-xxx

Some samples might be DALLAS-107, PITTSBURGH-994, or GREENBAY-303.

If you see code that looks like this:

public class SplitExample{
   public static void main(String args[]){
    // This is out input String
    String str = new String("DALLAS-107");
                 
    /* Using this variation of split method here. Since the limit is passed
     * as 2. This method would only produce two substrings.
     */

    System.out.println("split(String regex, int limit) with limit=2:");
    String array2[]= str.split("-", 2);
    for (String temp: array2){
          System.out.println(temp);
    }

    If (array2[0].equals(“DALLAS”)) { System.out.println(“Go Cowboys!”); }
Else if (array2[0].equals(“PITTSBURGH”)) { System.out.println(“Go Steelers!”); }
Else if (array2[0].equals(“GREENBAY”)) { System.out.println(“Go Packers!”); }
       
   }
}

There are a few test assertions that can be made:

  • The only valid inputs are DALLAS, PITTSBURGH, GREENBAY.
  • The only valid delimiter is a “-” and the field must contain at least one.
  • Nothing should happen if another city is specified.
  • The expected output behavior for the city names are as follows:
    • DALLAS : Go Cowboys!
    • PITTSBURGH : Go Steelers!
    • GREENBAY : Go Packers!

You can see how easy it would be to wrap test cases around these assertions. For instance, “Verify that other cities return nothing.” or a negative test case: “Verify that underscores are not acceptable for a delimiting character.”

In Summary

The QA Engineer should always look back at Requirements and the business-desired behavior to check to see that the code that has been written echoes the true ask of the business. The code should allow you to focus on how to test it, but we still need to perform regular QA business validation to ensure that’s the way it should be in the first place.

The code reading and analyzing techniques presented today are designed to add more skills into your QA repertoire. Use these to continue to refine and focus your testing, writing stronger and more complete test cases, and improve your QA effectiveness. The business will appreciate a higher quality product.

For QA professionals looking to grow, we encourage you to research new techniques, share and discuss them with peers, and continue to expand your skill set outside of traditional QA. Consider approaching and engaging management to elevate YOUR role within YOUR organization. And if you or your business is struggling with Quality Assurance resources, Rivers Agile welcomes the opportunity to craft your software solution. From QA Health Assessments to Software Testing or full-service Product Development, we’re eager to discuss your needs. Contact our team today!

1 https://stackoverflow.com/questions/11707879/difference-between-scaling-horizontally-and-vertically-for-databases