How can mutation testing benefit from Machine Learning?

Testura is our open source tool for mutation testing. When we asked ourselves if Machine Learning could help improve the tool, we decided to let two students from Luleå University come up with the answers through a degree project.

How do you secure the quality of test suites? One method is mutation testing, where errors deliberately are injected into the application code. This creates so-called mutations of the code. In the next step, the defined test cases are executed to see whether the errors are identified, or the mutations “survive”. If the latter, you will need to look into your test cases, and either revise them or add new.

As a company, we invest both time and resources to ensure we stay at the forefront of the test and quality assurance field. One of the ventures we are involved in is called TESTOMAT – a big pan-European test automation research project. Our work has resulted in an open source tool we’ve named Testura.

The mastermind behind the tool is Mille Boström, who works as a Senior Consultant here at System Verification. Test Strategist Magnus C Ohlsson has coordinated the work and helped out with ideas.

“Testura makes it possible to perform mutation tests of code written in C#. One issue that remains to be solved is when we have a case with a rapidly increasing number of mutations. We need to be able to reduce that number. We discussed whether ML, Machine Learning, could do the trick. And as cooperation with universities and schools is an important part of what we do, we created a suggestion for a degree project on this subject,” says Magnus.

And now, Nicklas Nordenfelt and Martin Modén, both students at the Systems Science programme at the Luleå University of Technology, have got their teeth into the project – with Magnus C Ohlsson as their tutor.

“I have a couple of friends that work for System Verification, and they told me about the opportunities the company offers,” says Nicklas. “So here we are, looking into how ML can help improve mutation testing.”

From the left: Nicklas Nordenfeldt and Martin Modén.

“ML and AI have great potential,” adds Martin. “You read and hear about it everywhere. We have taken courses where we’ve drilled into neural networks* and how different ML algorithms can be used. And we’re currently doing an AI course. It would be nice to work in this area in the future, and it feels like the possibilities are endless.”

Mutation testing is a concept none of them have explored before.

“It will be a challenge to combine different theories and to complete the assignment. But we’ll find a way,” says Nicklas.

”Our ambition is to implement the results of Nicklas’s and Martin’s work in Testura shortly,” concludes Magnus C Ohlsson.

* Neural networks = computing systems inspired by the biological neural networks of “real” brains.

For more information, please contact:

Magnus C Ohlsson
Test Strategist
+46 40 602 28 60

QA Greenhouse programme Applications Testing Vattenfall

Greenhouse – Learning by doing

A group of fresh engineers have been introduced to the field of QA through our Greenhouse program. Today, they’re doing application testing at energy provider Vattenfall.

Anniversary Frankfurt Bonn Germany Celina Strietzel Erwin Probst

Happy 1st anniversary Germany!

In the beginning of 2019, we entered the German market with the aim to grow fast. This has succeeded very well. Today the Frankfurt and Bonn offices have great teams that serve customers in the logistics, automotive, telecommunications and financial sectors.

Learning Conference Las Vegas Re:Invent Amazon Web Services

Learning in Las Vegas

Re:Invent is a learning conference hosted by Amazon Web Services that attracts more than 40,000 people from the global cloud computing community to Las Vegas every year. System Verification is here to be challenged and meet peers from all over the world.