Industry 4.0 has driven the digitalization of vast amounts of data, leveraging virtual models and simulations, including digital twins, to reduce costs. Now, advanced AI and machine learning (ML) models, particularly deep learning, and generative AI models, eg ChatGPT, are becoming increasingly integral to this process, assisting with tasks and ‘open’ information. Localized models can make your company’s information readily available and ensure data security. However, these models present challenges, including high energy consumption, potential copyright issues and the risk of generating incorrect answers. This raises a critical question: how can we determine what to trust?

In software development, models specially trained for code, are assisting in several tasks, eg redesign and restructuring code, porting code and code generation. When training such multi-modal systems, and utilizing them for software development, we immediately run into predicaments. How can we determine what is correct? The research community is working hard to get these tools to aid in bug localization and automatic correction – which shows the difficulty of determining a precise and correct result.

Ensuring the accuracy of results and the quality of data is paramount. If we train our tests on our previous testing, we might not improve the quality. Answering these questions requires robust skills in testing, test design techniques and test automation – integral components throughout the software DevOps lifecycle, as testing is not just a final step; it happens at every stage of development.  Here, ML can automate and enhance various testing tasks. If we use these models to generate our test, we probably face the same issues as with the code that’s being tested. Therefore, fundamental testing skills are more critical than ever. How can we effectively test these models and how can these models be used for testing? How do we determine what to trust and verify as correct? These are the core topics that will be addressed in this talk, providing insights into current best practices and future directions in the field.