The perfect quality assurance for BI systems – real life queries
We have a project featuring not only billions of data records but also huge amount of logic in ETL formed after years of development. In order to include principally new business logic and to optimize the performance we did a substantial reengineering of ETL processes. As you can imagine this should be covered with a laborious testing.
The multidimensional nature of the cube in combination with tens of dimensions and calculations makes it nearly impossible to make a manual comparison of cube values that covers not only a certain feature but the great part of the cube.
Fortunately we save all user queries to the system. Now having both the old and the new version of the cube and using our custom testing tool to execute query traces and compare multidimensional results we were able to test many thousands of user queries.
As a result the major issues were found after minutes of testing. And what is also important they were found by development team and not by users!
Often one has a predefined set of queries in order to test certain features of the system. This is actually the must for systems featuring a lot of data logic. But the real life queries emphasize what the users analyze on daily basis! That is why we recommend to use the combination of both as a part of your quality assurance strategy.