Comparison of Static Analysis Tools for Quality Measurement of RPG Programs

The RPG programming language is a popular language employed widely in IBM i mainframes nowadays. Legacy mainframe systems that evolved and survived the past decades usually data intensive and even business critical applications. Recent, state of the art quality assurance tools are mostly focused on popular languages like Java, C++ or Python. In this work we compare two source code based quality management tools for the RPG language. The study is focused on the data obtained using static analysis, which is then aggregated to higher level quality attributes. SourceMeter is a command line tool-chain capable to measure various source attributes like metrics and coding rule violations. SonarQube is a quality management platform with RPG language support. To facilitate the objective comparison, we used the SourceMeter for RPG plugin for SonarQube, which seamlessly integrates into the framework extending its capabilities. The evaluation is built on analysis success and depth, source code metrics, coding rules and code duplications. We found that SourceMeter is more advanced in analysis depth, product metrics and finding duplications, while their performance of coding rules and analysis success is rather balanced. Since both tools were presented recently on the market of quality assurance tools, we expect additional releases in the future with more mature analyzers.