Abstract
The software industry is continuously growing, and software source code quality is becoming more critical than ever. As a result, methodologies for software validation and verification are being developed, including Static Code Analysis (SCA) tools. SCA tools generate warnings for potential code quality violations, and existing studies have reported that they produce false-positives, which hinder their popularity among the practitioner community. Developers waste valuable time on false-positives that could otherwise be spent improving the business logic in source code. To reduce the time developers spend identifying false-positive warnings, this thesis aims to develop solutions to automatically identify false-positives among the warnings generated by SCA tools. We conducted systematic mapping and grey literature studies to understand the evaluated SCA tools and those that generate fewest false-positive warnings. As a result, we selected two SCA tools (PMD and SonarQube) and two publicly available source code datasets (Apache Tomcat and the Stack Overflow), which were then analysed by these SCA tools, yielding four warning datasets. These datasets were sampled, yielding four smaller datasets, which were manually annotated in accordance with established standards. We then conducted a series of analyses on the sampled datasets to identify the rules that triggered warnings, rules prone to false-positives, the reasons for reported false-positives, and gaps in code violation classification between selected SCA tools and established standards. We then used Machine Learning (ML) and Large Language Model (LLM) technologies to assess the performance of ML, including Code Representation Learning (CRL) models and ML algorithms, and LLM in classifying SCA warnings in these sample datasets. Finally, a survey questionnaire was administered to the practitioner community to understand practitioners’ perceptions of the key findings from this thesis. Among our main findings, we found that academia and the practitioner community mostly evaluated a unique set of SCA tools, and only 26.7% of the SCA tools overlapped. Additionally, the two communities used distinct methodologies and performance metrics/criteria. Also, among the SCA tools evaluated by both communities, PMD and SonarQube were the best in terms of the false-positives they generated. Our analyses revealed that these two tools produced false-positive percentages ranging from 4.64% to 18.45%, and only a few SCA rules contributed to most false-positive warnings. Moreover, ML model performance varied with the CRL, ML algorithm and dataset characteristics, with mean F1-Score ranging from 0.62 to 0.82. LLM performance varied with the LLM variant, dataset characteristics, and evaluation metrics, with F1-Score for the best models ranging from 0.52 to 0.96. Moreover, the survey among practitioners revealed mostly positive perceptions of SCA tools, with an average score of 7.97 out of 10. Additionally, survey participants were positive on the use of ML/LLM technologies for detecting false-positive warnings. Most survey participants reported that their companies use some form of customised code quality standards not supported by SCA tools, leading to an increasing number of false-positive warnings. That said, participants believed that using ML/LLM approaches to identify false-positives generated by SCA tools could be beneficial, thereby encouraging broader adoption of SCA tools among practitioners, and improving global software quality.