Continuous Benchmarking of Android Taint Tools

Considering the future extension of the TaintBench suite, we set up GitHub Actions for continuous benchmarking of the following Android taint tools.

• FlowDroid April 2017 Nightly
• FlowDroid 2.7.1
• Amandroid 3.1.2
• Amandroid 3.2.0

Using tools from the TaintBench framework, we configured the evaluation of each tool on TaintBench as an automated workflow of Github Actions.

Source and sink configuration (App-level) in a workflow:

• For each benchmark app, a list of sources and sinks defined in this app is used to configure the evaluated tool. Each tool analyzes each benchmark app with the associated list of sources and sinks.

Outcome of each workflow:

• A benchmark result file containing performance metrics (precision, recall, F-measure, analysis time).
• Raw analysis results outputed by the evaluated tool.

Status	GitHub Actions Workflow	Latests Evaluation Results
	Workflow FlowDroid April 2017 Nightly	Results FlowDroid April 2017 Nightly
	Workflow FlowDroid 2.7.1	Results FlowDroid 2.7.1
	Workflow Amandroid 3.1.2	Results Amandroid 3.1.2
	Workflow Amandroid 3.2.0	Results Amandroid 3.2.0

New contributions are welcome!

TaintBench serves as a starting point for automatic benchmarking of Android taint analysis tools. We welcome new contributions to TaintBench.

Contribute a new finding to an existing benchmark app

If you find more taint flows in a TaintBench benchmark app. Please document it with TB-Extractor in TAF-format and make a pull request to the GitHub repositiory of the benchmark app.

Contribute a new benchmark app to the TaintBench suite

Please contact us by leaving an issue here.