AMPBenchmark allows benchmarking of models for antimicrobial (AMP) peptide prediction.
Follow the instructions in our GitHub repository: https://github.com/BioGenies/AMPBenchmark. The computations might take a bit of time.
Katarzyna Sidorczuk, Przemysław Gagat, Filip Pietluch, Jakub Kała, Dominik Rafacz, Laura Bąkała, Jadwiga Słowik, Rafał Kolenda, Stefan Rödiger, Legana C H W Fingerhut, Ira R Cooke, Paweł Mackiewicz, Michał Burdukiewicz, Benchmarks in antimicrobial peptide prediction are biased due to the selection of negative data, Briefings in Bioinformatics, 2022;, bbac343, https://doi.org/10.1093/bib/bbac343.
If you have any questions, suggestions or comments, contact Michal Burdukiewicz.
Each of heat maps represents an architecture, the x- and y-axis describe the training and benchmark method of negative data sampling, respectively. Each architecture was trained and benchmarked on five replicates of the training and benchmark sample. The mean value of AUC for the five replicates is indicated as shades of red, orange and yellow, and the standard deviation as black dots of varying sizes. The diagonals mark results for architectures trained and benchmarked on the data generated by the same sampling method.
This chart allows monitoring of the impact of the sampling method on a learning architecture method. The diagonal line represents all models with a similar performance regardless of the discrepancy between the sampling method for training and benchmarking. The closer the architecture is to that line, the more robust it is.