LiveBench-1: Continuous benchmarking of protein structure prediction servers

Janusz M. Bujnicki, Arne Elofsson, Daniel Fischer, Leszek Rychlewski

Research output: Contribution to journalArticlepeer-review

112 Scopus citations

Abstract

We present a novel, continuous approach aimed at the large-scale assessment of the performance of available fold-recognition servers. Six popular servers were investigated: PDB-Blast, FFAS, T98-lib, Gen-THREADER, 3D-PSSM, and INBGU. The assessment was conducted using as prediction targets a large number of selected protein structures released from October 1999 to April 2000. A target was selected if its sequence showed no significant similarity to any of the proteins previously available in the structural database. Overall, the servers were able to produce structurally similar models for one-half of the targets, but significantly accurate sequence-structure alignments were produced for only one-third of the targets. We further classified the targets into two sets: easy and hard. We found that all servers were able to find the correct answer for the vast majority of the easy targets if a structurally similar fold was present in the server's fold libraries. However, among the hard targets - where standard methods such as PSI-BLAST fail - the most sensitive fold-recognition servers were able to produce similar models for only 40% of the cases, half of which had a significantly accurate sequence-structure alignment. Among the hard targets, the presence of updated libraries appeared to be less critical for the ranking. An "ideally combined consensus" prediction, where the results of all servers are considered, would increase the percentage of correct assignments by 50%. Each server had a number of cases with a correct assignment, where the assignments of all the other servers were wrong. This emphasizes the benefits of considering more than one server in difficult prediction tasks. The LiveBench program (http://BioInfo.PL/LiveBench) is being continued, and all interested developers are cordially invited to join.

Original languageEnglish
Pages (from-to)352-361
Number of pages10
JournalProtein Science
Volume10
Issue number2
DOIs
StatePublished - 31 Mar 2001

Keywords

  • Automated protein structure prediction
  • Benchmarking
  • CAFASP
  • LiveBench
  • Meta server

ASJC Scopus subject areas

  • Biochemistry
  • Molecular Biology

Fingerprint

Dive into the research topics of 'LiveBench-1: Continuous benchmarking of protein structure prediction servers'. Together they form a unique fingerprint.

Cite this