Assessing the accuracy of contact predictions in CASP13

Rojan Shrestha, Eduardo Fajardo, Nelson Gil, Krzysztof Fidelis, Andriy Kryshtafovych, Bohdan Monastyrskyy, Andras Fiser

Research output: Contribution to journalArticle

Abstract

The accuracy of sequence-based tertiary contact predictions was assessed in a blind prediction experiment at the CASP13 meeting. After 4 years of significant improvements in prediction accuracy, another dramatic advance has taken place since CASP12 was held 2 years ago. The precision of predicting the top L/5 contacts in the free modeling category, where L is the corresponding length of the protein in residues, has exceeded 70%. As a comparison, the best-performing group at CASP12 with a 47% precision would have finished below the top 1/3 of the CASP13 groups. Extensively trained deep neural network approaches dominate the top performing algorithms, which appear to efficiently integrate information on coevolving residues and interacting fragments or possibly utilize memories of sequence similarities and sometimes can deliver accurate results even in the absence of virtually any target specific evolutionary information. If the current performance is evaluated by F-score on L contacts, it stands around 24% right now, which, despite the tremendous impact and advance in improving its utility for structure modeling, also suggests that there is much room left for further improvement.

Original languageEnglish (US)
JournalProteins: Structure, Function and Bioinformatics
DOIs
StateAccepted/In press - Jan 1 2019

Fingerprint

Proteins
Data storage equipment
Experiments
Deep neural networks

Keywords

  • CASP13
  • contact prediction
  • protein structure modeling

ASJC Scopus subject areas

  • Structural Biology
  • Biochemistry
  • Molecular Biology

Cite this

Assessing the accuracy of contact predictions in CASP13. / Shrestha, Rojan; Fajardo, Eduardo; Gil, Nelson; Fidelis, Krzysztof; Kryshtafovych, Andriy; Monastyrskyy, Bohdan; Fiser, Andras.

In: Proteins: Structure, Function and Bioinformatics, 01.01.2019.

Research output: Contribution to journalArticle

Shrestha, Rojan ; Fajardo, Eduardo ; Gil, Nelson ; Fidelis, Krzysztof ; Kryshtafovych, Andriy ; Monastyrskyy, Bohdan ; Fiser, Andras. / Assessing the accuracy of contact predictions in CASP13. In: Proteins: Structure, Function and Bioinformatics. 2019.
@article{eabe1c0a14644a9d926cd27ea82cce6d,
title = "Assessing the accuracy of contact predictions in CASP13",
abstract = "The accuracy of sequence-based tertiary contact predictions was assessed in a blind prediction experiment at the CASP13 meeting. After 4 years of significant improvements in prediction accuracy, another dramatic advance has taken place since CASP12 was held 2 years ago. The precision of predicting the top L/5 contacts in the free modeling category, where L is the corresponding length of the protein in residues, has exceeded 70{\%}. As a comparison, the best-performing group at CASP12 with a 47{\%} precision would have finished below the top 1/3 of the CASP13 groups. Extensively trained deep neural network approaches dominate the top performing algorithms, which appear to efficiently integrate information on coevolving residues and interacting fragments or possibly utilize memories of sequence similarities and sometimes can deliver accurate results even in the absence of virtually any target specific evolutionary information. If the current performance is evaluated by F-score on L contacts, it stands around 24{\%} right now, which, despite the tremendous impact and advance in improving its utility for structure modeling, also suggests that there is much room left for further improvement.",
keywords = "CASP13, contact prediction, protein structure modeling",
author = "Rojan Shrestha and Eduardo Fajardo and Nelson Gil and Krzysztof Fidelis and Andriy Kryshtafovych and Bohdan Monastyrskyy and Andras Fiser",
year = "2019",
month = "1",
day = "1",
doi = "10.1002/prot.25819",
language = "English (US)",
journal = "Proteins: Structure, Function and Bioinformatics",
issn = "0887-3585",
publisher = "Wiley-Liss Inc.",

}

TY - JOUR

T1 - Assessing the accuracy of contact predictions in CASP13

AU - Shrestha, Rojan

AU - Fajardo, Eduardo

AU - Gil, Nelson

AU - Fidelis, Krzysztof

AU - Kryshtafovych, Andriy

AU - Monastyrskyy, Bohdan

AU - Fiser, Andras

PY - 2019/1/1

Y1 - 2019/1/1

N2 - The accuracy of sequence-based tertiary contact predictions was assessed in a blind prediction experiment at the CASP13 meeting. After 4 years of significant improvements in prediction accuracy, another dramatic advance has taken place since CASP12 was held 2 years ago. The precision of predicting the top L/5 contacts in the free modeling category, where L is the corresponding length of the protein in residues, has exceeded 70%. As a comparison, the best-performing group at CASP12 with a 47% precision would have finished below the top 1/3 of the CASP13 groups. Extensively trained deep neural network approaches dominate the top performing algorithms, which appear to efficiently integrate information on coevolving residues and interacting fragments or possibly utilize memories of sequence similarities and sometimes can deliver accurate results even in the absence of virtually any target specific evolutionary information. If the current performance is evaluated by F-score on L contacts, it stands around 24% right now, which, despite the tremendous impact and advance in improving its utility for structure modeling, also suggests that there is much room left for further improvement.

AB - The accuracy of sequence-based tertiary contact predictions was assessed in a blind prediction experiment at the CASP13 meeting. After 4 years of significant improvements in prediction accuracy, another dramatic advance has taken place since CASP12 was held 2 years ago. The precision of predicting the top L/5 contacts in the free modeling category, where L is the corresponding length of the protein in residues, has exceeded 70%. As a comparison, the best-performing group at CASP12 with a 47% precision would have finished below the top 1/3 of the CASP13 groups. Extensively trained deep neural network approaches dominate the top performing algorithms, which appear to efficiently integrate information on coevolving residues and interacting fragments or possibly utilize memories of sequence similarities and sometimes can deliver accurate results even in the absence of virtually any target specific evolutionary information. If the current performance is evaluated by F-score on L contacts, it stands around 24% right now, which, despite the tremendous impact and advance in improving its utility for structure modeling, also suggests that there is much room left for further improvement.

KW - CASP13

KW - contact prediction

KW - protein structure modeling

UR - http://www.scopus.com/inward/record.url?scp=85074557304&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85074557304&partnerID=8YFLogxK

U2 - 10.1002/prot.25819

DO - 10.1002/prot.25819

M3 - Article

C2 - 31587357

AN - SCOPUS:85074557304

JO - Proteins: Structure, Function and Bioinformatics

JF - Proteins: Structure, Function and Bioinformatics

SN - 0887-3585

ER -