Structure validation

Structure validation concept: model of a protein (each ball is an atom), and magnified region with electron density data and 3 bright flags for problems

Macromolecular structure validation is the process of evaluating reliability for 3-dimensional atomic models of large biological molecules such as proteins and nucleic acids. These models, which provide 3D coordinates for each atom in the molecule (see example in the image), come from structural biology experiments such as x-ray crystallography or nuclear magnetic resonance (NMR). The validation has three aspects: 1) checking on the validity of the thousands to millions of measurements in the experiment; 2) checking how consistent the atomic model is with those experimental data; and 3) checking consistency of the model with known physical and chemical properties.

Proteins and nucleic acids are the workhorses of biology, providing the necessary chemical reactions, structural organization, growth, mobility, reproduction, and environmental sensitivity. Essential to their biological functions are the detailed 3D structures of the molecules and the changes in those structures. To understand and control those functions, we need accurate knowledge about the models that represent those structures, including their many strong points and their occasional weaknesses.

End-users of macromolecular models include clinicians, teachers and students, as well as the structural biologists themselves, journal editors and referees, experimentalists studying the macromolecules by other techniques, and theoreticians and bioinformaticians studying more general properties of biological molecules. Their interests and requirements vary, but all benefit greatly from a global and local understanding of the reliability of the models.

Historical summary

Macromolecular crystallography was preceded by the older field of small-molecule x-ray crystallography (for structures with less than a few hundred atoms). Small-molecule diffraction data extends to much higher resolution than feasible for macromolecules, and has a very clean mathematical relationship between the data and the atomic model. The residual, or R-factor, measures the agreement between the experimental data and the values back-calculated from the atomic model. For a well-determined small-molecule structure the R-factor is nearly as small as the uncertainty in the experimental data (well under 5%). Therefore that one test by itself provides most of the validation needed, but a number of additional consistency and methodology checks are done by automated software as a requirement for small-molecule crystal structure papers submitted to the International Union of Crystallography (IUCr) journals such as Acta Crystallographica section B or C. Atomic coordinates of these small-molecule structures are archived and accessed through the Cambridge Structural Database (CSD) or the Crystallography Open Database (COD).

The first macromolecular validation software was developed around 1990, for proteins. It included Rfree cross-validation for model-to-data match, bond length and angle parameters for covalent geometry, and sidechain and backbone conformational criteria. For macromolecular structures, the atomic models are deposited in the Protein Data Bank (PDB), still the single archive of this data. The PDB was established in the 1970s at Brookhaven National Laboratory, moved in 2000 to the [1] (Research Collaboration for Structural Biology) centered at Rutgers, and expanded in 2003 to become the wwPDB (worldwide Protein Data Bank), with access sites in Europe ([2]) and Asia ([3]), and with NMR data handled at the BioMagResBank (BMRB) in Wisconsin.

Validation rapidly became standard in the field, with further developments described below. *Obviously needs expansion*

A large boost was given to the applicability of comprehensive validation for both x-ray and NMR as of February 1, 2008, when the worldwide Protein Data Bank (wwPDB) made mandatory the deposition of experimental data along with atomic coordinates. In 2012 strong forms of validation are being adopted for wwPDB deposition from recommendations of the wwPDB Validation Task Force committees for x-ray crystallography, for NMR, for SAXS (small-angle x-ray scattering), and for cryoEM (cryo-Electron Microscopy).

Validation for Crystallography (X-ray and Neutron)

See also: X-ray crystallography

Overall Considerations

Global vs Local Criteria

Many evaluation criteria apply globally to an entire experimental structure, most notably the resolution, the anisotropy or incompleteness of the data, and the residual or R-factor that measures overall model-to-data match (see below). Those help a user choose the most accurate among related Protein Data Bank entries to answer their questions. Other criteria apply to individual residues or local regions in the 3D structure, such as fit to the local electron density map or steric clashes between atoms. Those are especially valuable to the structural biologist for making improvements to the model, and to the user for evaluating the reliability of that model right around the place they care about - such as a site of enzyme activity or drug binding. Both types of measures are very useful, but although global criteria are easier to state or publish, local criteria make the greatest contribution to scientific accuracy and biological relevance. As expressed in the Rupp textbook, "Only local validation, including assessment of both geometry and electron density, can give an accurate picture of the reliability of the structure model or any hypothesis based on local features of the model."

What can be seen in low vs high resolution macromolecular crystal structures

Model Validation

Conformation (Dihedrals): Protein & RNA

The backbone and sidechain dihedral angles of protein and RNA have been shown to have specific combinations of angles which are allowed.

Software and Websites

• PROSESS - Protein Structure Evaluation Suite & Server
• Resolution by Proxy, ResProx - protein model resolution-by-proxy
• VADAR - Volume, Area, Dihedral Angle Reporter
• rcsbPDB validation/deposition site
• MolProbity web service
• Protein structure validation database PDBREPORT.
• EDS (Electron Density Server)[20]
• What_Check software
• ProCheck software
• Coot modeling software (built-in validation)[21]
• OOPS2, part of the Uppsala Software Factory
• ProSA web service
• Verify-3D profile analysis
• PDB_REDO optimized X-ray structure models[22]

For NMR (Nuclear Magnetic Resonance)

Data Validation: Chemical Shifts, NOEs, RDCs

PROSESS. PROSESS (Protein Structure Evaluation Suite & Server) is the first web server that offered an assessment of protein structural models by NMR chemical shifts as well as NOEs, geometrical, and knowledge-based parameters.

AVS. Assignment validation suite (AVS) checks the chemical shifts list in BioMagResBank (BMRB) format for problems.

LACS. Linear analysis of chemical shifts is used for absolute referencing of chemical shift data.

Model-to-Data Validation

TALOS+. Predicts protein backbone torsion angles from chemical shift data. Frequently used to generate further restraints applied to a structure model during refinement.

Model Validation: As Above

NMR structural ensemble for PDB file 2K5D, with well-defined structure for the beta strands (arrows) and undefined, presumably highly mobile regions for the orange loop and the blue N-terminus

Dynamics: Core vs Loops, Tails, and Mobile Domains

One of the critical needs for NMR structural ensemble validation is to distinguish well-determined regions (those that have experimental data) from regions that are highly mobile and/or have no observed data. There are several current or proposed methods for making this distinction such as Random Coil Index, but so far the NMR community has not standardized on one.

Software and Websites

• PROSESS - Protein Structure Evaluation Suite & Server
• ResProx - protein model resolution-by-proxy
• VADAR - Volume, Area, Dihedral Angle Reporter
• PSVS (Protein Structure Validation Server at the NESG)[23]
• CING (Common Interface for NMR structure Generation) software
• ProCheckNMR software[24]
• MolProbity (includes analyses for NMR)
• TALOS+ Software & Server(server for predicting protein backbone torsion angles from chemical shift)

For Cryo-Electron Microscopy and Hybrid Methods

Software and Websites

• EM Data Bank, for EM map deposition
• EMDB at the PDB, info on ftp download of maps

For Computational Biology

It is difficult to do meaningful validation of an individual, purely computational, macromolecular model in the absence of experimental data for that molecule, because the model with the best geometry and conformational score may not be the one closest to the right answer. Therefore, much of the emphasis in validation of computational modeling is in assessment of the methods. To avoid bias and wishful thinking, double-blind prediction competitions have been organized, the original example of which (held every 2 years since 1994) is CASP (Critical Assessment of Structure Prediction) to evaluate predictions of 3D protein structure for newly solved crystallographic or NMR structures held in confidence until the end of the relevant competition. The major critierion for CASP evaluation is a weighted score called GDT-TS for the match of Calpha positions between the predicted and the experimental models.

Software and Websites

• CASP experiments home page

Further reading

• Cavanagh, John, Fairbrother, Wayne J., Palmer, Arthur G. III, Skelton, Nicholas J. (2006). Protein NMR Spectroscopy: Principles and Practice (2nd ed.). Academic Press. ISBN 0-12-164491-X.{{cite book}}: CS1 maint: multiple names: authors list (link)
• Rupp, Bernhard (2009). Biomolecular Crystallography: Principles, Practice, and Application to Structural Biology. Garland Science. ISBN 0815340818. {{cite book}}: Invalid |ref=harv (help)