CRAM (file format)
CRAM was designed to be an efficient reference-based alternative to the Sequence Alignment Map (SAM) and Binary Alignment Map (BAM) file formats. It optionally uses a genomic reference to describe differences between the aligned sequence fragments and the reference sequence, reducing storage costs. Additionally each column in the SAM format is separated into its own blocks, improving compression ratio. CRAM files typically vary from 30 to 60% smaller than BAM, depending on the data held within them.
The basic structure of a CRAM file is a series of containers, the first of which holds a compressed copy of the SAM header. Subsequent containers consist of a container Compression Header followed by a series of slices which in turn hold the alignment records themselves, formatted as a series of blocks.
Magic number Container
Slice ... Slice
Block Block ... Block
CRAM constructs records from a set of data series, describing the components of an alignment. The container Compression Header specifies which data series is encoded in which block, what codec will be used, and any codec specific meta-data (for example a table of Huffman symbol code lengths). While data series can be mixed together within the same block, keeping them separate usually improves compression and provides the opportunity for efficient selective decoding where only some data types are required.
Selective access to a CRAM file is granted via the index (with file-name suffix ".crai"). On chromosome and position sorted data this indicates which region is covered by each slice. On unsorted data the index may be used to simply fetch the Nth container. Selective decoding may also be achieved using the Compression Header to skip specified data series if partial records are required.
|2010-11||pre-CRAM||Initial paper describing the reference based format. This did not use the name CRAM. This software was implemented in Python as a prototype and demonstration of the basic concepts.|
|2011-12||0.3 - 0.86||Vadim Zalunin of the European Bioinformatics Institute (EBI) produced the first implementation named CRAM as a package called CRAMtools, written in the Java programming language.|
|2012||1.0||Implemented in Java CRAMtools.|
|2013||C implementation added to the Scramble tool, by James Bonfield of the Wellcome Sanger Institute.|
|2013||2.0||Changes included support for more than one reference per slice (useful with highly fragmented assemblies), better encoding of SAM auxiliary tags, splitting soft-clip and inserted bases into their own data-series, meta-data to track the number of records and bases per slice, and corrections to the BF (BAM flag) data-series.|
|2013||Added to htslib (0.2.0).|
|2014||2.1||Added EOF blocks, to help identify truncated files.|
|2014||Added to htsjdk (1.127).|
|2014||3.0||Inclusion of lzma and rANS codecs for block compression, along with multiple checksums for ensuring data integrity|
CRAM version 4.0 exists as a prototype in Scramble, initially demonstrated in 2015, but has yet to be adopted as a standard.
- SAM (file format)
- Binary Alignment Map
- Compression of Genomic Re-Sequencing Data
- List of file formats for molecular biology
- Hsi-Yang Fritz, Markus; Leinonen, Rasko; Cochrane, Guy; Birney, Ewan (May 2011). "Efficient storage of high throughput DNA sequencing data using reference-based compression". Genome Research. 21 (5): 734–740. doi:10.1101/gr.114819.110. ISSN 1549-5469. PMC 3083090. PMID 21245279.
- "Htsjdk by Broad Institute". samtools.github.io. Retrieved 2018-10-14.
- "Samtools". www.htslib.org. Retrieved 2018-10-14.
- Bonfield, James K. (2014-06-14). "The Scramble conversion tool". Bioinformatics. 30 (19): 2818–2819. doi:10.1093/bioinformatics/btu390. ISSN 1460-2059. PMC 4173023. PMID 24930138.
- "GA4GH". www.ga4gh.org. Retrieved 2018-10-14.
- EMBL-EBI. "CRAM toolkit < Software < European Nucleotide Archive < EMBL-EBI". www.ebi.ac.uk. Retrieved 2018-10-14.
- "vadimzalunin/crammer". GitHub. 2017-08-08. Retrieved 2018-10-14.
- "CRAM 1.0 Specification" (PDF).
- "enasequence/cramtools". GitHub. 2018-10-02. Retrieved 2018-10-14.
- "jkbonfield/io_lib". GitHub. 2018-10-16. Retrieved 2018-10-14.
- "CRAM 2.1 Specification" (PDF).
- "CRAM 3.0 Specification" (PDF).