Loading a single sequence from a file

In this case, the filename suffix is used to infer the data format.

from cogent3 import load_seq

seq = load_seq("data/mycoplasma-genitalium.fa", moltype="dna")
seq

	0
NC_000908.2 Mycoplasmoides genitalium G37, complete sequence	TAAGTTATTATTTAGTTAATACTTTTAACAATATTATTAAGGTATTTAAAAAATACTATT

DnaSequence, length=580,076 (truncated to 60)

Warning

If a file has more than one sequence, only the first one is loaded.

seq = load_seq("data/brca1-bats.fasta", moltype="dna")
seq

	0
FlyingFox	TGTGGCACAAATGCTCATGCCAGCTCTTTACAGCATGAGAAC---AGTTTATTATACACT

DnaSequence, length=3,009 (truncated to 60)

Note

It’s also possible to load a sequence from a url.

Directly use the fasta format parser to load a sequence

The cogent3 parsers return standard Python data types. The iter_fasta_records() is a generator, so it yields one record at a time. Because I know there’s a single sequence in this file, I wrap the call with list and select the first record.

from cogent3.parse.fasta import iter_fasta_records

label, seq = list(iter_fasta_records("data/mycoplasma-genitalium.fa"))[0]
label, seq[:10]

('NC_000908.2 Mycoplasmoides genitalium G37, complete sequence', 'TAAGTTATTA')

You can provide a converter that will transform the sequence data to the type you want. In this example, we use a cogent3 builtin to return a numpy array of unsigned 8-bit integers. We first get all the IUPAC characters for DNA and construct the converter. The converter maps an integer the provided characters in their order of occurrence in dna_alpha.

import numpy

from cogent3.core.alphabet import bytes_to_array
from cogent3.core.moltype import DNA

dna_alpha = "".join(DNA.most_degen_alphabet())
converter = bytes_to_array(dna_alpha.encode("utf8"), delete=b"\r\n\t ", dtype=numpy.uint8)

Note

The characters provided to the delete argument are white space and essential to ensure line feeds are removed.

We then use the parser as before but provide our custom converter.

label, seq = list(iter_fasta_records("data/mycoplasma-genitalium.fa", converter=converter))[0]
label, seq[:10]

('NC_000908.2 Mycoplasmoides genitalium G37, complete sequence',
 array([0, 2, 2, 3, 0, 0, 2, 0, 0, 2], dtype=uint8))

Directly use the genbank format parser to load a sequence and annotations

The cogent3 parsers return standard Python data types. The iter_genbank_records() is a generator, so it yields one record at a time. Because I know there’s a single sequence in this file, I wrap the call with list and select the first record.

from cogent3.parse.genbank import iter_genbank_records

label, seq, anns = list(iter_genbank_records("data/mycoplasma-genitalium.gb"))[0]
label, seq[:10], anns.keys()

('NC_000908',
 'TAAGTTATTA',
 dict_keys(['locus', 'length', 'mol_type', 'topology', 'db', 'date', 'definition', 'accession', 'version', 'dblink', 'keywords', 'source', 'species', 'taxonomy', 'references', 'comment', 'features', 'contig']))

As the output indicates, variable anns is a dictionary. The features in the GenBank feature table are available as a list under the "features" key. (See getting GenBank features as primitives.)

Directly use the fastq format parser to load reads with quality scores

The iter_fastq_records() generator yields (label, sequence, quality) tuples for each record in a fastq file. By default both the sequence and quality strings are decoded to str.

from cogent3.parse.fastq import iter_fastq_records

for label, seq, qual in iter_fastq_records("data/fastq.txt"):
    ...  # do your work here

To get a transformation of read quality into Phred scores, pass a converter built with make_qual_converter to get quality as a numpy.uint8 array. The scoring scheme can be selected by a PhredEncoding member or by name ("phred+33" or "phred+64", case insensitive).

from cogent3.core.alphabet import PhredEncoding, make_qual_converter

qual_converter = make_qual_converter(PhredEncoding.PHRED_33)
label, seq, qual = next(
    iter(iter_fastq_records("data/fastq.txt", qual_converter=qual_converter))
)
label, seq, qual

('SIM:1:FCX:1:15:6329:1045 1:N:0:ATCCGA',
 'GCTCAGCATCGAGAAGCTTAGCAACTTGGCAACGT',
 array([40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
        40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
        40], dtype=uint8))

Note

As with the fasta and genbank iterative parsers, a converter argument exists for providing a custom transformer to be applied to the sequence data too.

Loading sequence collections from a file or url

Loading aligned sequences

Any file in which the sequences have exactly the same length can be loaded as an alignment.

from cogent3 import load_aligned_seqs

aln = load_aligned_seqs("data/long_testseqs.fasta", moltype="dna")
type(aln)

cogent3.core.alignment.Alignment

Note

The load functions record the origin of the data in a .source attribute.

aln.source

'data/long_testseqs.fasta'

Loading unaligned sequences

Files containing sequences that may differ in length can be loaded using load_unaligned_seqs(), which returns a sequence collection.

from cogent3 import load_unaligned_seqs

seqs = load_unaligned_seqs("data/long_testseqs.fasta", moltype="dna")
type(seqs)

cogent3.core.alignment.SequenceCollection

Loading unaligned sequences from multiple files

You can create a single sequence collection containing sequences from all files in a directory that match a wildcard (or glob) pattern (e.g. "path/to/dir/*.<filename suffix>"). We load data from files ending with .fa using load_unaligned_seqs(). This approach can be taken for all supported sequence file formats.

from cogent3 import load_unaligned_seqs

seqs = load_unaligned_seqs("data/*.fa", moltype="dna")
seqs

	0
I dna:chromosome chromosome:WBcel235:I:1:15072434:1 REF	GCCTAAGCCTAAGCCTAAGCCTAAGCCTAAGCCTAAGCCTAAGCCTAAGCCTAAGCCTAA
NC_000908.2 Mycoplasmoides genitalium G37, complete sequence	TAAGTTATTATTTAGTTAATACTTTTAACAATATTATTAAGGTATTTAAAAAATACTATT
Mouse Lemur	AAACAGTACGTTGATGGAGATCAGTGTGAGTCCAATCCGTGTTTAAATGGTGGCAGTTGC
Orangutan	AGGGCTGGGAGCGTGCTTTCCACGACGGTGACACGCTTCCCTGGATTGGGTAAGCTCCTG

4 x {min=150, median=299542.5, max=15072434} dna sequence collection

Note

The function limits loading to just one sequence per file.

Loading from a url

The cogent3 load functions support loading from a url. We load the above fasta file directly from GitHub.

from cogent3 import load_aligned_seqs

aln = load_aligned_seqs(
    "https://raw.githubusercontent.com/cogent3/cogent3/develop/doc/data/long_testseqs.fasta",
    moltype="dna",
)

Discovering the supported file formats

Use available_seq_formats() to see the formats that load_seq(), load_aligned_seqs() and load_unaligned_seqs() can read. The “suffixes” column lists the filename suffixes that map to each format.

from cogent3 import available_seq_formats

available_seq_formats()

Specify a format by its name or a file by one of its suffixes.

name	suffixes	aligned	unaligned
c3h5a	c3h5a	True	False
c3h5s	c3h5s	True	False
c3h5u	c3h5u	False	True
clustal	aln, clustal	True	True
fasta	fa, faa, fasta, fna, mfa	True	True
gbseq	gbseq	True	True
gde	gde	True	True
genbank	gb, gbff, gbk, genbank	True	True
msf	msf	True	True
nexus	nex, nexus, nxs	True	True
paml	paml	True	True
phylip	phy, phylip	True	True
tinyseq	tinyseq	True	True

13 rows x 4 columns

Specifying the file format

The loading functions use the filename suffix to infer the file format. This can be overridden using the format argument.

from cogent3 import load_aligned_seqs

aln = load_aligned_seqs("data/long_testseqs.fasta", moltype="dna", format_name="fasta")
aln

	0
DogFaced	TGTGGCACAAATACTCATGCCAACTCATTACAGCATGAGAACAGCAGTTTATTATACACT
Human	......................G...............................CT....
HowlerMon	......................G...........................G...CT....
Mouse	.........G..G.........G...........C.....C............GCT..T.
NineBande	.........................T............................CT....

5 x 2532 (truncated to 5 x 60) dna alignment

Specifying the sequence molecular type

from cogent3 import make_aligned_seqs

protein_seqs = {"seq1": "DEKQL-RG", "seq2": "DDK--SRG"}
proteins_loaded = make_aligned_seqs(protein_seqs, moltype="protein")
proteins_loaded.moltype
proteins_loaded

	0
seq1	DEKQL-RG
seq2	.D.--S..

2 x 8 protein alignment

Making an alignment from standard python objects

From a dict of strings

from cogent3 import make_aligned_seqs

seqs = {"seq1": "AATCG-A", "seq2": "AATCGGA"}
seqs_loaded = make_aligned_seqs(seqs, moltype="dna")

From a dict of numpy arrays

from cogent3 import make_aligned_seqs
from numpy import array, uint8

seqs = {
    "seq1": array([2, 2, 0, 1, 3, 9, 2], dtype=uint8),
    "seq2": array([2, 2, 0, 1, 3, 3, 2], dtype=uint8),
}
seqs_loaded = make_aligned_seqs(seqs, moltype="dna")

From a series of strings

The sequence names will be automatically created.

from cogent3 import make_aligned_seqs

data = ["AATCG-A", "AATCGGA"]
coll = make_aligned_seqs(data, moltype="dna")
coll

	0
seq_1	AATCGGA
seq_0	.....-.

2 x 7 dna alignment

Changing sequence labels on loading

Load a list of aligned nucleotide sequences, while specifying the DNA molecule type and stripping the comments from the label. In this example, we rename sequences by passing a function that removes everything after the first whitespace to the label_to_name parameter.

from cogent3 import make_aligned_seqs

data = {
    "sample1 Mus musculus": "AACCTGC--C",
    "sample2 Gallus gallus": "AAC-TGCAAC",
}
loaded_seqs = make_aligned_seqs(
    data, moltype="dna", label_to_name=lambda x: x.split()[0]
)
loaded_seqs

	0
sample2	AAC-TGCAAC
sample1	...C...--.

2 x 10 dna alignment

Making a sequence collection from standard python objects

This is done using make_unaligned_seqs(), which returns a SequenceCollection instance. The function arguments match those of make_aligned_seqs(). We demonstrate only for the case where the input data is a dict.

from cogent3 import make_unaligned_seqs

seqs = {"seq1": "AATCA", "seq2": "AATCGGA"}
seqs = make_unaligned_seqs(seqs, moltype="dna")
seqs

	0
seq1	AATCA
seq2	AATCGGA

2 x {min=5, median=6.0, max=7} dna sequence collection

Loading sequences using format parsers

load_aligned_seqs() and load_unaligned_seqs() are just convenience interfaces to format parsers. It can sometimes be more effective to use the parsers directly, say when you don’t want to load everything into memory.

Loading FASTA sequences from an open file or list of lines

To load FASTA formatted sequences directly, you can use iter_fasta_records. This parser returns data as python strings.

Note

This returns the sequences as strings.

from cogent3.parse.fasta import iter_fasta_records

seqs = list(iter_fasta_records("data/long_testseqs.fasta"))
seqs

[('Human',
  'TGTGGCACAAATACTCATGCCAGCTCATTACAGCATGAGAACAGCAGTTTATTACTCACTAAAGACAGAATGAATGTAGAAAAGGCTGAATTCTGTAATAAAAGCAAACAGCCTGGCTTAGCAAGGAGCCAACATAACAGATGGGCTGGAAGTAAGGAAACATGTAATGATAGGCGGACTCCCAGCGAAAAAAAGGTAGATCTGAATGCTGATCCCCTGTGTGAGAGAAAAGAATGGAATAAGCAGAAACTGCCATGCTCAGAGAATCCTAGAGATACTGAAGATGTTCCTTGGATAACACTAAATAGCAGCATTCAGAAAGTTAATGAGTGGTTTTCCAGAAGTGATGAACTGTTAGGTTCTGATGACTCACATGATGGGGAGTCTGAATCAAATGCCTTGGACGTTCTAAATGAGGTAGATGAATATTCTGGTTCTTCAGAGAAAATAGACTTACTGGCCAGTGATCCTCATGAGGCTTTAATATGTGAAAGAGTTCACTCCAAATCAGTAGAGAGTAATATTGAAGACAAAATATTTGGGAAAACCTATCGGAAGAAGGCAAGCCTCCCCAACTTAAGCCATGTAACTGAAATTATAGGAGCATTTGTTACTGAGCCACAGATAATACAAGAGCGTCCCCTCACAAATAAATTAAAGCGTAAAAGGACATCAGGCCTTCATCCTGAGGATTTTATCAAGAAAGCAGATTTGGCAGTTCAAAAGACTCCTGAAATGATAAATCAGGGAACTAACCAAACGGAGCAGAATGGTCAAGTGATGAATATTACTAATAGTGGTCATGAGAATAAAACAAAAGGTGATTCTATTCAGAATGAGAAAAATCCTAACCCAATAGAATCACTCGAAAAAGAATCTTTCAAAACGAAAGCTGAACCTATAAGCAGCAGTATAAGCAATATGGAACTCGAATTAAATATCCACAATTCAAAAGCACCTAAAAAGAATCTGAGGAGGAAGTCTACCAGGCATATTCATGCGCTTGAACTAGTCAGTAGAAATCTAAGCCCACCTAATTGTACTGAATTGCAAATTGATAGTTGTTCTAGCAGTGAAGAGATAAAGAAAAAAAAGTACAACCAAATGCCAGTCAGGCACAGCAGAAACCTACAACTCATGGAAGGTAAAGAACCTGCAACTGGAGCCAAGAAGAACAAGCCAAATGAACAGACAAGTAAAAGACATGACAGCGATACTTTCCCAGAGCTGAAGAATGCACCTGGTTCTTTTACTAAGTGTTCAAATACCAGTGAACTTAAAGAATTTAATCCTAGCCTTCCAAGAGAAGAAAAAGAGAAACTAGAAACAGTTAAAGTGTCTAATAATGCTGAAGACCCCAAAGATCTCATGTTAAGTGGAGAAAGGGTTTTGCAAACTGAAAGATCTGTAGAGAGTAGCAGTATTTCATTGGTACCTGGTACTGATTATGGCACTCAGGAAAGTATCTCGTTACTGGAAGTTAGCACTCTAGGGAAGGCAAAAACAGAACCAAATAAATGTGTGAGTCAGTGTGCAGCATTTGAAAACCCCAAGGGACTAATTCATGGTTCCAAAGATAATAGAAATGACACAGAAGGCTTTAAGTATCCATTGGGACATGAAGTTAACCACTCAAATCCAGAAGAGGAATGTGCACACTCTGGGTCCTTAAAGAAACAAAGTCCAAAAGTCACTTTTGAATGTGAACAAAAGGAAAATCAAGGAAAGAATGAGTCTAATAAGCCTGTACAGACAGTTAATATCACTGCAGGCTTTCCTGTGGTTGGTCAGAAAGATAAGCCAGTTGATAATGCCAAATGTAAAGGAGGCTCTAGGTTTTGTCTATCATCTCAGTTCAGAGGCAACGAAACTGGACTCATTACTCCAAATAAACATGGACTTTTACAAAACCCATATCGTATACCACCACTTTTTCCCATCAAGTCATTTGTTAAAACTAAATGTAAGAAAAATCTGCTAGAGGAAAACTTTGAGGAACATTCAATGTCACCTGAAAGAGAAATGGGAAATGAGAACATTCCAAGTACAGTGAGCACAATTAGCCGTAATAACAGAGAAAATGTTTTTAAAGAAGCCAGCTCAAGCAATATTAATGAAGTAGGTTCCAGTGATGAAAACATTCAAGCAGAACTAGGTAGAAACAGAGGGCCAAAATTGAATGCTATGCTTAGATTAGGGGTTTTGCAACCTGAGGTCTATAAACAAAGTCTTCCTGGAAGTAATAAGCATCCTGAAATAAAAAAGCAAGAAGTTCAGACTGTTAATACAGATTTCTCTCCACTGATTTCAGATAACTTAGAACAGCCTATGAGTAGTCATGCATCTCAGGTTTGTTCTGAGACACCTGATGACCTGTTAGATGATGGTGAAATAAAGGAAGATACTAGTTTTGCTGAAAATGACATTAAGGAAAGTTCTGCTGTTTTTAGCAAAAGCGTCCAGAAAGGAGAGCTTAGCAGGAGTCCTAGCCCTTTCACCCATACACATTTGGCTCAGGGTTACCGAAGAGGG'),
 ('HowlerMon',
  'TGTGGCACAAATACTCATGCCAGCTCATTACAGCATGAGAACAGCAGTTTGTTACTCACTAAAGACACACTGAATGTAGAAAAGGCTGAATTCTGTAATAAAAGCAAACAGCCTGGCTTAGCAAGGAGCCAACATAACAGATGGGCTGAAAGTGAGGAAACATGTAATGATAGGCAGACTCCCAGCGAGAAAAAGGTAGATGTGGATGCTGATCCCCTGCATGGGAGAAAAGAATGGAATAAGCAGAAACCTCCGTGCTCTGAGAATCCTAGAGATACTGAAGATGTTGCTTGGATAATGCTAAATAGCAGCATTCAGAAAGTTAATGAGTGGTTTTCCAGAAGTGATGAACTGTTAACTTCTGATGACTCACATGATGGGGGGTCTGAATCAAATGCCTTGGAAGTTCTAAATGAGGTAGATGGATATTCTAGTTCTTCAGAGAAAATAGACTTACTGGCCAGTGATCCTCATGATCATTTGATATGTGAAAGAGTTCACTGCAAATCAGTAGAGAGTAGTATTGAAGATAAAATATTTGGGAAAACCTATCGGAGGAAGGCAAGCCTCCCTAACTTGAGCCACGTAACTGAAATTATAGGAGCATTTGTTACTGAGCCACAGATAATACAAGAGCATCCTCTCACAAATAAATTAAAGCGTAAAAGGACATCAGGACTTCATCCTGAGGATTTTATCAAGAAAGCAGATTTGGCAGTTCAAAAGACTCCTGAAAAGATAAATCAGGGAACTAACCAAACAGAGCGGAATGATCAAGTGATGAATATTACTAACAGTGGTCATGAGAATAAAACAAAAGGTGATTCTATTCAGAATGAGAACAATCCTAACCCAGTAGAATCACTGGAAAAAGAATCATTCAAAAGTAAAGCTGAACCTATAAGCAGTAGTATAAGCAATATGGAATTAGAATTGAATGTCCACAATTCCAAAGCATCTAAAAAGAATCTGAGAAGGAAGTCTACCAGGCATATTCATGAGCTTGAACTAGTCAGTAGAAATCTAAGCCCACCTAATTATACTGAAGTACAAATTGATAGTTGTTCTAGCAGTGAAGAGATAAAGAAAAAAAATTACAACCAAATGCCAGTCAGGCACAGCAGAAAGCTACAACTCATGGAAGATAAAGAACGTGCAGCTAGAGCCAAAAAGAGCAAGCCAAATGAACAAACAAGTAAAAGACATGCCAGTGATACTTTCCCAGAACTGAGGAACATACCTGGTTCTTTTACTAACTGTTCAAATACTAATGAATTTAAAGAATTTAATCCTAGCCTTCCAAGAGAACAAACAGAGAAACTAGAAACAGTTAAACTGTCTAATAATGCCAAAGACCCCAAAGATCTCATGTTAAGTGGAGAAAGTGTTTTGCAAATTGAAAGATCTGTAGAGAGTAGCAGTATTTTGTTGATACCTGGTACTGATTATGGCACTCAGGAAAGTATCTCATTACTGGAAGTTAGCACTCTGGGGAAGGCAAAAACAGAACCAAATAAATGTGTGAGTCAGTGTGCAGCATTTGAAAACCCCAAGGAACTAATTCATGGTTCTAAAGATACTAGAAATGGCACAGAAGGCTTGAAGTATCCATTGGGACCTGAAGTTAACTACTCAAATCCAGAAAAGGAATGTGCATGCTCTAGGTCCTTAAAGAAACAAAGTCCAAAGGTCACTCCTGAATGTGAACAAAAGGAAAATCAAGGAGAGAAAGAGTCTAATGAGCTTGTAGAGACAGTTAATACCACTGCAGGCTTTCCTATGGTTTGTCAGAAAGATAAGCCAGTTGATTATGCCAGATGTGAAGGAGGCTCTAGGCTTTGTCTATCATCTCAGTTCAGAGGCAACGAAACTGGACTCATTATTCCAAATAAACATGGACTTTTACAGAACCCATATCATATGTCACCGCTTATTCCCACCAGGTCATTTGTTAAAACTAAATGTAAGAAAAACCTGCTAGAAGAAAACTCTGAGGAACATTCAATGTCACCTGAAAGAGCAATGGGAAACAAGAACATTCCAAGTACAGTGAGCACAATTAGCCATAATAACAGAGAAAATGCTTTTAAAGAAACCAGCTCAAGCAGTATTTATGAAGTAGGTTCCAGTGATGAAAACATTCAAGCAGAGCTAGGTAGAAACAGAAGGCCAAAATTGAATGCTATGCTTAGATTAGGGCTTCTGCAACCTGAGATTTGTAAGCAAAGTCTTCCTATAAGTGATAAACATCCTGAAATTAAAAAGCAAGAAGTTCAGACTGTTAATACAGACGTCTCTCTACTGATTTCATATAACCTAGAACAGCATATGAGCAGTCATACATCTCAGGTTTGTTCTGAGACACCTGACAACCTGTTAGATGATGGTGAAATAAAGGAAGATACTAGTTTTGCTGAATATGGCATTAAGGAGACTTCTACTGTTTTTAGCAAAAGTGTCCAGAGAGGAGAGCTCAGCAGGAGCCCTAGCCCTTTCACCCATACACATTTGGCTCAGGTTTACCAAAGAGGG'),
 ('Mouse',
  'TGTGGCACAGATGCTCATGCCAGCTCATTACAGCCTGAGACCAGCAGTTTATTGCTCATTGAAGACAGAATGAATGCAGAAAAGGCTGAATTCTGTAATAAAAGCAAACAGCCTGGCATAGCAGTGAGCCAGCAGAGCAGATGGGCTGCAAGTAAAGGAACATGTAACGACAGGCAGGTTCCCAGCGGGGAAAAGGTAGGTCCAAACGCTGACTCCCTTAGTGATAGAGAGAAGTGGACTCACCCGCAAAGTCTGTGCCCTGAGAATTCTGGAGCTACCACCGATGTTCCTTGGATAACACTAAATAGCAGCGTTCAGAAAGTTAATGAGTGGTTTTCCAGAACTGGTGAAATGTTAACTTCTGACAGCGCATCTGCCAGGAGGCACGAGTCAAATGCTTTGGAAGTTTCAAACGAAGTGGATGGGGGTTTTAGTTCTTCAAGGAAAACAGACTTAGTAACCCCCGACCCCCATCATACTTTAATGTGTGGAAGAGACTTCTCCAAACCAGTAGAGGATAATATCAGTGATAAAATATTTGGGAAATCCTATCAGAGAAAGGGAAGCCGCCCTCACCTGAACCATGTGACTGAAATTATAGGCACATTTATTACAGAACCACAGATAACACAAGAGCAGCCCTTCACAAATAAATTAAAACGTAAGAGAAGTACATCCCTTCAACCTGAGGACTTCATCAAGAAAGCAGATTCAGCAGGTCAAAGGACTCCTGACAACATAAATCAGGGAACTGACCTAATGGAGCCAAATGAGCAAGCAGTGAGTACTACCAGTAACTGTCAGGAGAACAAAATAGCAGGTAGTAATCTCCAGAAAGAGAAAAGCGCTCATCCAACTGAATCATTGAGAAAGGAACCTTCCACAGCAGGAGCCAAATCTATAAGCAACAGTGTAAGTGATTTGGAGGTAGAATTAAACGTCCACAGTTCAAAAGCACCTAAGAAAAATCTGAGGAGGAAGTCTATCAGGTGTGCTCTTCCACTTGAACCAATCAGTAGAAATCCAAGCCCACCTACTTGTGCTGAGCTTCAAATCGATAGTTGTGGTAGCAGTGAAGAAACAAAGAAAAACCATTCCAACCAACAGCCAGCCGGGCACCTTAGAGAGCCTCAACTCATCGAAGACACTGAACCTGCAGCGGATGCCAAGAAGAACGAGCCAAATGAACACATAAGGAAGAGACGTGCCAGCGATGCTTTCCCAGAAGAGAAAAACAAAGCTGGTTTATTAACTAGCTGTTCAAGTCCTAGAAAATCTCAAGGGCCTAATCCCAGCCCTCAGAGAACAGGAACAGAGCAACTTGAAACACGCCAAATGTCTGACAGTGCCAAAGAACTCGGGGATCGGGTCCTAGGAGGAGAGCCCAGTGGCAAAACTGACCGATCTGAGGAGAGCACCAGCGTATCCTTGGTACCTGACACTGACTACGACACTCAGAACAGTGTCTCAGTCCTGGACGCTCACACTGTCAGATATGCAAGAACAGGATCCGCTCAGTGTATGACTCAGTTTGTAGCAAGCGAAAACCCCAAGGAACTCGTCCATGGCTCTAACAATGCTGGGAGTGGCACAGAGGGTCTCAAGCCCCCCTTGAGACACGCGCTTAACCTCTCAAAACCTCAAAAGGACTGTGCTCACTCTGTGCCCTCAAAGGAACTGAGTCCAAAGGTGACAGCTAAAGGTAAACAAAAAGAACGTCAGGGACAGGAAGAATTTGAAAGTCACGTACAAGCAGTTGCGGCCACAGTGGGCTTACCTGTGCCCTGTCAAGAAGGTAAGCTAGCTGCTGATACAATGTGTGATAGAGGTTGTAGGCTTTGTCCATCATCTCATTACAGAAGCGGGGAGAATGGACTCAGCGCCACAGGTAAATCAGGAATTTCACAAAACTCACATTTTAAACAATCAGTTTCTCCCATCAGGTCATCTATAAAAACTGACAATAGGAAACCTCTGACAGAGGGACGATTTGAGAGACATACATCATCAACTGAGATGGCGGTGGGAAATGAGAACCTTCAGAGTACAGTGCACACAGTTAGCCTGAATAACAGAGGAAATGCTTGTCAAGAAGCCGGCTCGGGCAGTATTCATGAAGTATGTTCCACTGGTGACTCCTTCCCAGGACAACTAGGTAGAAACAGAGGGCCTAAGGTGAACACTGTGCCTCCATTAGATAGTATGCAGCCTGGTGTCTGTCAGCAAAGTGTTCCTGTAAGTGATAAGTATCTTGAAATAAAAAAGCAGGAGGGTGAGGCTGTCTGTGCAGACTTCTCTCCACTATTCTCAGACCATCTTGAGCAATCTATGAGTGGTAAGGTTTTTCAGGTTTGCTCTGAGACACCTGATGACCTGCTGGATGATGTTGAAATACAGGGACATACTAGCTTTGGTGAAGGTGACATAATGGAGAGATCTGCTGTCTTTAACGGAAGCATCCTGAGAAGGGAGTCCAGTAGGAGCCCTAGTCCTGTAACCCATGCATCGAAGTCTCAGAGTCTCCACAGAGCG'),
 ('NineBande',
  'TGTGGCACAAATACTCATGCCAACTTATTACAGCATGAGAACAGCAGTTTATTACTCACTAAAGACAGAATGAATGTAGAAAAGGCTGAATTCTGTAATAAAAGCAAACAGCCTGGCTTAGCAAGGCGCCAACAGAGCAGATGGGCTGAAAGTAAGGAAACATGTAATGATAGGCAGACTCCCAGCGAGAAAAAGGTAGATGTGGATGCTGATCCCCTGTATGGGCGAAAAGAACTGAATAAGCAGAAACCTCCATGCTCTGAGAGTCATAGAGATACCCAAGATATTCCTTGGATAATGCTGAATAGTAGCATTCAGAAAGTTAACGAGTGGTTTTCCAGAGGTGATGACATATTAACTTCTGATGACTCACACGATAGGGGGTCTGAATTAAATGCATTGAAAGTTTCAAAAGAAGTAGATGAATATTCTAGTTTTTCAGAGAAGATAGACTTAATGGCCATTAATCCTCATGATACTTTACAATTTGAAAGAGTCCAATTGAAACCAGCAGAGAGTAACATCAAAGATAAAATATTTGGGAAAACCTATCATAGGAAGGCAAGCCTCCCTAACTTGAGCCACATAACCCGATTTATAGGAGCTATTGCTGCAGAGCCCAAGATAACACAAGAGCATTCCCTCCAAAATAAAATAAAGCGTAAAAGGGCATCAGGCCTTCGTCCTGAGGATTTATCCAAGAAAGTAGATTTGACAGTTCAAAAAACCCCTGAAAAGATAAATCAGGGAACTGACCAAATGGAGCAGAATGATCCAGTGATGAATATTGCTAATAGTGGTCATGAGAATGAAACAAAAGGTGATTGTGTTCAGAAAGAGAAAAATGCTAATCCGACAGAATCATTGGGAAAAGAATCTTTCAGAACTAAAGGCGAACCTATAAGCAGCAGTATAAGCAATATGGAACTAGAATTAAATATTTTAAATTCAAAAGCATCTAAGAAGAATCCGAAGAGGATGTCCACCAGGCATATTCATGCACTTGAACTAGGCAGTAGAAATCCAAGCCCACCTAATCATACTGAACTACAAATTGATAGTTGTTCTAGCATTGAAGAGATAGAGAAAATAAATTCTAACCAAAAGCCAATCAGACACAACAGAATGCTTCAACTCACGAAAGAAAAAGAAACCACAACTGGAGCCAAAAAGAATAAGCCAAATGAACAAATAAGTGAAAGACATGCCAGTGATGCTTTCCTAGAACTTAAAAATGTAACTGATTTTCTTCCTAAATGTTCAAGTTCTGATAAACTTCAAAAATTTAATTCTAGCCTGCAAGGAGAAGTAGCAGAGAACCTAGAAACAATTCAAGTGTCTGATAGTACCAGGGACCCTGAAGATCTGGTGGTAAGTGGAGAAAAGTGTTTGCAAACTGAAAGATCTGCAGAGAGTACCGGTATTTCAGTGGTACCTGATACTGATTATGGCACTCAAGACAGTATCTCATTACTGGAAGCTGACACCCTGGGGAAGGCAAAAACAGCACTAAATCAACATGTGAGTCAGTATGTAGCAATTAGAAATGCCACTGAACTTTCCCATGGTTCTAAAGACACTAGAAATGACACTGAAGATTTTAAGGATTCATTGAGACATGAAGTTAACCACTCGAATCCAGAAAATGAATGTGCACACTCCAGGTTCTTAGGGAAACAAAGTCCAAAAGTCACCTTTGAATGTAGACATAAAGAAAATCAGGGGAAGAAAGAGTCTAAAAAACATGTGCAGGTAATTCACACAACTGCAGGCTTTCCTATAGTTTGTCAGAAAGATAAGCCAGGTGATTATGCCAAAGGTCAAGGAGTCTCTAGGCTTTGTCAGTCCTCTCAGGCCAGAGGCAATGAATCTGAACTCATTAATTCAAATGAACATGAAATTTCACAAAACCCAGATCAAATGCCATCACTTTCTCACATGAAGTCATCTGTTAAAACTAAATGTAAGGAAAACCTGTCAGAGGAAAAGTTTGAGGAACTTACAGTGTCACTTGAAAGAACAATGGTAAATGAGAACATTCAAAGTACAGTAAGCACAATTAGCCACAGTAACAGAGAAAACACTTTTAAAGAAGCCAGCTCAAGCAGTATTAATGAAGTAGGGTCCAGTGATGAGAACATTCAAGCAGAAGTAGGTAGAAACAGAGCACCTAAATTAAATGCTATGCTCAGATTAGGTCTTATGCAACCTGAAGTCTATAAGCAAAGTCTTCCTATAACCAATAAATATCCTGAAATAAAAAGTCAAGGAATTCGGGCTGTTGATATAGACTTCTCTCCACTAATTTCAGATAACCTACAACTACCTATGAATAGTTGTGCTTCCCAGATTTGTTCTGAGACACCTGATGACTTGTTAGATGATGATGAAATAAAGGAAAATAACTGCTTTGCTGAAAGTGACATTAAGGAAAGATCTGCTATTTTTAGCAAAACTGTCCAGAAAAGAGAGTTCAGAAGGAGCCCTAGCCCTTTAGTCCATACAAGTTTTGCTCAGGGTCACCAAAGAAAG'),
 ('DogFaced',
  'TGTGGCACAAATACTCATGCCAACTCATTACAGCATGAGAACAGCAGTTTATTATACACTAAAGACAGAATGAATGTAGAAAAGACTGACTTCTGTAATAAAAGCAAACAGCCTGGCTTAGCAAGGAGCCAGCAGAACAGATGGGTTGAAACTAAGGAAACATGTAATGATAGGCAGACTTCCAGCGAGAAAAAGGTAGTTCTGAATGCTGATCCCCTGAATGGAAGAATAAAACTGAATAAGCAGAAACCTCCATGCTCTGACAGTCCTAGAGATTCCAAAGATATTCCTTGGATAACACGGAATAGTAGCATACAGAAAGTTAATGAGTGGTTTTCCAGACGTGATGAAACATTAACTTCTGATGTCTTACTTGATGAGAGGTCTGAATCAAATGTGGTAGAAGTTCCAAATGAAGTAGATGGATACTCTGGTGCTTCAGAGGAAATAGCCTTAAAGGCCAGTGATCCTCATGGTGCTTTAATATGTGAAAGAGTTCACTCCAAATTGATAGAAAGTAATATTGAAGATAAAATATTTGGGAAAACATATCGGAGGAAAGCAAGCCTCCCTAACTTAAGCCACATAACTGAAATTACAAGAGCATCTGCTACAGAACCTCAGATAACACAAGAGTGCCCCCTCACAAATAAACTAAAACGTAAAAGAACATCAGGCCTTCATCCTGAGGATTTTATCAAGAAAATAGATTTGACAACTCAAAAAACTTCTGAAAATATAATTGAGGGAACTGACCAAATAGAGCAGAATGGTCATGTGATGAATAGTTCTAATGATGGTCATGAGAATGAAACAAAAGGTGATTATGTTCAGAAGAAGAAAAATACAAACCCAACAGAATCATTGGAAAAAGAATCTTTCAGAACTAAAGTTGAGTCTGTACCCAACAACATAAGCAATGTGGAACTAGAATTAAATATTCACGGTTCAAAAGCACTCAAGAAGAATCTGAGGAGGAAGTCCACCAGGCATATTCATGCACTTGAACTAGTCAATAGAAATTCAAGCCCACCTAATCATACTGAACTACAAATTGATAGTTGTTCCAGCAGTGAAGAACTGAAGGAAAAAAATTCTGACCGAATGCCAGACAGACACAGCAAAAAACTTCAGTTCGTAGAAGATAAAGAATCTGCAACTGGAGCCAAGAAGAACATGCCAAATGAGGCAATAAATAAAAGACTTTCCAGTGAAGCTTTTCCCGAATTAAATAACGTACCTGGTTTTTTTACTAATGGTTCAAGTTCTAATAAACGTCAAGAGTTTAATCCTAGCCTTCAAGGAGAAGAAATAGAGAATCTACGAACAATTCAAGTGTCTAATAGCACCAAAGACCCCAAAATTCTAATCTTTGGTGAAGGAAGAGGTTCACAAACTGATCGATCTACAGAGAGTACCAGTATTTTATTGGGACCTGAAACGGATTATGGCACTCAAGATAGTATCTCATTACTGGAATCTGACATCCCAGGGAGGGCAAAGACAGCACCAAACCAACATGCAGATCTGTGTGCAGCAATTGAAAACCCCAGAGAACTTATTCATGATTTTAAAGAAACTAGAAATGACACAGAGAGCTTTAAAGATCCATTGAGACATGAAGTTAACTCCTCAGACCCAGAAAAGGAATGTGCACACTCCAGGTCCTTGATAAAACAAAGTCCAAAAGTCACTCTTGAATGTGACCGAAAAGGAAATCAGGGAAAGAAAGAGTCTAACGAGCATGTGCAGGCAGTTTATACAACTATAGGCTTTCCTGGGGTTTCTGAGAAAGACAAACCAGGAGATTATGCCAGATATAAAGAAGTCTCTAGGCTTTGTCAGTCATTTCAGTCTAGAAGAAATGAAACTGAGCTCACTATTGCAAATAAACTTGGACTTTCACAAAACCCATATCATATGCCATCCATTTCTCCCATCAAGTCATCTGTTAAAACTATATGTAAGAAAAATCTGTCAGAGGAAAAGTTTGAAGAACATTCAATATTCCCTGAAAGAGCAATAGGAAATGAGACCATTCAAAGTACAGTGGGCACAATTAGCCAAAATAACAGAGAAAGCACTTTTAAAGAAGGCAGCTCAAGCGGTATTTATGAAGCAGGTTCCAGTGGTGAAAACATTCAAGCAGAACTAAGTAGAAACAGAGGACCAAAATTAAATGCTGTGCTTCAGTTGGGTCTCATGCAGCCTGAAGTCTATGAGCAAAGCCTTCCTCTAAGTAATAAACATTCTGAAATAAAAAGGCAAGGAGTTCAGGCTGTTAATGCAGATGTCTCTCCACAAATTTCAGATAACTTAGAGCAACCTATGAACAGTAATATTTCTCAGGTTTGTTCTGAGACACCGGATGACCTGTTAAATGATGACAAAATAAAGGACAATATCAGCTTTGATGAAAGTGGCATTCAGGAAAGATCTGCTGTTTTTAGCAAAAATGTCCAGAAAGGAGAATTCAGAAGGAGCCCTAGTCCCTTAGCCCATGCAAGTTTGTCTCAAGGTCGCCCAAGAAGG')]

Handling overloaded FASTA sequence labels

The FASTA label field is frequently overloaded, with different information fields present in the field and separated by some delimiter. This can be flexibly addressed using the LabelParser. By creating a custom label parser, we can decide which part we use as the sequence name. We show how to convert a field into something specific.

from cogent3.parse.fasta import LabelParser

def latin_to_common(latin):
    return {"Homo sapiens": "human", "Pan troglodtyes": "chimp"}[latin]

label_parser = LabelParser(
    "%(species)s", [[1, "species", latin_to_common]], split_with=":"
)
for label in ">abcd:Homo sapiens:misc", ">abcd:Pan troglodtyes:misc":
    label = label_parser(label)
    print(label, type(label))

human <class 'cogent3.parse.fasta.RichLabel'>
chimp <class 'cogent3.parse.fasta.RichLabel'>

RichLabel objects have an Info object as an attribute, allowing specific reference to all the specified label fields.

from cogent3.parse.fasta import LabelParser, iter_fasta_records

fasta_data = [
    ">gi|10047090|ref|NP_055147.1| small muscle protein, X-linked [Homo sapiens]",
    "MNMSKQPVSNVRAIQANINIPMGAFRPGAGQPPRRKECTPEVEEGVPPTSDEEKKPIPGAKKLPGPAVNL",
    "SEIQNIKSELKYVPKAEQ",
    ">gi|10047092|ref|NP_037391.1| neuronal protein [Homo sapiens]",
    "MANRGPSYGLSREVQEKIEQKYDADLENKLVDWIILQCAEDIEHPPPGRAHFQKWLMDGTVLCKLINSLY",
    "PPGQEPIPKISESKMAFKQMEQISQFLKAAETYGVRTTDIFQTVDLWEGKDMAAVQRTLMALGSVAVTKD",
]
label_to_name = LabelParser(
    "%(ref)s",
    [[1, "gi", str], [3, "ref", str], [4, "description", str]],
    split_with="|",
)
for name, seq in iter_fasta_records(fasta_data, label_to_name=label_to_name):
    print(name)
    print(name.info.gi)
    print(name.info.description)

NP_055147.1
10047090
 small muscle protein, X-linked [Homo sapiens]
NP_037391.1
10047092
 neuronal protein [Homo sapiens]

Using a third-party plugin for sequence storage

Sequence collections and alignments have a .storage attribute which holds the underlying sequence data and provides basic functions for obtaining it. Users can install a third-party plugin which is customized for different types of sequence data. The following examples require you install the cogent3-h5seqs plugin. This project provides alternative storage for both unaligned sequences and for alignments.

$ pip install cogent3-h5seqs

Selecting an alternate storage backend

Specify the storage using the storage_backend argument.

from cogent3 import load_aligned_seqs

aln = load_aligned_seqs(
    "data/long_testseqs.fasta", moltype="dna", storage_backend="h5seqs_aligned"
)
aln

	0
DogFaced	TGTGGCACAAATACTCATGCCAACTCATTACAGCATGAGAACAGCAGTTTATTATACACT
Human	......................G...............................CT....
HowlerMon	......................G...........................G...CT....
Mouse	.........G..G.........G...........C.....C............GCT..T.
NineBande	.........................T............................CT....

5 x 2532 (truncated to 5 x 60) dna alignment

That’s it!

type(aln.storage)

cogent3_h5seqs.aligned.AlignedSeqsData

For the cogent3-h5seqs package you specify a different storage backend for unaligned sequences.

from cogent3 import load_unaligned_seqs

seqs = load_unaligned_seqs(
    "data/long_testseqs.fasta", moltype="dna", storage_backend="h5seqs_unaligned"
)
type(seqs.storage)

cogent3_h5seqs.unaligned.UnalignedSeqsData

Set the default storage

You can set the default storage process-wide, so you don’t need to use the storage_backend argument.

import cogent3

cogent3.set_storage_defaults(
    unaligned_seqs="h5seqs_unaligned", aligned_seqs="h5seqs_aligned"
)

aln = cogent3.get_dataset("brca1")
type(aln.storage)

cogent3_h5seqs.aligned.AlignedSeqsData

When you apply operations, the new backend storage setting is applied.

coll = aln.degap()
type(coll.storage)

cogent3_h5seqs.unaligned.UnalignedSeqsData

Note

To revert to the cogent3 defaults, use the reset argument.

cogent3.set_storage_defaults(reset=True)