Thursday 10:45-12:35, roughly 1 h 50 min. The 12:35-13:00 slot is protected writing time for Q2, not an extension of this practical. Submit Q2 by end of day.
Yesterday you were shown records. Today we will go and get them, from three resources that were built by different communities for different purposes. We will find out how much of what you need is actually there.
From here on we work in R, because that is the environment you will use for most of the master’s. Python stays where it belongs, in sequence handling.
library(jsonlite)
library(dplyr)
library(readr)
GenBank, ENA and DDBJ mirror each other under the INSDC agreement, so the same accession resolves in all three. Fetch it from both and compare what you are given.
acc <- "GU706282"
gb <- readLines(paste0(
"https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi",
"?db=nuccore&id=", acc, "&rettype=gb&retmode=text"))
ena <- readLines(paste0("https://www.ebi.ac.uk/ena/browser/api/embl/", acc))
writeLines(head(gb, 25))
writeLines(head(ena, 25))
You should see the first 25 lines of the same record, first from GenBank, and then from ENA.
The sequence is the same. Is the metadata? If need be, look through more lines of the record. List two fields that appear in one rendering and not the other. Which rendering would you cite, and does it matter?
Record the versioned accession, not the bare one. GU706282 is a moving target;
GU706282.1 is not.
p <- fromJSON("https://rest.uniprot.org/uniprotkb/P00395.json")
p$primaryAccession
p$entryType # reviewed or unreviewed
p$proteinDescription$recommendedName$fullName$value
You should see fields from an R list objects. In RStudio you can also explore the
object in the ‘environment’ tab.
UniProtKB has two halves: Swiss-Prot, curated by hand, and TrEMBL, annotated automatically. The accession does not tell you which you have.
entryTypedoes. Why does that distinction matter more than the accession itself?
occ <- fromJSON(paste0(
"https://api.gbif.org/v1/occurrence/search",
"?scientificName=Danaus%20plexippus&limit=20"))
occ$results |>
select(any_of(c("key", "scientificName", "eventDate",
"decimalLatitude", "decimalLongitude",
"coordinateUncertaintyInMeters",
"basisOfRecord", "datasetKey", "recordedBy"))) |>
as_tibble()
You should see twenty records (in any case, there were at time of writing), some of which with a missing audit field.
Now go up one level, to the dataset the record belongs to:
ds <- fromJSON(paste0("https://api.gbif.org/v1/dataset/", occ$results$datasetKey[1]))
ds$title
ds$license
ds$publishingOrganizationKey
An occurrence has two kinds of provenance: who observed it, and who published it. Both matter, for different reasons. Which of the two would you need in order to decide whether you may reuse the record?
Pick one record to audit properly. Pick any record from the three resources above that interests you. Prefer one with a physical specimen behind it: those have more metadata to be missing.
Work through this checklist and write the answers down as you go. You are not looking for a verdict, you are looking for specifics.
This audit is the substance of Q2. Keep it complete.
| Question | Where you looked | What you found |
|---|---|---|
| What identifies this record, and is the identifier versioned? | ||
| What kind of evidence stands behind it? | ||
| Who made the determination, and when? | ||
| Where was it collected, and how well is that known? | ||
| What is missing that a reuser would need? | ||
| What licence applies? |
Two of the rows usually are problematic. coordinateUncertaintyInMeters is often empty,
and the licence is frequently stated at the dataset level and nowhere on the record
itself.
Yesterday we left a question open: a study deposited its data in three places under three identifier schemes. Look at what a machine can see of one of them.
dc <- fromJSON("https://api.datacite.org/dois/10.5061/dryad.4sd51d4b")
dc$data$attributes$titles
dc$data$attributes$rightsList # is there a licence?
dc$data$attributes$relatedIdentifiers # does it point at the other deposits?
A DOI that resolves makes a deposit findable. What would have to be in
relatedIdentifiersfor the three deposits to be findable as one study? Is it there?
Due end of day today. Half a page plus a brief annotated record, as R Markdown or Quarto. The 12:35-13:00 slot immediately after this practical is protected writing time for Q2 and is not extra practical time.
Keep it to 30-45 minutes. This is a diagnostic, not an essay.
Next: TP2.3 Annotation, after the protected Q2 writing period.