I’m jumping onto some of Cheryl’s recent posts on the “lab leak” hypothesis and the argument that the lack of a confirmed zoonotic host 17 months into the pandemic is a good reason to change our priors.
Science is often slow. Clinical trials and secondary data analysis can be fairly fast. COVID has led to pandemic related research to get prioritized review, decision-making, and web-only publication for some research so turn-around times for some types of science from submission to publication is short, but not all science is clinical trials, observational trials or clinical experience.
I’m in a field (health policy) that has a culture/expectation of frequent and relatively fast publication. On Friday, we got notified that one of our COVID related policy papers had received a soft acceptance. We formulated the question in March 2020, needed six months for the data to actually happen, and then the team spent a few months analyzing data and writing. It was under review and revision for just under six months. We are looking at fifteen months from an idea to acceptance, and given the journal in question, we are looking at likely eighteen to twenty four months from idea to official publication. Twenty four months is FAST.
I have another paper that was recently accepted. This was super-quick with perhaps five months from “hmm… this is a peer-reviewable idea” to acceptance. My co-authors and I performed an analysis on data that we already had for several years for other projects. We had been informally and casually talking through the idea for at least a year or more and all of us are publishing papers that are adjacent to this manuscript on a regular basis. This was an opportunistic publication that we think is timely, relevant and useful. It is only timely because we had pre-exisiting data, and we had been thinking hard about the problem in slightly different contexts for several other manuscripts and grant applications.
Those two papers are fast within the academic context because they are using administrative data that is fundamentally clean and we were applying fairly straightforward techniques.
Science is slow when new data needs to be collected, when new concepts and techniques need to be validated and explored. Science is slow when it is messy and complex. The belief that if nothing has been published in eighteen months after the barest parameters of the problem space were laid out means that we should significantly update our priors is likely to be a false belief. The belief that a year and a half is enough time for complex data collection to be complete and exhaustive for any problem greater than “what flavor Mac and Cheese is best for my eight year old” shows that the holder of that belief has never had to do raw data collection, cleaning and processing.
Some science can be fast. Some science is, necessarily, working on a time scale of years instead of weeks or months.
Baud
But I want it now!
Another Scott
+1 Indeed. And if one happens to get a reviewer that sits on the paper for a few weeks before rejecting it, or demanding huge revisions and more data, (sometimes by a competitor group that doesn’t like you potentially stealing their thunder) then additional months can easily be added to the process. Or an editor that doesn’t know what to do with your paper in a new area and cannot find reviewers for 6+ months… :-/
The premier journals in hot areas that have enough critical mass can be very, very fast in the review and publication process, if you’re a recognized authority. If not, well, it can be a very long slog. But that’s usually Ok, science and medicine and medical policy should strive to get it right and that indeed does take time.
Thanks.
Cheers,
Scott.
Betty
Thanks for a clear, straight forward explanation, David. Sadly most, if not all, those pushing lab leak stories have a different agenda
Villago Delenda Est
Totally unacceptable to the click driven swine of the Village.
Spanky
“Lab leak” idiots are one thing, health professionals refusing vaccination because it’s “experimental” is another.
Sure, it’s not fully fda-cleared but 1 billion doses give you a pretty damn clear picture of how this “experimental” vaccine works.
RSA
Right. Unfortunately, it’s maybe inevitable in the areas I’m familiar with. Reviewing is a volunteer activity, which means that for almost everyone who’s doing it, it gets lower priority than other stuff that’s part of their job.
Capri
The peer-review process is the last bastion of the “English gentleman” concept. All about reputation, service, and honor. Works great if everyone does it in good faith. Review has strengthened at least 90% of the papers I’ve published in my field, but you hear some horror stories.
I know someone who submitted a paper and the reviewer requested that some data be removed from the abstract but not the actual paper. Hard to believe that’s anything other than the reviewer knowing that most people only read the abstract and wanting to keep the information from becoming widely know.
billcinsd
In my field, academic publications used to take up to more like 24 months to go from being submitted to being published. It gets submitted, the editor then assign it to an assistant editor who then chooses reviewers. The reviewers then make recommendations which the writer must respond to then assuming these changes are sufficient, then it could easily take up to a year to actually get into the physical journal. Now, most journals will publish the article online immediately after acceptance and receipt and correction of the proofed manuscript and they also try to push the review time up considerably. My most recent paper took about 3 months to go from submission to publication. Depending on how you want to count the time, it took 6 months to several years to get to the point of being able to write the paper
CaseyL
I’ve worked in medical/medical research/intellectual property environments for nearly all of my 40-year adult professional life. This is an area I feel knowledgeable in (on a lay level; I certainly don’t compare myself to actual doctors and researchers) when it comes to the scientific method, peer review, and how science works.
Trying to convey to UNeducated laypersons the concepts that I’ve absorbed over those 40 years is a Sisyphean task – not least because so much of what I’ve come to know resembles a kind of coral reef, with the beginning concepts long since grown over by later information.
Trying to convey any of that to people who are not only uneducated, and not only uninterested, but essentially opposed to the whole concept, and have their own destructive (always destructive) agendas to push… is simply impossible.
So I have enormous respect and appreciation for the people, like David, who keep trying.
Another Scott
@billcinsd: (I know you know the following; for other readers :-)
In the physical sciences, one can put manuscripts on arXiv the when one submits them to a physical journal. And there are similar “pre-print servers” in other fields. And that’s a good thing (and is especially helpful when one cannot attend conferences (work in conferences is often a year or more ahead of the journals)), but papers there usually are viewed as an early part of the peer-review process, not a substitute.
So, one can get information out early, but it doesn’t have the weight that something surviving the peer-review process does.
Cheers,
Scott.
WhatsMyNym
@Capri: I don’t think you can blame it on the English. It was a natural progression from the earliest days of sharing scientific information within a larger sub-community. You could trace it back to the Greeks.
JoyceH
@Spanky:
This speaks to something I’ve been thinking lately. While ‘science is slow’, there is so much that has been in the works for years that is all coming to a head almost simultaneously.
I recall at the beginning of the pandemic, the health care professionals tried to get across to the public that we HAD to follow the health guidelines (that the administration was ignoring) to stop the spread of the pandemic and not expect a vaccine to show up any minute and bail us out of our own carelessness, because making vaccines takes a long time. I recall four years being mentioned as a good estimate for the development of a vaccine, along with the fact that after decades of trying, there is still no vaccine for HIV.
And then within a year of the virus first entering public awareness, vaccines were rolling out and being administered. I understand that good fortune is due to all the preliminary work that had gone on for years with mRNA. And now that mRNA has proved itself, researchers are now working on a universal flu vaccine and a universal coronavirus vaccine – which means we could see a world without a ‘cold and flu season’ within our lifetimes! (And most of us on these discussions aren’t exactly spring chickens.) There’s also mRNA research going on for applications in cancer treatment, Alzheimer’s, and yes, an HIV vaccine.
And that’s just one field that’s coming to fruition in this moment. Renewable energy has already become competitive with fossil fuels – you can go out today and buy an electric car!
Space exploration is back in vogue, and we’re looking at pictures taken on the surface of Mars.
At the same time, there’s growing awareness of racial and financial disparities along with determination to actually do something about it.
So we could very well be hovering on the brink of a new Golden Age.
And yet, as the pandemic and our recent political scene have also shown us, not a majority but a substantial minority of the public is not only profoundly stupid or at least ignorant, but are also aggressively violent in support of their profound ignorance. Which means we could also very well be hovering on the brink of a new Dark Age.
So… just… dang.
Stay sharp, everybody.
Spanky
@Another Scott: Likewise, in astrophysics one can go the fully review process in the Astrophysical Journal (ApJ), or …
dmsilev
@Capri: I’ve had a few bad review experiences. Reviewers that either didn’t read or didn’t understand the material. I had one reviewer once willfully ignore roughly twenty years of prior work in order to insist that one particular model could trivially explain our data (said prior work unequivocally established that that model was fundamentally wrong for this particular material). Pointing this out to both the reviewer and the editor made no difference: paper was rejected, and we ended up publishing it somewhere else.
Cheryl Rofer
The preprint archives have speeded the publication process up, and SARS papers are getting priority. Quite a few have been published in Nature and Science.
But slogging through the forests to find bats and pangolins and whatever else will take some time, as will identifying the viruses. And then going back to find what you missed, clarify things about the virus or the pangolin you got it from…
bcw
I don’t think the intermediate animal between bats and humans for SARS-COV-1 was ever identified for certain or where the disease was when it passed to humans. You’re trying to find some sick population somewhere that is carrying something close enough to what showed up in humans and then a pathway for it to undergo the changes necessary to move to humans even as the diseases themselves keep changing. Also, any papers coming out now will be based on research that was already going on where the work happened to be relevant.
Earl
For SARS, we had strong evidence of zoonotic hosts and the transmission route within 13 months. Yes blah blah blah science is slow, but in the most relevant analogue of covid19 — and one that caused a couple trillion dollars less damage — we were doing a lot better 17 months in.
Given the deliberate hiding of data by the Chinese government, it gives reasonable people pause. Add in the (self-admitted!) evidence of careless handling of SARS-like viruses by the Wuhan lab.
See the review by (ironically) Shi Zhengli https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7114516/
ps — David, you’re deliberately moving the goalposts here. Good evidence for a zoonotic transmission route would be shared much faster than your claimed 15 month timeline. In particular, it happened much more rapidly for SARS.
steve g
The SARS epidemic was in 2003. What was the state of research on its origin ten years later, in 2013?
Isolation and characterization of a bat SARS-like coronavirus that uses the ACE2 receptor
“…attempts to isolate the progenitor virus of SARS-CoV from bats have been unsuccessful. Diverse SARS-like coronaviruses (SL-CoVs) have now been reported from bats in China, Europe and Africa, but none is considered a direct progenitor of SARS-CoV…”
So yes, it might take a while to find the animal reservoir and origin virus of SARS-CoV-2.
Chetan Murthy
@Earl:
This is very fair. But as a mild counterpoint: I was thinking about the anthrax attacks right after 9/11. Imagine that instead of them being directed at Americans, they’d been directed at (say) Russia. Does anybody think that we’d open our weapons research labs to Russian inspectors? Anybody? anybody? Bueller? And that really is the right comparison: the PRC isn’t a democracy and *everything* related to national defense (and virology sure is) will be treated by the leadership as sensitive.
Now imagine there were a natural explanation: they *still* wouldn’t allow us to investigate. Again, b/c “sensitive to national defense”.
So post-hoc, how do we tell apart these two scenarios, and does the PRC refusal to allow investigations give us any indication of the prior state? I don’t see how
ETA: the two scenarios being “lab escape” and “natural origin”. The PRC’s behaviour would be *identical* in both.
Tom Levenson
@Cheryl Rofer: Not to mention that the slogging through the jungles is only the first step of a much longer process. As those who actually work with this material keep pointing out and the lab-escape useful idiots keep ignoring, it’s very hard to extract intact virus genomes from bats, for example, which means exhaustively establishing what they carried is waaaaay different from the popular view of “draw a little blood, put it in the magic box and hey, presto!”
Forensic epidemiology, and not just the publication of same, is really hard, and can be very slow.
Earl
@steve g: nonsense — because *it didn’t for sars*. See the link I shared.
Tom Levenson
@Earl: Yours is an example of cherrypicking, in two ways.
1: other zoonosis pathways have taken much longer to come up with even partial confirmation. What’s relevant isn’t a good, or the best, experience of such research but the spectrum of such research outcomes.
2: It’s not a confirm/no information question. There is accumulating supporting evidence that suggests a zoonotic origin for COVID-19. See e.g. https://www.nature.com/articles/s41467-021-21240-1, for just one recent paper that adds to the picture.
Chetan Murthy
@Earl: @steve g: For all of us nonspecialists, it might be helpful if you presented your bona fides (up to whatever level of anonymization you wish). Just so we can understand what we’re seeing here.
I looked at both papers (I’m a PhD in computer science, so *not* an expert) and certainly to an unexpert eye, the 2008 paper (that Earl cites about 2003 findings) does *not* seem to show much better progress than what we laypeople know about the search for animal hosts: there are various close matches we’ve all heard about, but nothing that looks like a smoking gun.
And it’s …. *interesting* that Dr. Shi (author on that 2008 paper) is also an author on the 2013 paper that says “no success yet, guys”.
Look: I’m not saying one of you is wrong. Just that, this is a specialized subject. There seem to be two key grafs in the 2008 paper, starting with
If this is the part citing, Earl, it sure doesn’t seem as dispositive as you make it sound. And that one of the authors is an author, 5yr later, of an article that says pretty clearly
makes it all pretty muddy.
Chetan Murthy
@Earl: Urk. My comment is in moderation. Summary: the two articles you both cite both have Dr. Shi as an author. The earlier one is (to a layperson’s reading) nowhere near as conclusive as you’ve made it out to be. So it might be useful for both of you to present your qualifications to read these articles as better-than-laypeople. It’s also relevant that Dr. Shi wrote that article in 2008 you cite, Earl, and yet 5yr later, wrote another, where she’s pretty firm that, no, we don’t know where SARS came from.
Earl
@Tom Levenson: It’s fair to point out other zoonotic routes took longer.
But the SARS one isolates multiple variables: same country, similar government, similar personnel, similar disease, (quite likely) similar origin.
The fact that it isn’t even touched on in David’s argument indicates he didn’t bother to do minimal prep work to make his own argument; that he’s bullshitting in defense of authority (his and his profession’s); or ???
Chetan Murthy
@Earl:
The evidence you adduce doesn’t appear to support your position
ETA: I’m not saying you’re wrong. But I think you need to show us where in the 2008 paper the firm claim is made, and then argue why the 2013 paper by one of the same authors (Dr. Shi) is invalid.
Tom Levenson
@Earl: You make a bunch of assumptions that are not in evidence. Some political ones, and some biological, including the notion that the fact two viruses are in the same family means they will have similarly easy or difficult to trace histories in the wild.
And if I’m betting on your rigor vs. David’s, I know where my money goes. YMM-and I’m sure does-V
Feathers
Having worked at a bleeding edge research lab (AKA doesn’t work on work that is a follow on to somebody else’s work and inventing the tools to do so), the “peer reviewed research only” attitude is so limiting in the real world. What we can do peer research on is limited to what we can measure. That is even before looking at questions of funding and if it is a topic that interests one of the fairly small number of people who do original research in a given area. And I’m not even talking about which topics do and don’t “interest” the powers that be in the research world. I sat through lots of presentations by other researchers who wanted to partner with our lab. A few we did, vast majority we didn’t. Ninety-five percent of that was simply a matter of time. Research takes attention and in the end the limiting factor was the amount of time people had to offer.
During the pandemic, I’ve how people fell into the “peer research only” trap, without acknowledging that there was no peer reviewed research available. It seems so often that this was and is a willful, panic-induced closing of the mind. We are in a new situation. This will involve looking at research on similar topics, observing what is going on in the real world, looking at the resources available, and making decisions based on the information at hand. No wonder people were grabbing at strawmen and falling for conspiracy theories. We’ve based so much of our education system on “research” and having sources to cite for every idea that you put in a paper. Too many people, even (or especially) highly educated people are unable to reach beyond that.
There needs to be a way to say we are out beyond the peer-reviewed realm and in Here Be Dragons land. Tread wisely and carefully, observing the world and studying the works of those who have gone before. We are trying to defeat the liars and the conspiracy theorists, but there is too much of the world where there is no “research” to guide us. We have to be able to admit what we don’t know.
Feathers
@JoyceH: And you’ve now got people building conspiracy theories that the COVID vaccines prove that we could have had an HIV vaccine all along if only we hadn’t hated the gays so much.
Sad laugh, having known so many researchers bitter over all the resources allocated to HIV, rather than their own disease of interest. That said, all of those psoriasis medication ads that you see on TV are an outgrowth from HIV research. Psoriasis is an autoimmune disorder, much understudied and misunderstood. Psoriasis became a complication of AIDS, got funding and now we have medicine for it. My dad has psoriasis and was in a number of excruciating studies in the 70s and early 80s which left him too debilitated to take advantage of the new medications, but he is happy to see them.
Wag
Perfect. Thank you.
JoyceH
@Feathers:
Gaah! Back in the ’70s, the first symptom of blocked arteries was a heart attack, because they didn’t have the preventative tests that we do now. Killed my dad, but that was because he was born too soon, not because ‘they’ hate preachers.
But then again, the cicada landing on Biden’s neck was a Message from Q. Because? Because Brood X are 17 year cicadas and Q is the 17th letter of the alphabet – it’s all so OBVIOUS! Stuff like that makes the trend toward Dark Age over Golden Age look a little more likely.
bcw
@Chetan Murthy: And does anyone think we actually found out what happened with the anthrax attacks? The effect of the attacks was to increase fear in the US and undermine Congressional support for the UN inspections into Iraq WMD’s, ultimately enabling the invasion of Iraq by the BUSH administration. The FBI screwed up the investigation badly, first blaming one unlikely scientist and then shifting to one who conveniently committed suicide. There was never an actual investigation with access to the national military biolabs (just the CDC labs, not Sandia or Fort Detrick.) There were extravagant claims of the ability to identify tracing characteristics which turned out to not be true. They never established that the dead scientist had the tools in his lab to make the anthrax and alter it to make it airborne while keeping himself alive and leaving no traces. This is the only conspiracy theory I ever thought possible as deliberate use of military anthrax would have been one of the few such stories with the elements for a successful conspiracy – only a few people, possibly only one,would need to know what was actually happening and, those people and any one with any information would already be covered by secrecy laws.
bcw
@JoyceH: And you’ve now got people building conspiracy theories that the COVID vaccines prove that we could have had an HIV vaccine all along if only we hadn’t hated the gays so much.:
So what does that tell us about the conspiracy to keep Africa subjugated by not making a malaria vaccine! – It can’t possibly be because the diseases are so different. Jokes aside, there does finally look like there may be at least one effective malaria vaccine coming out which is huge news.