Category Archives: Science & Ethics

On the other side of an experiment

Posted by
Reply

I’m three weeks into having type 1 diabetes (T1D) and very grateful that it’s 2017. If it were 1919, I’d have a painful year and a half to live while slowly dying from starvation. That’s a scary thought.

Today, my endo says I have a normal life expectancy and she wouldn’t expect me to ever need anything amputated. That’s a relief. She also says that within my life time there will be a cure. “I’m a pessimist, Jo. And even I think there will be a cure in the next ten years,” is how she phrased it last week. Continue reading →

p-hacking and science with an agenda

Posted by
Reply

I recently read this post about p-hacking (see also: data dredging, fishing, snooping). Two things that I found to be noteworthy were an interactive example of how p-hacking works, and a description of an experiment where different research teams analyzed the same data set:

 

“Twenty-nine teams with a total of 61 analysts took part. The researchers used a wide variety of methods, ranging — for those of you interested in the methodological gore — from simple linear regression techniques to complex multilevel regressions and Bayesian approaches. They also made different decisions about which secondary variables to use in their analyses.

 

Despite analyzing the same data, the researchers got a variety of results. Twenty teams concluded that soccer referees gave more red cards to dark-skinned players, and nine teams found no significant relationship between skin color and red cards.”

 

To reiterate, all of the methods used were justifiable. There wasn’t any fudging or fabricating data. A group of skilled analysts sat down and came up with 29 defensible methods for analyzing the same data that gave different answers. To me, this is the stuff of existential crises. To quote the article, “[e]very result is a temporary truth”. Which I think is pretty concerning if you’re working in a situation where temporary truths don’t cut it.

 

Joshua Tewksbury is a biologist who spent 10 years as a professor at the University of Washington before moving to a position with the World Wildlife Fund. About a year ago, he wrote a post about transitioning to an NGO position where, he writes, “[s]cience shows up as just another wrench in the toolkit.” A deeply malleable tool, apparently. On the one hand, it’s troubling to think about making decisions with temporary truths. On the other hand, and this strikes me as almost heretical to type, if you deeply believe in your cause, maybe it’s not so bad to (ethically and with full disclosure) make subjective decisions in how you analyze your data to advance your cause.

 

After thinking about it for a while, I’m still not sure how bad my crisis should be. In the first post, one of the project leaders is quoted as saying:

 

“On the one hand, our study shows that results are heavily reliant on analytic choices,” Uhlmann told me. “On the other hand, it also suggests there’s a there there. It’s hard to look at that data and say there’s no bias against dark-skinned players.”

 

At first pass, this didn’t help me. As somebody who takes comfort in certainty (and don’t most scientists?) the “squint at it” method of assessing data is an endless source of frustration. But I’ve also realized that we might feel confident about one other thing from the soccer data set. No groups concluded that lighter skinned players received more red cards. Maybe there are some relatively permanent truths, it’s just that they don’t answer the question we set out to answer.

Desert Island Books.

Posted by
Reply

For domestic reasons, I have been recently reunited with my book collection. There is a fantastic BBC radio series (70+ years old now) where celebrities talk about the 8 songs/tracks/music pieces they would take to a Desert Island. Music is tough. But much to my surprise, re-studying my ‘library’, my list of eight books is easy. In no particular order:

Failure Is Not An Option. A testament to what humans can achieve freed from Health and Safety, HR and the corporate bullshit of ‘Your safety is our top priority’. Management everywhere need to read this. The best of the Apollo books, by far. As my colleague Marcel Salathe is fond of saying, quoting I think one of the Roosevelts: When safety comes first, America is lost. These guys had higher ambitions, and they walked in the heavens. Americans, read this: it is what you are capable of.

A Bright Shining Lie. I see I first read this a quarter century ago. It is still with me. Searing.

The Donkeys. The folly of man. Even more powerful since Sean and I, and later son Matthew and I, went to the battlefields, this book in hand. The ‘hills’ are slight rises. The mud is awful. The inanity of the carnage unimaginable.

Lindberg. An amazing man, described by an amazing biographer. A biographer who never discovered the extra families his subject raised.

Into the Silence. “They had seen so much of death that life mattered less than the moments of being alive.” A ballsy lesson for us all: exploration of the world, of ourselves, trumps everything.

The Idea Factory. The ambition of these guys. Let’s bounce a telephone signal from California to New York off an earth-orbit satellite the size of a basket ball… if only someone could figure a way to put a satellite in earth orbit (it was the 1940s). Before that – before that – they had the math of cell phones sorted. Humanity has lost so much ambition.

Steve Jobs. I suppose there are people on the planet who have not read this book. For me, it induced calm. It is ok to imagine that computers should be better, easier to use. I look forward to the day. Meantime, important lesson: one should suffer ass-holes, just in case they’re the one.

Lovelock. The dilemma of the Berlin moment. Better to achieve perfection, just once in a life after years of planning? Or to aspire and never make it? Or to never be in the running?

If you love something, set it free

Posted by
1

Artistic recreation of uploading data to Dryad

I was talking with my friend Ben today when he asked me what I thought about #PLoSFail. For those of you not up on your twitter lingo, a hashtag is a way of tagging a tweet so that people can easily follow tweets based on keywords or phrases. In this case, tweets concerning the new data policy at PLoS which requires that published data be made freely available. There are, of course, exceptions in the policy for ethical and legal reasons. And PLoS doesn’t require the whole data set, just the “minimal dataset… used to reach the conclusions drawn in the manuscript with related metadata and methods, and any additional data required to replicate the reported study findings in their entirety.”

Andrew has already blogged about open date making him a bit tense and apparently a lot of scientists on Twitter feel the same way. Personally, I don’t go out of my way to post my data and I’m hardly an open data proselytizer, but I have no problem with making my data available once a paper is published. Science is a peer-reviewed, iterative process and I suspect open data benefits this kind of process. I think that, at worse, open data is a wash. That said, if you know of anybody making a career by parasitizing Dryad and/or GenBank without making any substantial intellectual contribution in their own right, I’d be very interested to hear about it.

Feelings on open data aside, the thing that puzzles me the most about #PLoSFail is how strongly people are reacting. Almost every journal I have ever submitted to has some kind of data archiving policy similar to the new one at PLoS. I can’t imagine how you could do science – biology at least – today without having come to terms with open access data.

It’s not just what you say, it’s how you say it

Posted by
5

I’ve been taught science writing should be terse and pithy. In that spirit, here is a distillation of this post: Does pressure to be both concise and persuasive drive the development of biased terminology in science?

I started thinking about this a little bit last week when the term “evolutionary rescue” came up as an aside during Dave’s talk about genetic diversity and the rate of evolution. The basic idea with evolutionary rescue is if a population experiencing rapid environmental change has a lot of genetic diversity, then some individuals can adapt rapidly to the new conditions and “rescue” the population, preventing extinction (dependent on population size and the magnitude of change).  I have become so used to thinking about evolution in a framework of parasites and drug resistance where the parasites are the bad guys that it wasn’t immediately obvious to me that in this case the drug resistant parasites would be the heroes that would “rescue” a declining parasite population under drug pressure.

There is a lot of jargon in science, some of which is certainly useful specific shorthand, and certainly some that adds unnecessary complexity, confusion, and arguably bias. I think part of this is storytelling, which I agree is an art and part of what makes science sticky and accessible to non-scientists. I attended a great lecture last year called “Making Tricky Science into Sticky Stories” and was convinced. Then again, is calling a single-celled organism “devious” (even if it’s a parasite) when it is just doing what every other life form on Earth is doing – living and reproducing – really necessary? Terms like evolutionary rescue give a positive connotation to rapid adaptation in changing conditions, and perhaps not surprisingly is used in the climate change literature where this is a pleasing and hopeful concept, but this concept is so closely tied to other rapid adaptations, such as in parasites for drug resistance, that it might be better just to spell out what we mean instead of making up a new term.

As on the fields of Omdurman

Posted by
1

Did you know that rats laugh?

I didn’t. Not until the other day anyway. The other day I was enumerating the oocysts in Courtney’s Wolbachia laden (or not as the case may be) mosquitoes while at the same time continuing Court’s much needed education into the available variety of BBC wireless programmes. Having exhausted the News Quiz’s back catalogue we had moved onto “The Infinite Monkey Cage,” a fun and at times funny science programme. This particular episode was looking at ‘brain science’ and well into the banter one of the guests, a neuroscientist from UCL, said, almost as an aside, almost as if everyone should know this, that rats laugh. One of the presenters, the physicist Brian Cox, after a moment of incredulity, pulled on the reins and said “hold the phone, did you just say that rats laugh,” or something to that effect, just as I was thinking the same thing.

’Tis true. Rats are ticklish it turns out, particularly around the nape of the neck, and when tickled they laugh. Naturally I can hear you saying, “laugh, are you sure?” and maybe the description has the taint of anthropomorphism. What the rats actually do is make high pitched chirps in the 50 kilohertz ultrasonic range. But since these chirps are distinct from other vocalisations and are only produced in response to being tickled, or ‘heterospecific hand play’ (quiet at the back there) as the jargon goes, ‘laugh’ seems a rather good description. Nor, before you all get carried away, is anyone suggesting that rats have a sense of humour. Giggling in response to a quick scratch of the neck is not the same as laughing at Derek’s jokes. I mean, it is not as though one of the rats gets up after the lights go out in the Read-Group animal room, taps a microphone, “pof, pof, pof,” intones self-consciously into it, “one, two, one, two,” and then goes into its Saturday night routine..

Stand-up Rat: Good evening ladies and gents and welcome to the animal house, lovely to see so many of the same faces here again tonight. A human, a neanderthal and an australopithecine walk into a bar….

.. to the considerable mirth, or approbation, of those assembled in neighbouring cages. Much as it is an image to conjure with there is obviously no nascent Eddie Izzard in the rat diaspora. But the ability to laugh seems real and the pleasure apparent because tickled rats actively seek out the same human hand that made them laugh before, actually start laughing when someone who has tickled them previously enters the room and rat pups prefer to be with adult rats that still laugh. Mice? Don’t know, but laughter is a common mammalian behaviour they say so yes, probably. Ultrasonically. And mice sing anyway. Duet even.

It’s an interesting line of research and one that could easily drag one into a little reading off the set list. But the neuroscientist’s comment made me think in another direction. The vets routinely suggest that we should get in and among our rats. Pick them up as soon as they arrive as barely-bigger-than-mice rats, let them get used to us and to being handled. A bit of neck scratching and the inaudible laughter resulting eases nerves, makes the animals more amenable to handling in the future.

The trouble is that learning a snippet about the private lives of lab animals, particularly where they show a response similar to one of our most treasured and intimate behaviours, creates a certain empathy, an, albeit brief, familial feeling that places in stark contrast our actual aims and intentions for these animals. We can rationalise what we do of course, the lack of alternative, for the greater good etc. But, standing amid the carnage of another experiment I find that the pungent whiff of cordite is very apparent.

How do we balance respect for individual privacy with the demands of epidemiological research?

Posted by
Reply

Human disease epidemiologists love Scandinavian countries. Not only are they delightful sites for conferences (side trip to Norway’s fjords, anyone?), they also maintain fanatically detailed health records for their citizens that they are more than willing to share with scientists. These datasets offer amazing opportunities for disease modeling and allow breakthroughs that simply wouldn’t be possible with less complete data. It’s much harder to do this type of work in the US. Our health records are incomplete, noncentralized and difficult to access. And when datasets are identified that do offer opportunities to bypass these problems (Guthrie cards, for instance), legal maneuvering by privacy advocates can make meaningful studies untenable.

As a researcher, I am awed by the potential of detailed personal health data to revolutionize disease management. We’re now able to track influenza outbreaks by analyzing Twitter streams; imagine what we could do with unrestricted access to Guthrie cards, computerized health records, network data from Facebook, GPS data from cell phones? But I also understand the perspective of the private citizen, who may or may not be informed about his data are being used, and who may be subject to increased insurance premiums, job discrimination or public shunning if these data are not properly handled.

Very few people would feel that private testing of infants for easily treatable genetic disorders is a bad thing. Likewise, very few would be comfortable with full public disclosure of all genetic data, health records and spatial information. But there is a huge gray area in between these two extremes that would respect individual privacy while allowing research to move forward. Public outreach and education, opt-in vs. opt-out programs and a firm commitment to privacy and transparency will increase public trust and facilitate scientific inquiry, but will this be enough? What kinds of safeguards and incentives do you think will be necessary to public health research forward in the US?

Science on the Hill: How science can work with policy-makers

Posted by
5

Recently, a member of the House of Representaives Committee on Science, Space, and Technology made comments suggesting that human females could control internal fertilization (clip). Italian courts don’t seem to understand how to interpret a P-value. (see Katey’s post ). Politicians continue to fight about whether global climate is changing, instead of making policy decisions about whether we need to do anything about it.

There seems to be a general lack of understanding about the facts science produces, the processes by which scientists arrive at those facts, and what role science should play in policy. I find this trend deeply troubling. Scientists have clearly failed to communicate how we conduct science,  the methods we use to interpret results and the distinction between facts and opinions (interpretations of what our results mean) to policy makers. Attempting to bridge this disconnect seems impossible to me.

I have reached a place of complete frustration and think the situation might very well be hopeless. Clearly, I’m going to need help understanding the other point of view. I need to have a civil conversation with a Washington insider. I happen to know one and I think we can keep it civil. I actually love him. He is my dad after all.

Left, My father and I agree on many things. Surely we can have a reasonable conversation about a "real" issue. Right, Me at my father's desk assessing policy briefs circa 1984. Clearly, I've already had enough!

My father, Tom Cator, has worked on “on The Hill” for the last 36 years. He began as a staffer for the U.S Senate and for the last 30 years has worked as a lobbyist across a broad array of industries. We had a fairly long conversation about science on the hill this week. Here are the key points that we discussed.

“Follow the money.” Money is part of the reason that policy makers often will argue or ignore facts. For example, if most of your state’s income is dependent on oil and gas, you are not going to go on the record agreeing these industries have contributed to global warming. “Politicians often don’t come in with an open mind. Their view is often based on who paid for it and what constituents think.”

“They will try to take you down”. I posed a question, “So say a group of scientists descended on The Hill. We went door to door and we offered to answer scientific questions”.

My father responded to this with a slight sense of panic in his voice, as if he was imaging me running through the Rayburn Building with a mosquito net, “That won’t work. First, they won’t have questions for you. Presentations have to be timely, or appear timely, in a political context. Second, if your facts don’t support their agenda, they may go after you personally, attempt to impugn your credibility. They will find someone to poke holes in your research.”

“Everyone has an agenda.” This was troubling. Facts are facts. Science is objective. You can’t just poke holes in research unless there are actual weaknesses. Why don’t politicians understand that? My father continued to explain that many times politicians think scientists are playing political games. “If you came to me with data, I would want to know what your agenda is”.
I countered, “Facts don’t have an agenda.”
Do they? Well, sometimes.
Over the last twenty years, there has also been a noticeable increase in “think tanks”  in Washington. This term is broadly applied to many organizations, but some of the largest are privately funded entities. Many times politicians get their research from these sources. “Think tanks have to pay salaries and rent. It’s a business. Where do you think they get their money? Industry, organized labor, wealthy individuals, interest groups, and others.” my father points out.

“It is bleak and getting bleaker.” Over his career, Tom has noticed a disturbing shift in how business gets done in Washington. “It used to be that congressmen would get facts from lobbyists. A good lobbyist would approach a legislator with their case, represent all of the facts and facets of the issue, and then would explain why their policy position had merit. If an issue like climate change had come up 20 years ago there would have been multiple hearings with recognized experts. There would have been a long discussion about the facts. There is no longer a comprehensive look at the issue.”

There has also been a shift in how legislators define compromise. “It used to be you would find points on which you could agree and disagree and meet in the middle. Now, compromise tends to be ‘I don’t budge and you meet me here’.” This change in attitude has made debate, whether scientific, legal, or moral, stagnant.

“Nothing moves quickly”. “Policy does not change quickly. Within the next decade, Medicare is going to go bankrupt. They aren’t doing anything about that. You think they are going to jump on something like climate change? There is little effort to look at long term trends and how policy should respond. Statesmanship is needed.”

Acceptance of scientific findings is also very slow. “The system is rigged to not let real science percolate up, at least not in the short term”. We agreed that this was not necessarily a bad thing. Scientific debate and discovery moves very quickly. When we make policy decisions, we need to be very certain that the scientific debate is satisfactorily over.

“Strike back” At this point in the conversation I was not feeling particularly hopeful. How do we motivate these congressmen to listen and get something done?
“There has been a failure of science to address this issue.”

Politicians are in Washington to represent their constituents. If voters in their district care about something, then congressmen will care. “Getting the public, voters, and community leaders involved makes it much harder for a politician to walk away from facts”.

And then Tom got real personal.

“The academic community has fallen down. They don’t try to involve the public or policy makers. You live in a politically conservative county in Pennsylvania. What are you doing to educate your public on science?”

Caught off guard, I mumbled something about public lecture series.

“You can’t do it from 9 am-5pm. People work. What about continuing education at night? How about explaining how climate change is and will impact the farming industry in central PA?”.

I quipped back, “I don’t see how I can talk to people who don’t want to hear what I have to say”.

My father responded, “Find a reason for people to come and make it convenient for them. Go to a local high school science teacher and have her tell students, ‘Come to this thing tonight and you get extra credit, bring your parents and we will double it.’ Then present facts carefully and be as scrupulous as possible. Present work that has been rigorously peer-reviewed, and is complete and unbiased. Maintain objectivity and project a service. Engage them in a discussion.”

I do not feel anymore empathy for Washington, but I think this conversation did get my wheels turning. I have been going at this the wrong way. Changing the way science is viewed on The Hill needs to come from the ground up and it can start here with us. “The issue here is that we need to inspire constituents to become informed about science and communicate with their representatives.”

What if experts from CIDD gave an evening seminar series at State (or Altoona or Bald Eagle) High? What would the challenges be? Would scientists participate? Would anyone come? What topics could we cover? I have no idea how to answer these questions, but perhaps I should be thinking about them.

Quality control

Posted by
1


Delicious looking health care (photo from Wikimedia Commons).

At CIDD lunch this week, we discussed this paper by Kumar et al (2005). It’s a compilation of data from randomized clinical trials of cancer treatments in children conducted between 1955 and 1997, and it shows that on average, new treatments are no better or worse than standard care.

Although scientists are not getting better at producing more effective treatments in comparison to standard care, the overall survival rate for children with cancer has been increasing since the 1950s, which suggests that standard care has changed for the better. During the CIDD lunch, someone referred to this improvement as “moving the goal posts” for cancer treatments. In my mind, this raises the question of how standard is standard care at any given time, for any given doctor, in any given hospital?

At the end of the discussion, Andrew touched on one of the problems with standard care, which is that ineffectual or even detrimental treatment regimes can become common because of misplaced confidence in anecdotal observation. An op-ed published last August in the New York Times suggested that the solution to this problem is to throw money at it – and of course, medical researchers to spend the money. More specifically, take some small fraction of health care spending and put it towards testing current treatments.

Yet even if all treatments were well supported by data, the widespread and timely implementation of best practices is not assured, as it is a long and leaky pipeline from the NEJM or the Lancet to your general practitioner’s office. Dr. Atul Gawande wrote about this problem in the New Yorker, also published this past August. As an example of the inconsistency in standard practices, Dr. Gawande wrote about his mother’s knee replacement and the extensive variation between surgeons in all aspects of the procedures, from anesthesia to physical therapy. A solution to this problem, according to Dr. Gawande, is strict and centralized quality control, just like in the Cheesecake Factory and other chain restaurants.

Which brings me to my final question: how important is standard care? For double blind, randomized clinical trials (i.e. science) it’s quite important to know that your control group is being treated consistently but for patient care (i.e. medicine), I can see some benefits, as well as costs, from multiple approaches.

NB: My experience with health care has been largely within the U.S., which certainly colors my perspectives.