Showing posts with label brain. Show all posts
Showing posts with label brain. Show all posts

Monday, 21 July 2014

Percentages, quasi statistics and bad arguments


© www.CartoonStock.com

Percentages have been much in the news lately. First, we have a PLOS One paper by John Ioannidis and his colleagues which noted that less than one per cent of all publishing scientists in the period from 1996 to 2011 published something in each and every year of this 16-year period.

Then there was have a trailer for a wonderfully silly forthcoming film, Lucy, in which Scarlett Johansson suffers from a drug overdose that leads her to learn Chinese in an hour and develop an uncanny ability to make men fall over by merely pouting at them. Morgan Freeman plays a top neuroscientist who explains that whereas the rest of us use a mere ten per cent of our brain capacity, Johansson's character has access to a full hundred per cent of her brain.

And today I've just read an opinion piece in Prospect Magazine by the usually clear-thinking philosopher, A. C. Grayling, which states: Neuropsychology tells us that more than ninety per cent of mental computation happens below the level of awareness.

Examples like these can be used to demonstrate just how careful you need to be when interpreting percentages. There are two issues. For a start, a percentage is uninterpretable unless you know the origin of the denominator (i.e., the total number of cases that the percentage is based on).  I'm sure the paper by Ioannidis and colleagues is competently conducted, but the result seems far less surprising when you realise that the 'less than one per cent' figure was obtained using a denominator based on all authors mentioned on all papers during the target period. As Ioannidis et al noted, this will include a miscellaneous bunch of people, including those who are unsuccessful at gaining research funding or in getting papers published, those taking career breaks, people who are trainees or research assistants, those working in disciplines where it is normal to publish infrequently, and those who fit in  research activity around clinical responsibilities. Presumably it also includes those who have died, retired, or left the field in the study period.

So if you are someone who publishes regularly, and are feeling smug at your rarity value, you might want to rethink. In fact, given the heterogeneity of the group on whom the numerator is based, I'm not sure what conclusions to draw from this paper. Ioannidis et al noted that those who publish frequently also get cited more frequently – even after taking into account number of publications and concluded that the stability and continuity of the publishing scientific workforce may have important implications for the efficiency of science. But what one should actually do with this information is unclear. The authors suggest that one option is to give more opportunities to younger scientists so that they can join the elite group who publish regularly. However, I suspect that's not how the study will be interpreted: instead, we'll have university administrators adding 'continuity of publishing record' to their set of metrics for recruiting new staff, leading to even more extreme pressure to publish quickly, rather than taking time to reflect on results. A dismal thought indeed.

The other two examples that I cited are worse still. It's not that they have a misleading denominator: as far as one can tell, they don't have a denominator at all.  In effect, they are quasi-statistics. Since the publication of the Lucy trailer, neuroscientists have stepped up to argue that of course we use much more than ten per cent of our brains, and to note that the origin of this mythical statistic is hard to locate (see, for instance here and here). I'd argue there's an even bigger problem – the statement can't be evaluated as accurate or inaccurate without defining what scale is being adopted to quantify 'brain use'. Does it refer to cells, neural networks, white matter, grey matter, or brain regions? Are we only 'using' these if there is measurable activity? And is that activity measured by neural oscillations, synaptic firing, a haemodynamic response or something else?

In a similar vein, in the absence of any supporting reference for the Grayling quote, it remains opaque to me how you'd measure 'mental computation' and then subdivide it into the conscious and the unconscious. Sure, he's right that our brains carry out many computations of which we have no explicit awareness. Language is a classic case – I assume most readers would have no difficulty turning a sentence like You wanted to eat the apples that she gave you into a negative form (You didn't want to eat the apples that she gave you) or a question (Did you want to eat the apples that she gave you?) but unless you are a linguist, you will have difficulty explaining how you did this. I don't take issue with Grayling's main point, but I am surprised that an expert philosopher should introduce a precise number into the argument, when it can readily be shown to be meaningless.

The main point here is that we are readily impressed by numbers. A percentage seems to imply that there is a body of evidence on which a statement is based. But we need to treat percentages with suspicion; unless we can identify the numerator and denominator from which they are calculated, they are likely to just be fancy ways of trying to persuade us into giving more weight to an argument than it deserves.

Thursday, 26 September 2013

Raising awareness of language learning impairments


A couple of years ago I did a Google search for ‘Specific language impairment’. I was  appalled by what I found. The top hit was a video by a chiropractor who explained he’d read a paper about neurological basis of language difficulties; he proceeded to mangle its contents, concluding that cranial osteopathy would help affected children.

I’ve previously described how I got together with colleagues in 2012 to try and remedy this situation, culminating in a campaign for Raising Awareness of Language Learning Impairments (RALLI). The practicalities have sometimes been challenging but I’m pleased to say that the collection of videos on our RALLI site has now attracted over 90,000 hits, providing an accessible and evidence-based source of information about developmental language impairments. As well as research-based films we have videos with practical information for parents, children and teachers.

So here, for those of you interested in this topic, is an index of what we have so far:

Background to RALLI

Research topics

Information for teachers

Support for parents and children

International
     Spanish translations/subtitled versions
Reference  
Bishop, D. V. M., Clark, B., Conti-Ramsden, G., Norbury, C., & Snowling, M. J. (2012). RALLI: An internet campaign for raising awareness of language learning impairments Child Language Teaching & Therapy, 28 (3), 259-262 DOI: 10.1177/0265659012459467

Saturday, 22 December 2012

Genes, brains and lateralisation: how solid is the evidence?


If there were a dictionary of famous neurological quotes, “Nous parlons avec l'hémisphère gauche” by Paul Broca (1865) would be up there among the top hits. Broca’s realisation that the two sides of the brain are functionally distinct was a landmark observation. It was based on a rather small series of patients, but has since been confirmed in numerous studies. After localised brain injury, aphasia (language impairment) is far more likely after damage to the left side than the right side. And nowadays, we can visualise greater activation of the left side in neurologically intact people as they do language tasks in a brain scanner.

There are many fascinating features of cerebral lateralisation, but I’m going to focus here on just one specific question: what do we know about genetic influences on brain asymmetry in humans?  There are really two questions here: (1) how do genes lead to asymmetric brain development? (2) are there genetic variants that can account for individual variation?  – e.g. the fact that a minority of people have right hemisphere language. I hope to return to question 2 at a later date, but for now, I’ll focus on question 1, because after reading a key paper on this topic, I've struck a whole load of questions that I can’t answer. I’m hoping that some of my genetically-sophisticated readers will be able to help me out.

It’s sometimes stated that cerebral lateralisation is a uniquely human trait, but that’s not true. Nevertheless, we are very different from our primate cousins, insofar as we show a strong population bias to right-handedness, and most people have left-hemisphere language. There are other species which show consistent brain asymmetries, but they are a long way from us on the evolutionary tree. Most of the research I’ve come across is on nematode worms, zebrafish, or songbirds. This is a long way from my comfort zone, but there are some nice reviews that document research on genes influencing asymmetries in these creatures (e.g. here and here). It’s clear, though, that it’s complicated: not just in terms of the range of genes involved, but in the different ways they can generate asymmetry. And there don't seem to be obvious parallels to human brain development.

Despite all this uncertainty, there’s growing evidence that brain asymmetries are present from very early on in life –in newborn babies and even in foetal life. This field is still in its infancy (forgive the pun), and samples of babies are typically too small to reveal reliable relationships between structure and function. Nevertheless, there’s considerable interest in the idea that physical differences between the two sides of the brain may be an indicator of potential for language development.

A particularly exciting topic is genetic determinants of cerebral lateralisation. One study in particular, by Sun et al made a splash when it was published in Science in 2005, since when it has attracted over 140 citations. The authors looked for asymmetric gene expression in post mortem embryonic brains. Their conclusions have been widely cited: “We identified and verified 27 differentially expressed genes, which suggests that human cortical asymmetry is accompanied by early, marked transcriptional asymmetries.” The fact that several different genes were identified was of particular interest to me, because genetic theories by neuropsychologists have typically assumed that just a single gene is responsible for human cerebral lateralisation. I’ve never found a single-gene theory plausible, so I was all too ready to accept evidence that involved multiple genes. But first I wanted to drill down deeper into the methods to find out how the authors reached their conclusions. I’m a psychologist, not a geneticist, and so this was rather challenging. But my deeper reading raised a number of questions.

Sun et al used a method called Serial Analysis of Gene Expression (SAGE) which compares gene expression in different tissues or – as in this case – in corresponding left and right regions of the embryonic brain. The analysis looks for specific sequences of 10 DNA base-pairs, or tags, which index particular genes. SAGE output consists of simple tables, giving the identity of each tag, its count (a measure of cellular gene expression) and an identifier and more detailed description of the corresponding gene. These tables are available for left and right sides for three brain regions (frontal, perisylvian and occipital) for 12- and 14-week old brains, and for perisylvian only for a 19-week-old brain. The perisylvian region is of particular interest because it is the brain region that will develop into the planum temporale, which has been linked with language development.  One brain at each age was used to create the set of SAGE tags.

To identify asymmetrically expressed genes the authors state performed a Monte Carlo test and verified this using the chi square test. I haven’t tracked down the specifics of the Monte Carlo test, which is part of the SAGE software package, but the chi square is pretty straightforward, and involves testing whether the distribution of expression on left and right is significantly different from the distribution of left vs. right expression across all tags in this brain region – which is close to 50%.  In the left-right perisylvian region of a 12-week-old embryonic human brain, there were 49 genes with chi square greater than 6.63 (p < .01): 21 were more highly expressed on the left and 28 more highly expressed on the right.  But for each region the authors considered several thousand tags. So I wondered whether the number of asymmetrically expressed genes was any different from what you’d expect if asymmetry was just arising by chance.

It was possible to check this out from the giant supplementary Excel files that accompany the paper, but this proved far from straightforward.  It turns out that the relationship between tags and genes is not one-to-one.  For around 40% of the tags, there is more than one corresponding gene. It was not clear which gene was selected in such cases, and why. I did find some cases where two genes were assigned to a tag, but my impression was that this was unintentional and in general the authors aimed to avoid double-counting tags. We also have the further problem that some genes are indexed by numerous tags, a point I will return to below.

But let’s just focus first on the individual tags. I compiled a master list of all tags that were expressed in any region at any age, and then made a chart of the frequency of expression in each brain region/age. I excluded any tags where the total expression count on both sides was three or less, as this is too small to show lateralisation, and this left me with 3800 to 4600 tags for analysis in each brain region. I did compute chi square as described by Sun et al, but this is not recommended for small numbers, and so I also evaluated the significance of asymmetry using a two-tailed binomial test. This doesn’t make a huge difference, but is more accurate when comparing small numbers.  Figure 1 shows the proportion of the sample for each brain region where the binomial test gives a p-value of a given size. If the distribution of expression in left and right was purely determined by chance, we’d expect the points to fall on the line. If there were genes for asymmetry we would expect the observed values to fall above the line, especially at low levels of p. It is clear this is not the case. I did cross-check my figures against those of Sun et al, and found they appeared to have missed some cases of significant asymmetry, which meant that in general they found rather fewer cases of significant asymmetry than are shown in Figure 1.

Fig 1. Proportion of tags with "significant" asymmetry, by Age/Brain Region

Sun et al didn’t rely solely on statistical tests of SAGE data to establish asymmetrical expression.  They reported validation studies using a different method for assessing gene expression (real-time PCR). But this used genes selected on the basis of a chi square value of 1.9 or greater (P < .17), which included many where the degree of asymmetry was not large. One goal of PCR analysis was to confirm asymmetric expression levels in the same embryonic brains as the SAGE analysis. Of more interest is whether the findings generalise to new brains. The authors did further cross-validation using real-time PCR with six additional brains of different ages, and reported results for the LMO4 gene, where higher perisylvian expression on the right was evident in two brains at 12 and 14 weeks of age, as well as in the original two brains of the same age. Four other brains, aged 16 to 19 months, did not show asymmetry of expression. Some of the other asymmetrically expressed genes were also tested using real-time PCR in the two other brains, and 27 showed consistent asymmetric expression. It was, however, not clear to me how the significance of asymmetry was assessed in these replication samples.

There is one particular issue I find confusing when I try to evaluate the robustness of the asymmetry results. My expectation was that if a gene was asymmetrically expressed, then this should be evident in all the tags indexing that gene. But Table 1 shows that this isn’t so. For the LMO4 gene, which is the focus of special attention in this paper, there are seven tags that are linked with the gene in at least one brain region: only one of these (in red) shows the rightward asymmetry that is the focus of the paper. Another tag (in blue) shows leftward asymmetry in one sample, and the rest have low levels of expression. Maybe there’s a simple explanation for this – if so I hope that expert geneticists among my readers may be able to comment on this aspect.
Table 1. Left- and right-expression levels for seven tags for the LMO4 gene
I’m aware of two other studies (here and here) that looked for asymmetric gene expression in embryonic human brains but failed to find it . One possible reason for this discrepancy is that these studies focused on later stages of development, rather than the 12-14 week-old period where Sun et al found asymmetry. In addition, power is always low in these studies because of the small number of brains available. As Lambert et al (2011) noted, as well as possible effects of age and gender, there may be individual variation from brain to brain, but typically only one or two samples are available at each age.

So what do I conclude from all of this? I realise for a start that these studies are very hard to do. I also realise we have to make a start somewhere, even if the amount of post mortem material is limited. But I have to say I’m not convinced from the evidence so far that the researchers have demonstrated significant asymmetry of genetic expression in embryonic brains. The methods seem to take insufficient account of the possibility of chance fluctuations in the measurements, and the numbers of asymmetries that have been found don't seem impressive, given the huge number of genes that were investigated. Clearly, something has to be responsible for the physical asymmetries that have been found in foetal and neonatal brains, and the odds seem high that genes are implicated. But is the evidence from Sun et al convincing enough to conclude that we have found some of those genes? I'd love to hear views from readers who have more expertise in this area of research.

P.S. 7th Jan 2013
Thanks to Silvia Paracchini, who drew my attention to further relevant articles:
Johnson, M. B., et al (2009). Functional and evolutionary insights into human brain development through global transcriptome analysis. Neuron, 62(4), 494-509. doi: 10.1016/j.neuron.2009.03.027
This paper looked at a slightly later developmental stage - 18 to 23 weeks gestational age - and did correct for the number of genes considered (False Discovery Rate). They reported striking symmetry of gene expression in the mid-gestational period, even though structural brain asymmetries have been described at this stage of development. Note, however, that this is not incompatible with Sun et al, who did not find evidence of asymmetry after 17 weeks gestational age.

Kang, H. J., et al (2011). Spatio-temporal transcriptome of the human brain. Nature, 478(7370), 483-489. 
This is a much larger study, covering the range from 4 weeks gestational age through childhood up to adulthood and old age. This paper does not explicitly report on asymmetry, but they describe genes where the expression varies from brain region to region, or from age to age, after adjustment for False Discovery Rate. I could find no overlap in the list of the genes identified by Sun et al and Kang et al's list of differentially expressed genes.

Reference

Abrahams, B. S., Tentler, D., Peredely, J. V., Oldham, M. C., Coppola, G., & Geschwind, D. H. (2007). Genome-wide analyses of human perisylvian cerebral cortical patterning. Proceedings of the National Academy of Sciences, 104, 17849-17854.
 

Dehaene-Lambertz, G., Hertz-Pannier, L., & Dubois, J. (2006). Nature and nurture in language acquisition: anatomical and functional brain-imaging studies in infants. Trends in Neurosciences, 29, 367-373.
 

Kivilevitch, Z., Achiron, R., & Zalel, Y. (2010). Fetal brain asymmetry: in utero sonographic study of normal fetuses. American Journal of Obstetrics and Gynecology, 202(4). doi: 359.e1
10.1016/j.ajog.2009.11.001
 

Lambert, N., Lambot, M.-A., Bilheu, A., Albert, V., Englert, Y., Libert, F., . . . Vanderhaeghen, P. (2011). Genes expressed in specific areas of the human fetal cerebral cortex display distinct patterns of evolution. PLOS One, 6(3), e17753. doi: 10.1371/journal.pone.0017753
 

Lash, A. E., Tolstoshev, C. M., Wagner, L., Schuler, G. D., Strausberg, R. L., Riggins, G. J., & Altschul, S. F. (2000). SAGEmap: A public gene expression resource. Genome Research, 10(7), 1051-1060. doi: 10.1101/gr.10.7.1051
 

Sagasti, A. (2007). Three ways to make two sides: Genetic models of asymmetric nervous system development. Neuron, 55(3), 345-351. doi: 10.1016/j.neuron.2007.07.015
 

Sun T, Patoine C, Abu-Khalil A, Visvader J, Sum E, Cherry TJ, Orkin SH, Geschwind DH, & Walsh CA (2005). Early asymmetry of gene transcription in embryonic human left and right cerebral cortex. Science (New York, N.Y.), 308 (5729), 1794-8 PMID: 15894532

Sun, T., & Walsh, C. A. (2006). Molecular approaches to brain asymmetry and handedness. Nature Reviews Neuroscience, 7, 655-662.