Word Up

Brain Translation of Words:An fMRI decoding study of speech recognition
Joao Correia, Milene Bonte, Giancarlo Valente, Lars Hausfeld, Elia Formisano, Bernadette Jansma; Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, and Maastricht Brain Imaging Center (M-BIC), Maastricht, The Netherlands
Neurobiology of Language Conference; San Sebastian, Spain; October 2012

Wegman reading two books

How do we represent the meaning of words independent of the language we are listening to?

This fMRI study investigates the neural network of speech processing responsible for transforming sound to meaning, by exploring the semantic similarities between bilingual wordpairs. Eight native Dutch participants with high proficiency of English listened to four different nouns (animals), either spoken in Dutch or in English.

These nouns were presented in separate runs for each language while participants were asked to detect non-animal targets (11% of the trials) within a list of animal non-target items. Activity patterns elicited by these non-target stimuli was analyzed using Machine learning methods and multivariate classifiers.
Firstly, to identify brain regions generally involved in spoken word processing, we let the classifier discriminate between word pairs within the same language (e.g. bull vs. horse).

Secondly, to isolate language-independent semantic/conceptual representations in these regions, we assessed the ability of multivariate classifiers trained within one language (e.g. bull vs. horse) to generalize to the other language (e.g. the Dutch equivalents ‘stier’ vs. ‘paard’).

The results of our discrimination analysis show that word decoding involves a distributed network of brain regions consistent with the proposed ‘dual-stream model’ (Hickok and Poeppel, 2007). The results of our generalization analysis highlights a focal and specific role of a left anterior temporal area in semantic/concept decoding. Together, these distributed and focal brain activity patterns subserve the extraction of abstract semantic concepts from acoustically diverse English and Dutch words during bilingual speech comprehension.

note:
There is one major drawback to the process, which quashes any visions of a full-on real-time mind translation machine hitting stores anytime soon — the neural activity patterns differed slightly from person to person. Our neurons learn and identify in unique ways, and understanding these pathway patterns through machine learning would be a long process. “You would have to scan a person as they thought their way through a dictionary,” said Matt Davis of the MRC Cognition and Brain Sciences Unit in Cambridge. It would be difficult to translate a mind now without this concept map. However, we are only at the beginning of this line of study, and an algorithm could potentially be devised to aggregate hundreds of neural activity patterns to help indicate what the brain activity of an individual unable to communicate represents.
via

POST SCRIPT
Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
H. Andrew Schwartz, Johannes C. Eichstaedt, Margaret L. Kern, Lukasz Dziurzynski, Stephanie M. Ramones, Megha Agrawal, Achal Shah, Michal Kosinski, David Stillwell, Martin E. P. Seligman, Lyle H. Ungar
Sept. 25, 2013

We analyzed 700 million words, phrases, and topic instances collected from the Facebook messages of 75,000 volunteers, who also took standard personality tests, and found striking variations in language with personality, gender, and age. In our open-vocabulary technique, the data itself drives a comprehensive exploration of language that distinguishes people, finding connections that are not captured with traditional closed-vocabulary word-category analyses. Our analyses shed new light on psychosocial processes yielding results that are face valid (e.g., subjects living in high elevations talk about the mountains), tie in with other research (e.g., neurotic people disproportionately use the phrase ‘sick of’ and the word ‘depressed’), suggest new hypotheses (e.g., an active life implies emotional stability), and give detailed insights (males use the possessive ‘my’ when mentioning their ‘wife’ or ‘girlfriend’ more often than females use ‘my’ with ‘husband’ or 'boyfriend’). To date, this represents the largest study, by an order of magnitude, of language and personality.
-via io9