EDDIE JACKSON: This is Metastasis. I'm your host, Eddie Jackson.

​

In this special episode, our senior producer Yesmar Oyurzun sat down with linguistic anthropologist Dr. Beth Semel, to talk about how different technological tools are used in biomedicine.

​

In her own work, Dr. Semel studies how data driven diagnostic techniques are used to screen for stages of mental health. She explained that psychiatrists tried to use speech to predict, for instance, a manic episode, psychiatrist went in to extract signals from what's known as para linguistic components of speech, like pitch, energy, rate of articulation, and breathiness.

 

The engineers would then analyze these kinds of sounds as signals, to find connections between the physical properties of speech sounds and changes in the brain that would happen in someone with a mental illness.

​

The only catch was that this technology didn't work.

​

Today, we're asking a couple of different questions. How do we know what kinds of information are useful? When can a useful signal cut through noise? How do we recognize the noise?

​

BETH SEMEL: So what does it mean to suggest or imply that a machine or a computer is listening to machine listening a metaphor? Is it an analogy? And moreover, what does the gendering of technologies mean for humans?

​

BETH SEMEL: I had had these two, what felt like somewhat competing interests in kind of like the state of psychiatry and psychology as a science, offensively it's a medical field. And yet it seems to revolve around what we might call like low tech kind of tools, like a lot of pen and paper based inventories. And just the fact alone to that the main mechanism for doing mental health care is conversation and speech.

 

And so I was interested both in the kind of politics of what it means to call something biomedicine when it is kind of dealing with these objects that don't really have like a stably- pinned down biomedical existence, like they haven't been kind of pushed through that grid. And at the same time, just from the from a language and interpretation standpoint, what is it like for mental health care workers, what is it like for people who are who are from a practitioner standpoint, to come to understand and come to, I guess, really feel like they are experts in something that is so you know, obviously, the stakes are very high, especially in a place like the United States where the mental health care system is so tied up with insurance and also with prisons, and just because of the way that social services in the US are kind of wrapped up with the prison industrial complex.

 

I initially started out looking more at the practitioner side and learning a little bit more about kind of what was going on in psychiatric research, mental healthcare research, in what at the time when I happened to be doing my PhD, which was a time of great change led me into this world where mental health care people are working with engineers to really radically kind of like totally change the way that they ask questions, and also the technologies that they that they typically use. I happen to find this strange corner of research on vocal biomarkers which do united together my interest in the you know, the language side. And then in the science-y side.

​

YESMAR OYARZUN: I know when the my a-ha moment was, but I did not come to Rice do not start my PhD program with that idea. I actually got it during preliminary research for a different idea. After being really inspired and really sort of interested in some medical student movements in the US white coats for Black Lives in similar movements, where medical students were saying that medicine is racist, but also medical education is racist. And also we're being taught how to use race inappropriately in medicine, and also medicine is built on racism and  racism as a public health issue. And as a medical issue. That movement grew sort of in tandem with Black Lives Matter movement, in the sense that it was sort of started by people with the same interest in sort of broke up off in its own ways and says like, this is going to be our corner of that. And so that's what I started graduate school trying to do.

​

I had started off in DC and witnessed it and DC is kind of an interesting place to be Because there's something about being sort of close to legislation and being able to advocate like literally at the President's doorstep, that's a little bit different about any sort of social movement. And so I went and talk to students there, because they just have like a little bit of a different perspective than students really anywhere else.

​

But when I asked students what was of interest to them, what was like the most pressing issue for them right now, and specifically, where in medical school, they felt they weren't getting the right education, or they weren't getting enough education about race or racism, or black people, whatever their sort of political issue was, and students were coming from all kinds of different areas. And it so happened that white students, black students, Latino students, many of them said, you know, we don't get a good education in dermatology. If you open up a dermatology textbook, all you see is white skin. And that was like my moment, that was where I said, you know, what, this is the project that I need to do, I need to figure out why it is that dermatology have sort of any field has non diverse text textbooks. And what I've started to like piece together since then, is that like, maybe all textbooks actually feature mostly white people. But on dermatology, people can see because it's not a picture of a sort of internal organ that you can't really like, that's not how we judge race.

​

BETH SEMEL: I just want to talk about that forever. But I think what's so what's so great about their project, and so fascinating, but also so important is just how it really gets that how much the skin is involved in a kind of like racial imaginary as being like the start and end point of race. But then at the same time, as you just pointed out, it's like, clearly, there's so many other things are raised and racialized and intersect with these, these categories that we call that we call race and race in the context of the people I study, it's such a, it's such a weird floating variable, it was really not talked about in my fieldwork at all, it came up in very strange ways.

 

And it came up in ways that you wouldn't, maybe the people working on it wouldn't even necessarily call it race. So these technologies or these, you know, projects that people are doing, where they're trying to detect, they're saying we can we come up with an algorithm or whatever that can detect COVID by analyzing people's coughs. But when you look into the patents that are associated with these technologies, they will have like, kind of nested or coupled within them. Things like okay, well, we are controlling for gender and country of origin, and race.

 

BETH SEMEL: But what that means is that they're treating those as like, these are our you know, our algorithm will detect those things, too, and hold them in abeyance and just consider COVID, for instance. So a weird kind of byproduct ends up being like replicating the notion that things like country origin, or gender or race are like biological things, and they exist through like the voice that can map onto and therefore like, reifies, and affirms the biological existence of all these other things. And it's not ever something that somebody will come out and say, This is what we're trying to do. Or this is like what, you know, our technologies are designed to identify, but there's something about there's something in that act of saying, well, we can, oftentimes technologies, these technologies as a kind of throwaway will be like, well, we detect gender, the gender of a speaker, when you really look at it, that's not that's not the case at all, there's no such thing as it, a voice is just like a bundle of sounds doesn't have any gender to it. And voices change. Over time. They're not consistent, they're pliable, they change as we grow, and our vocal cords, take shape, and, you know, degrade or whatever.

​

YESMAR OYARZUN: This is interesting, because you're talking about sound, which is sort of a product of the body, and I'm talking about skin, which is on on the body. And I think what's really interesting in my case is that when people try to like look at race in isolation from some kind of like body part, they also make those mistakes, but we don't like assume that those mistakes can be made, because we are so sure that we know what race is. And that skin color does tell us what races and in our interview with Dr. Murina who we're going to have on for another episode, she works in New Orleans and I don't know if you know the demographics of New Orleans, but in short, there are a lot of light skinned black people, especially in New Orleans.

 

And when she goes and she goes to a dermatology conference and she so someone a picture of sort of an isolated body part that's not a person's whole face or a person so it's just a skin tag on an arm, right? And she says, you know, an African American patient came in, someone will be like, "But that's not a black patient." She's just like, "Yeah, they they identify as black." But when you see the body in isolation and you see light skin, you can be so sure that you're looking at a white person that you can still get race wrong. Even if if you had seen the whole body, you'd have gotten race, right? Which just like, I think, shows how tenuous all of these like categorizations are.

 

I think it's similar to sound and that in isolation, people are so sure that they're hearing someone black talk, or they're hearing someone white talk about their hearing a man or woman or whatever, you know, they're so sure that they can get it wrong, but they would have gotten it wrong if it were a real person or a person in full, because those things aren't actually real. And we're also like, enculturated, to speak certain ways. And there's like, a gay accent or you know what I mean, and like, part of it is adopted, and part of it is not, and so like, it's just like very, other than, like, maybe an albino person, people think I can race people, for the most part. And that's just like, really strange.

​

BETH SEMEL: Yeah, yeah, the people I study and maybe just in this kind of weird world of automated voice analysis, or machine listening, that sometimes is called, it's sort of like an argument made in the opposite direction, where like, as you like, you were saying, if you break the voice down, and totally extract it, and extrapolate it from the body from its context of utterance, like, we shouldn't be thinking about context, because that somehow is racist and sexist.

 

So it's this weird kind of liberal like, colorblind, or, you know, color, not hearing of difference. Indeed, the way that they transform these spoken utterances, which are in a highly choreographed setup, where people interviewing research subjects trying to get speech for a data set, by the time you get to an algorithm, it's not even a voice. It's like code that's based on analysis of like a signal. And so it's so abstracted from any kind of context.

 

BETH SEMEL: And indeed, they're trying to get at these like, signals in the brain that we can better reverse engineer from sound to tell us something about what mental illness is like and how it manifests. But like, what then happens nevertheless, is that they're still it's very productive, the way that Benjamin talks about racism and sexism, transphobia, etc, being productive, like these things like produce and continually press, things like race and gender and also ability into association with biological things. So even while they're being contested as like, No, no, that's, you know, we're just dealing with the brain. We're dealing with sound and it's, you know, at the level of physics and its purest form, still, nevertheless, it's always tangled up in the experimental setups that produced the datasets or in the marketing or advertising, or even just in terms of like, who even are the research subjects because most of the research subjects that as we said I experienced were were white.

​

​

​