Rachel Feltman: There has been a lot of hype around artificial intelligence lately. Some companies would like us to believe that machine learning is powerful enough to actually tell the future. But what about using AI to explore the past and converse with people from long-extinct civilizations?
for scientific americansan’s science quickly, I’m Rachel Feltman. Today’s guest is Michael Varnum, chair of the social psychology field and associate professor at Arizona State University. He is one of the co-authors of a recent opinion paper proposing a rather spooky new use for tools like ChatGPT.
Michael, thank you so much for joining us today.
About supporting science journalism
If you enjoyed this article, please consider supporting our award-winning journalism. Currently subscribing. By subscribing, you help ensure future generations of influential stories about the discoveries and ideas that shape the world today.
Michael Varnum: you’re welcome. Thank you for having me on board.
Feltman: This new paper has the feel of a “ghost in the machine” (lol). Please tell us a little about the problem you are trying to solve.
Varnum: Well, I’ve been interested in thinking about cultural change for a while and have done a lot of research in that area. But when we try to gain insight into the minds and behaviors of people who are no longer with us, we run into some limitations. Obviously we don’t have a time machine, right? You can’t bring the dead back to life to participate in experiments or economic games.
So usually what people like me have to do is use a rather indirect proxy. Perhaps we can obtain archival data on marriages, divorces, crimes, etc., or look at cultural artifacts such as the language people used in books to understand what kinds of values people had. and try to guess what kind of emotions they were having. I had it for different kinds of groups. But it’s all indirect.
What would be amazing is if we could actually get the kind of data that we get from people today, like the ancient Romans or the Vikings or the medieval Persians. One of the things that’s really excited me in the last year or two is that you can use programs like ChatGPT to at least simulate modern participants, and amazingly, you can use programs like ChatGPT to simulate all the classic effects in behavioral science. What people are starting to realize is that it can be replicated.
So we thought, “If we can do this based on a model created from the texts of modern humans, maybe we can do this based on the texts of ancient humans.” This could open up a whole new world of possibilities. ”
Feltman: Well, could you tell us a little more about some of the experiments that used language learning models to recreate psychological phenomena?
Varnum: One of the more powerful experiments aimed to replicate 70 different large-scale survey experiments with simulated ChatGPT participants, and the results were about 0.9 times better than what people observed with real humans. It turns out that there is a correlation. And of course this wasn’t designed for someone to do Llama or ChatGPT…
Feltman: Hmm, hmm.
Varnum: But in the process of creating these models that can talk to us in a very natural way, they seem to have captured quite a bit of human psychology.
Feltman: And you mention in your paper that some people are already using historical documents to train large-scale language models, but what have they done so far? mosquito?
Varnum: So far, these are just small steps.
Feltman: Hmm, hmm.
Varnum: People are just trying to understand, “If I train a model on medieval European texts, how does that model understand the solar system, or medicine or biology?” And the number of planets is wrong. They believe in the four humors of the body.
So far, to my knowledge, no one has implemented this kind of fine-tuned model through modern experimentation or research. In fact, I think it will start happening soon. I’m really excited to see what people discover.
Feltman: Yeah. So one of the things that came to mind when I was reading your paper was the inherent biases that we see in the fossil record. You know, our sense of what life was like in the past is influenced by what’s preserved, and that’s all sorts of things like climate and the bodies of the creatures we’re talking about. influenced by factors. And I think that in most historical times and places, certain people were overrepresented in written texts. So how do researchers suggest navigating that so that we don’t really get a biased sense of what people were like?
Varnum: For this kind of proposal, it’s a really tricky challenge…
Feltman: Hmm, hmm.
Varnum: After all, for most of human history, no one was literate, right? The writing is relatively recent…
Feltman: right.
Varnum: And in the days when writing existed in some societies, very few people actually knew how to read or write. Even fewer wrote down what has survived to modern times. This means that the data obtained will be biased towards the more elite and better educated.
Feltman: Hmm.
Varnum: I think there are several ways to deal with this, but they’re imperfect, right? But perhaps if you use them in combination, you can address a little bit of the bias that goes into these models.
One way is that we know quite a bit about how things like social class affect the psychology of modern humans…
Feltman: Hmm.
Varnum: So potentially you could tweak these models a little bit or run them through experimentation or research and wait for the model’s response to account for its biases. As you know, in some cases other historical records and sources of analysis also exist. Perhaps as long as we have a broader view of the spirituality and behavior patterns of people in the past, we can see whether the results of these large-scale historical language models are consistent with those kinds of conclusions. But it’s certainly difficult. That’s going to be a real challenge to overcome.
Feltman: Of course, that’s not a challenge unique to using historical data. This is also a challenge when training LLMs using up-to-date data.
Varnum: Oh, of course you do, right? And, you know, one of the things that inspired this idea was the work of people like Mohammad Attari and Yang Tao, and that our current large-scale language models are more closely aligned with Western psychology. It actually looks kind of strange in the sense that it does. And that’s not surprising, given that the training data over-represents these societies. But this suggests that another kind of corpus could capture some of the cultural zeitgeist and culturally specific spirituality of the people who created it, so It’s also exciting in a way.
Feltman: Well, could you tell me what WEIRD stands for in this context? I think it’s a very good abbreviation (lol), so…
Varnum: Yes, this is an acronym developed by Joe Henrich about 15 years ago that stands for Western, Educated, Industrialized, Wealthy (and Democratic). And it turns out that a minority of the current human race lives in such a society.
Feltman: Hmm, hmm.
Varnum: However, depending on how you look at it, the majority of behavioral science participants come from these samples.
And this is important because culture influences us in many ways, from the values we hold, to the social distance we prefer in public, to our basic patterns of visual attention and cognition, to our viewer ratings. Because we know that it influences the way we think and behave. of cooperation. It’s a very long list.
Feltman: No, I mean, you can definitely imagine — you know, it’s obviously very exciting to talk about ancient history, but it’s hard to imagine that researchers, for example, in the 19th century, 20th century, even the 21st century… I can definitely imagine trying to use a century of texts from underrepresented groups to sort of re-examine these psychological studies that may have left out large parts of the population. Ta.
Varnum: Yeah, I think that’s an incredibly good idea. And in some ways, the further we go back into the past, the easier this kind of research becomes.
Feltman: Yeah.
Varnum: So it’s exciting to think about pushing the limits so far back in time, but maybe the starting point is, you know, the starting point is, “Let’s look back 100 or 150 years.” .
Feltman: Hmm, yeah, yeah, now that I think about it, if you just imagine this idea taking off completely and having a bunch of undead psychology projects going on, what would it look like? , do you have any dream use cases for this?
Varnum: So I’m doing a lot of research based on evolutionary psychology.
Feltman: Hmm, hmm.
Varnum: And sometimes we run experiments and surveys and try to get data from all continents around the world to see if some parts of human psychology are universal. And when we find that, it’s really exciting, but we go from saying, “You know, it’s universal and it makes sense adaptively,” to saying, “This is how people thought in the past.” Yes, especially in the deep past.”
And you can push back that temporary window…
Feltman: Hmm, hmm.
Varnum: You know, people like (Douglas) Kenrick and (David) Schmidt have found differences between men and women in their preferred sexual strategies. In other words, do you want or prefer to have a committed relationship with multiple partners? Do you have more exclusive relationships and fewer partners? And while that seems to be the case all over the world, if we start looking at them from the societies that lived hundreds and thousands of years ago, we have more confidence that these things are truly a core part of human nature. I think you will be able to have it.
Feltman: Completely.
Varnum: This idea is advanced and kind of speculative, isn’t it? My computer doesn’t have any of these capabilities readily available, but Sachin Banker and colleagues recently had GPT-4 generate dozens of new hypotheses for social psychology research and then put them into practice. published a paper that led social psychologists to generate a new hypothesis. hypothesis.
Feltman: Hmm.
Varnum: And it turns out other social psychologists thought the AI was coming up with more plausible, and possibly true, ideas.
Feltman: Hmm, that’s interesting.
Varnum: Therefore, in the future, AI may be used not only to simulate participants and code data, but also to generate ideas. You can imagine such strange kinds of closed loops. People like me may lose their jobs there.
Feltman: (Laughs) Well, I hope not. You know, I think there’s always room for that unique human element. But I think it’s great to think about how AI can become a really interesting tool for us. Thank you so much for taking the time to chat with us today.
Varnum: Oh, thank you, Rachel. This was my joy. I enjoyed the conversation.
Feltman: That’s all about this Friday’s charm. We’ll send you a weekly news roundup on Mondays. And on Wednesday, we’ll talk about something just as spooky as the ghost of AI: the psychology of Black Friday shopping.
science fast It is produced by me, Rachel Feltman, Fonda Mwangi, Kelso Harper, Madison Goldberg, and Jeff DelVisio. Shayna Possess and Aaron Shattuck fact-check the show. Our theme music was composed by Dominic Smith. Subscribe scientific american Check out more latest and in-depth science news.
for scientific american, Rachel Feltman. Have a wonderful weekend!
