Sasha Mitts 0:01 So I've always found myself drawn to understanding the way people make sense of the world before really knowing that AI was going to become this big thing. Did AI research all those years ago, maybe like 15 years ago now, was using early neural networks just to predict random decision making behavior because we thought maybe there were some patterns underlying what people thought they were doing randomly. You know, we were asking them to pull marbles out of the bag and guess the color, which is. There's no good way to do that. But it turned out you actually can make predictions over random behavior. And that curiosity grew over time. We realized that there was not just an opportunity to design better things there, but to leverage all of the data from across the health system and work with various departments to do more predictive understanding of their patients.

Speaker B 0:45 Hello, fellow insight seekers. I'm your host, Molly, and welcome to the Curiosity Current. We're so glad to have you here.

Stephanie Vance 0:53 And I'm your host, Stephanie. We're here to dive into the fast moving waters of market research where curiosity isn't just encouraged, it's essential.

Speaker B 1:03 Each episode, we'll explore what's shaping the world of consumer behavior. From fresh trends and new tech to the stories behind the data, from bold

Stephanie Vance 1:12 innovations to the human quirks that move markets, we'll explore how curiosity fuels smarter research and sharper insights.

Speaker B 1:20 So whether you're deep into the data or just here for the fun of

Stephanie Vance 1:23 discovery, grab your life vest and join us as we ride the Curiosity Current. Today on the Curiosity Current, we are joined by Sasha Miz, an independent researcher who studies human centered AI. Sasha spent over four years as a UX researcher at Fair Meta's Fundamental AI Research lab before setting out to run his own research practice.

Speaker B 1:48 As AI becomes even more and more involved in research workflows, it brings an interesting contradiction to the surface. These models are designed to produce confident answers. While great research often depends on uncertainty, curiosity, and exploring multiple possibilities, Sasha's work

Stephanie Vance 2:05 challenges us to think differently about what AI is actually optimized for and whether those strengths always align with what researchers need to understand about people.

Speaker B 2:15 So today we're exploring the tensions between AIs drive for confident answers and researchers needs for uncertainty and greater exploration.

Stephanie Vance 2:23 Sasha, welcome to the show.

Sasha Mitts 2:24 Thanks for having me. I'm very excited.

Stephanie Vance 2:27 Same. It's interesting. Your work really seems to sit between rapidly advancing AI systems and the inherent messy reality of human behavior. And I'm very curious to know what first drew you to that intersection. ome to the show.

Sasha Mitts 2:24 Thanks for having me. I'm very excited.

Stephanie Vance 2:27 Same. It's interesting. Your work really seems to sit between rapidly advancing AI systems and the inherent messy reality of human behavior. And I'm very curious to know what first drew you to that intersection.

Sasha Mitts 2:42 So I've always found myself drawn to understanding the way people make sense of the world way back when. I decided to study philosophy and neuroscience together in college, mostly because I was trying to understand better how do our anatomy and our ways of reasoning collectively explain various theories of what is really going on out in the world and how we make sense of what's going on out in the world, whether it's to do with ethics or theories of mind, or theories of perception, these kinds of things, before really knowing that AI was going to become this big thing. Did AI research all those years ago, maybe like 15 years ago now, was using early neural networks just to predict random decision making behavior because we thought maybe there were some patterns underlying what people thought they were doing randomly. You know, we were asking them to pull marbles out of the bag and guess the color, which there's no good way to do that. But it turned out you actually can make predictions over random behavior. And that curiosity grew over time. I was later at a hospital and working in a small design group in the hospital that was trying to bring human centered practices to medicine, which, as anyone who's worked in healthcare knows, doesn't necessarily happen automatically. And we realized that there was not just an opportunity to design better things there, but to leverage all of the data from across the health system and work with various departments to do more predictive understanding of their patients. So rather than react after patients get sick, can you anticipate and diagnose things early on? And obviously AI in medicine is not a field that we've pioneered. It's something that has had legs for quite a long time because of the wealth of data, but there have been a few stepping stones. So the two that I mentioned and then as you kindly introduced, I was at fair for a while and I think their song an even richer interaction between understanding people and building completely new technology simultaneously. So it wasn't just how do we add a feature to an app, but how do we change the frontier of what technology can do? And I joined pre chatgpt when people still would look at me when I said AI and weren't entirely sure what I meant. Now under the modern era where every company has a AI department or head of AI or AI innovation, whatever, but the question still remains, it is hyped right now, but that doesn't mean it's a solution to everything. And there's so many layers to unpack to make sense of how these technologies don't just show up on a marketing slide, but actually change the qualitative experience for people day to day. And it's not the case that you can at least yet. Just ask a model, solve all my problems and come back to a well reasoned good solution that is a fit for you, your users, et cetera. And that ambiguity, I think is just fun. Figuring out in a principled fashion how to make use of the tools, figuring out how and where they can fit into the world to not undercut human creativity and agency, but lift those things up. It's definitely not a solved problem. I would like to say that I'm certain it's going to go well, but I think it actually will require like a lot of deliberate work from a lot of people to go well. uring out how and where they can fit into the world to not undercut human creativity and agency, but lift those things up. It's definitely not a solved problem. I would like to say that I'm certain it's going to go well, but I think it actually will require like a lot of deliberate work from a lot of people to go well.

Speaker B 5:38 I'm glad that you brought up a lot of those things because it's a very interesting space. I want to dig into one of the things that you sort of alluded to, which was the idea around outsourcing the brain capacity and just putting things out there to then create results, to just get it done. Because I don't think the models are there yet to do that. We know that large language models are uniquely trained to produce the best or most preferred response. Even like you see that on ChatGPT, which response do you prefer? It helps to craft the better response for you and it learns each time that a unique user interfaces with it. Research, on the other hand, can often mean that it's challenging hypotheses about our current understanding of things. So given those discrepancies, where do you see as the biggest tension between those two different ways of understanding the world? or you and it learns each time that a unique user interfaces with it. Research, on the other hand, can often mean that it's challenging hypotheses about our current understanding of things. So given those discrepancies, where do you see as the biggest tension between those two different ways of understanding the world?

Sasha Mitts 6:26 I think actually is even a place in research for that more high probability, on the rails point of view. As long as you're aware that that's what it is. I think there's like a risk when you look at that and you see that it's convincing and persuasive and you are drawn to trust it. That's where things can tend to go wrong. But it is helpful to know how might an average, really intelligent, informed person reason about this thing. But given the median distribution and you know, as models get smarter, we do see that their ability to think in more divergent ways or think along the margins if things get better. Certainly there's a whole field of context engineering where you give them more of the unique information or the relevant information and they make more informed, closer to high quality human decisions. But I think your point is still well taken. Doing good research requires thinking not just divergently, but skeptically, critically, rigorously, and to be able to have an opinion that you can defend when other people who may disagree with you challenge it. I Think a lot of this comes down to understanding the model's capabilities well enough that you rely on them for what they're good at, which is writing software, looking across large sets of data and finding patterns. But even in those cases, you have to be aware that the way in which they're going to find patterns are not necessarily the same way a human would think. We can talk about this more later, but I think there's a responsibility that researchers have and any users of these technologies professionally have to equip themselves with enough of an understanding of their strengths and weaknesses that are not pulled in by the persuasiveness. It feels good to be told that you're right or that the problem's already solved. Of course, it would be great if I could tap on my keyboard for 30 seconds and then hand over a deck that was perfect and that really actually solved the business's needs. But we're not quite there. And as you said to that point,

Stephanie Vance 8:15 let's talk a little bit about what happens when a researcher hasn't maybe built that discernment layer and that ability to be able to judge whether content is hitting the mark or not. How does AI's tendency to produce confident answers actually affect that researcher's ability to remain open to things like ambiguity, conflicting evidence, changing hypotheses, like, what are the risks? maybe built that discernment layer and that ability to be able to judge whether content is hitting the mark or not. How does AI's tendency to produce confident answers actually affect that researcher's ability to remain open to things like ambiguity, conflicting evidence, changing hypotheses, like, what are the risks?

Sasha Mitts 8:39 I think this actually touches on an important point which any undergraduate or graduate program where you're learning research skills, or even in younger ages, people will point out that there's potential for bias, that there's potential for reaffirming things that you already believe, having unquestioned assumptions. And in some ways, I think AI actually gives us an opportunity to come back to the rigor and methods even more strongly maybe than we have before, where if you're the one doing the analysis and you're sharing it with peers, you may think, ah, as it's going, I'll sort of catch myself and keep track of things. I'm like, this is obviously not an ideal practice, but I've certainly seen it happen. But if you're going to be handing off large stretches of work that are unmonitored, it's actually even more important to upfront, be self aware and self critical and think like, where am I coming from? What do I anticipate the data are going to tell me? That belief inform the prompt that I write, the way I present the context, the way I draw conclusions from what's presented and sort of re prompt the model to continue. So I would say even if you don't yet have the discernment or think you may not have the discernment to interrogate a model's responses and its reasoning. Be really clear up front with all of the baggage ideas that you're bringing to the research. Obviously we're not perfectly self aware. This is an area where collaboration with another researcher can help, where you can say that's sort of a super, do you think what's top of mind for you? And they'll maybe write these things down even before you share them to avoid influencing each other and then recognize that the AI is not going to come back in an unbiased fashion. And part of actually what I advocate for and would suggest other researchers do is don't expect that dumping in all of your data and asking for an analysis is going to give a perfect answer. I say this for two reasons. One is it might give a good answer, but maybe not. And it's hard to tell all the time. But the other is that the process is not easy to interpret or interrogate. And the reason that that's a problem is research should be reproducible and it should be transparent. If someone on my team comes and says, why do you think this? I can't just say, well, the AI said it was true. Well, this was the method I used, this was the method of analysis, this was the sampling, et cetera, all of these things. And I can show them the whole trace of reasoning. And if you use AI not just to do the analysis end to end, but to build the tools that facilitate your own analysis, a lot of that transparency can be retained and you can still save a ton of time and in aggregate probably even more time than you would just dumping things into the model because you have tooling that is well suited to what your actual research practice is, rather than hoping that each time you run the model it it's just going to magically pull all the right pieces together and give you an output that is correct. probably even more time than you would just dumping things into the model because you have tooling that is well suited to what your actual research practice is, rather than hoping that each time you run the model it it's just going to magically pull all the right pieces together and give you an output that is correct.

Stephanie Vance 11:18 I love that point about the reproducibility. It's a good one. Yeah.

Speaker B 11:22 Well, let's talk about a concrete example. We know that predicting trends or not, sometimes it just straight up makes stuff up. It can just hallucinate in a wildly convincing way. So I'd love to walk through a moment with you where there was an obvious, but maybe perhaps buried AI generated insight, or even maybe not a synthetic created research insight, but even just an analysis of a set of data that truly wasn't the real story.

Sasha Mitts 11:48 I think this is actually instructive that even as we Move along this path, there's room for the tools that we're building and the tools that we're using to not be perfect. And maybe a good example of this is maybe a year and a half ago. So this is earlier model in AI time. This is like 3,000 years ago. Way dumber. But even when I was trying to take an early version of my own advice and build more replicable processes and build things that are small, discrete units that can be stacked together, one of the things that I often want to do is have an AI work with me to create a first, a code book to understand what patterns exist in a set of qualitative data, and then a separate set of models to apply those codes. In each of those steps, you can imagine how a person might either do that on their own or oversee an AI doing that work where you maybe read lots and lots of examples of responses, you write up some ideas of how the coding might go. You might do it the other way around, where you propose codes ahead of time and then see whether or not your data maps to it.

Stephanie Vance 12:44 Right. on their own or oversee an AI doing that work where you maybe read lots and lots of examples of responses, you write up some ideas of how the coding might go. You might do it the other way around, where you propose codes ahead of time and then see whether or not your data maps to it. Right.

Sasha Mitts 12:44 There's a many methods around this. And there was an instance where I had a code book that I felt reasonably good about, that I had worked with a model to produce. And then the next step is go through a bunch of data and apply the codes. And one benefit of using AI, if you know you're going to have access to it, is you can collect more and more complex qualitative data than you might otherwise, whether it's a survey or an AI moderated interview, or in a diary study or all these other things. And there's a temptation to lean on that. But as you lean on it more, especially if early on, you're not necessarily sure it's going to do a good job with the coding. There's potentially a risk there. And so there was a model I was using that was not doing a great job applying the codes that we'd identified. But that was not obvious at first, right? Because, like, the codes were applied, it wasn't making up new codes, but it was saying, this code applies to this question, this code applies to this question. And there's a step, of course, we have to take of. Well, we. Let's see if this is right. But that's time intensive and there's pressure to write out insights. So I'm starting to write what the conclusions I might draw from the distribution of codes that have been assigned and the inferences I'm making from those. I am not 100% sure this is Consistent with the samples I read earlier on. I'm not a hundred percent sure this is consistent with the quantitative data coming out of the study. Is it some interesting thing where the quantitative and qualitative are intention or maybe there's just something going off and it turned out that that model was just not strong enough. It was not up to the task, it was applying codes erroneously and that took obviously some human effort to correct. One concrete learning and I would say like a takeaway I'd recommend in for anyone taking on a similar process is use models from multiple providers and measure the inter rater reliability between them. There's a number of statistics you can compute to understand where divergence exists. Certainly many models now are sufficiently strong that I wouldn't expect them to do a bad job, but they will still disagree with each other even when I use now medium sized frontier current generation medium sized models to do these tasks. The integrator scoring only really ever gets to the mid 80% which is still higher than you might expect from humans over a really large sample. And certainly enough that if you interrogate the disagreements you can adjudicate them with a stronger model or by yourself. There's like lots of ways to handle this. This was certainly an instructive example. Are things as good as they seem early on in building the tools? Not so much. And certainly with earlier models, not so much. enough that if you interrogate the disagreements you can adjudicate them with a stronger model or by yourself. There's like lots of ways to handle this. This was certainly an instructive example. Are things as good as they seem early on in building the tools? Not so much. And certainly with earlier models, not so much.

Stephanie Vance 15:02 That answer harkened back to me. Cronbox alpha is that. Do you. I just haven't thought about that statistical test in a long time. But it's for VR rate of reliability.

Sasha Mitts 15:13 It's a great tip to my earlier part of methods. As you're doing this, you are forced to return to the ways you decide between methods very carefully because you're encoding them in a system, you want to build it and then have it be reliable and repeatable. And if you're doing it one off by yourself, you're like okay, I'm just going to like change the dropdown or change what I'm doing in Excel or in Jump or whatever else software using. But when you're deciding I'm going to run this kind of study 10 times in the next six months and I want to ensure it's really tightly tailored. You're going back and you're looking at the equations and you're comparing the trade offs, maybe you're talking to an AI about why one or another might be well suited the data and when you might want to flip it and it's a little bit time consuming, but it's also Fun. It's something that maybe AI is weirdly inviting us to be more in the weeds than even before.

Stephanie Vance 15:58 That's such an interesting way to think about it. Well, let's get a little bit practical too, with some tips from you. So what tools or practices can researchers use to keep an AI generated explanation from becoming that default hypothesis too early? to be more in the weeds than even before. That's such an interesting way to think about it. Well, let's get a little bit practical too, with some tips from you. So what tools or practices can researchers use to keep an AI generated explanation from becoming that default hypothesis too early?

Sasha Mitts 16:15 I mentioned a couple of these things. I want to touch on them more. So one is certainly being upfront about what you think is true or is not true going in. Another is if you decompose the steps enough, you're not actually ever asking the AI to reason across the data. And the more high level and the more end to end your expectations of it are, the more there is a risk of genericism. Because at that point maybe it's consuming depends on your process, but maybe it's consuming tens of thousands of responses, you know, in a large survey with multiple open ended questions or whatever. And even though the model can fit all that data into its context, asking it to reason carefully about it with no oversight invites generic responses. In the case of some models, maybe higher proclivity for hallucinated responses. It's not fully avoidable. Right. There are going to be cases where the data are rigorously analyzed and the narrative becomes the area where it feels a bit generic or weak. And that is really on the researcher to say, like, okay, well the AI has assisted me with maybe cleaning the data, maybe coding the data, whatever it is, applying statistical tests that I've selected now up to me to go through and think what is the story here? And I think an open question of where does research produce the most of its value? And I think a lot of it is from opinionated storytelling, enough stories in the invented sense. But being able to communicate a narrative that is informed and persuasive and that has a point of view through a lens. Exactly, exactly the lens. And if you're asking AI to have the point of view for you, I mean that is where the, I think the greatest risk of genericism is. You're saying, well, here's the data, I'm going to dump all of it in. It's going to tell us what to think. I mean, at that point you're really bringing pretty marginal value to the equation. And I can't tell everyone what point of view to have, but I would encourage that maintaining that perspective and cultivating the organizational awareness and product awareness to form that perspective is maybe the most important thing.

Stephanie Vance 18:13 Great advice. ng pretty marginal value to the equation. And I can't tell everyone what point of view to have, but I would encourage that maintaining that perspective and cultivating the organizational awareness and product awareness to form that perspective is maybe the most important thing. Great advice.

Speaker B 18:14 And that gets to something that I think is the core of when I'm thinking about specifically the research impact or the business impact. Research at a fundamental level is helping to discover things that we don't know yet or to have an understanding of human behavior that maybe hasn't been captured in any specific way so far. I think about who would have thought that we would buy out toilet paper and cans of beans in the middle of a pandemic? Like, there's all these different types of things that is really hard to understand. So. So how should researchers be using these models in a way that helps to achieve that goal when they're designed to predict something that is perhaps the most likely outcome, when that's not actually a mimic of human behavior? demic? Like, there's all these different types of things that is really hard to understand. So. So how should researchers be using these models in a way that helps to achieve that goal when they're designed to predict something that is perhaps the most likely outcome, when that's not actually a mimic of human behavior?

Sasha Mitts 19:00 I think this is an interesting question. There's maybe a few different threads to it. So it's true that there is some probabilistic sequencing of words, but a lot of what steers that is the context you give and the kind of question you ask and the kinds of counterexamples you also maybe append to your prompt. If you ask a model what's going to happen in the next pandemic, you're not going to get anything useful. I think if you have an opinion and you have data and you have preexisting hypotheses that you want to test, or you have questions that you think are unanswered and you want to understand maybe what data to try to collect to better address those questions, I think those are some of the areas that AI can have value. I've been harping on this point a little bit, but AI is a tool that needs to be slotted into the kinds of things that it is good at doing. For example, like recurring example, AI is not good at counting. It's not good at telling you how many times the letter R shows up in strawberry. And it's not because the models are bad or good. It's because the kinds of things they're good at are not similar to the kinds of things human brains are good at. So if you try to lean on them and say, like, do my human labor end to end, I'm going to not pay any attention to where your strengths are and where your weaknesses are, that's kind of like a recipe for failure. And it's the same with a person. If I go to someone who's like a famous painter and I say, like, oh, it would be really wonderful if you build me a car, I mean, maybe I get lucky and they happen to be a good mechanic, but it's probably not a good bet. And this is even more extreme with AI because we see them do some things maybe better than 99.9% of humans could. There's lots of AI now writing mathematical proofs and understanding the sciences better than almost anyone alive will. But that doesn't mean that they can, you know, walk your dog for you. Maybe not. Robots are coming, but not quite yet. And the models that will underlie them are a bit different. So I think that's one thing, right, is if you're trying to do something that they're not good at, end to end, assign them to the parts they're good at, the parts that they're not good at, you can do on your own. And there are. There's gray area where they could do it, but they need scaffolding to do it. And that's where we've actually seen a lot of progress. Cloud code and codex. And a lot of these tools are valuable because they give the appendages, whatever, you know, like mixed analogy you want to use. They give the tools and the facility to models to extend their capabilities beyond what they can just do by predicting the next token. And more and more, I think it's going to be the obvious step to build your own versions of those things. Not necessarily replacements, but additional ways to augment the capabilities so that when you have a request, it's not just can you predict the next word, but can you use the tool I designed for you to go do this thing that on your own would be hard if you were just trying to spit out language. rsions of those things. Not necessarily replacements, but additional ways to augment the capabilities so that when you have a request, it's not just can you predict the next word, but can you use the tool I designed for you to go do this thing that on your own would be hard if you were just trying to spit out language.

Speaker B 21:42 That's a fascinating way of thinking about the use case for it and that we in the research space are responsible for thinking of it as a tool that has a use case. And it's a change in the workflow, it's not a change in outsourcing the mental work that goes into, to any type of research work.

Sasha Mitts 22:00 I think that's exactly right. As long as there are new things to understand about the world, which there always will be. It requires this, the lens we were talking about. It requires curiosity. It requires the desire to go from not knowing something to thinking that thing is worth knowing and then putting in the work to get there. And amazing if AI helps us accelerate, answer more questions, answer questions differently, more deeply. In some cases, there's a ton of value there. But the fundamental curiosity of having an opinion about what is worth knowing is something that needs to be a skill.

Speaker B 22:35 The skill of what's worth knowing. That's the name of this episode. If it's not the name of this episode. It shouldn't be like that.

Stephanie Vance 22:41 Well, Sasha, we have a segment that we run on every episode that is called the current 101. We ask all of our guests the same question. In your experience, what is one trend or practice in UX research that you would like to see stop? And what is one thing that you would like to see more of? Well, Sasha, we have a segment that we run on every episode that is called the current 101. We ask all of our guests the same question. In your experience, what is one trend or practice in UX research that you would like to see stop? And what is one thing that you would like to see more of?

Sasha Mitts 22:58 I think my first response is just an appeal to everyone. Even outside of UX research, the amount of LLM written content online is exhausting. Individual case, it makes sense. There's something I want to write. I want to make sure it sounds professional. I want to make sure that search engines index it, whatever, you know, whatever your reason is. But the aggregate effect is, is this very tragic, depressing state of content online. And it's not just recreational social media. I see it on professional blogs, I see it everywhere. And I think if you want to write something concise, simple, whatever, just write a short thing. Most of these LLM posts are not that thought provoking. They really only make one or two points. Write down your two points in two sentences. Do whatever you have. You put it on Twitter, put it on LinkedIn, be done with it. Or if you want to write something interesting, write a long thing. If you need help writing a long thing, that's fine, like collaborate, but do the thinking or work with them on the thinking and write it yourself or something. And I'm not just saying this like I'm bitter and it's boring, whatever. But to the earlier point of this genericized routine content, this is an area where it crops up in its most obvious form because someone may say, like, I watched this keynote that some CEO gave. I want to like say a intriguing thing to my audience about it. Write me a post. What was the motivation? That you actually heard something interesting and you want to share it? Okay, if so, do that. If not, you don't need to comment on it. You can just be silent. If you have nothing that you actually. If you. If no thoughts have come to mind,

Stephanie Vance 24:24 I'm going to get this wrong. But it's the typical. It's not this, it's that it's interesting

Sasha Mitts 24:29 too, because these syntax and rhetoric of AI is partially nauseating because it's so stereotyped to each model. But at first it actually sounds really good. I mean, this is how it gets done, you know, whenever Claude 6 GPT 6. You know, like in whatever, 6, 12, 18 months. The first little bit of how those models talk might actually sound really novel. It might sound really compelling and well written and articulate. And interesting, but then everyone catches onto that and everyone's using it to write. And so all of a sudden it was always the case that the meaning was diluted, but now it's obviously the case the meaning is diluted.

Stephanie Vance 25:06 Yeah.

Speaker B 25:06 Yeah. And what's the second part of that too? The see more of this is somewhat e catches onto that and everyone's using it to write. And so all of a sudden it was always the case that the meaning was diluted, but now it's obviously the case the meaning is diluted.

Stephanie Vance 25:06 Yeah.

Speaker B 25:06 Yeah. And what's the second part of that too? The see more of this is somewhat

Sasha Mitts 25:11 selfish, but I would love to see more discussion of the tooling and approaches people are taking to use AI to do good research. I've seen a flavor of this, which is people saying, here are my prompts or here is the agent that I like configured in whatever interface. And I think that's fine. Leaving aside that a lot of people want to put these behind the paywall, I think that's, I'm somewhat neutral on that. I think there's, there's room for both more open source and more productized versions of this. But I don't see as much discussion of ways to build end to end or even in some substantive subpart the tools that people think are well suited to do research. And I think that's a little unfortunate just because there's relatively well understood, well defined methods that we all make use of every day. And it shouldn't be the case that every person needs to struggle in the dark through how do I get this to turn into a tool that is useful and repeatable, et cetera. It's not that it's so hard, right? If you describe what you want and ask Claude to build you a little tool, it will usually do a good job. But that's not how science has worked historically, at least in, you know, recent history of open sciences and academic papers, where you communicate the thing that you built, how it works, how someone else can use it. This happens somewhat more in academia, but a bit less in industry for, I guess, somewhat obvious competitive reasons. But I'd like to think that the tools that we use to do analysis are not the precious part of how researchers in industry do their work. It's that they are applying to highly relevant, valuable questions. So I'd love to see people having more open conversation and I may just be missing those conversations, so please let me know where they are.

Speaker B 26:50 So, last question that we have for you boils down to what you do in your work, which is for researchers navigating a very clearly AI driven future. It's becoming an expectation that people are versed in this and it's the nuance of how they do it. It's not just do you have it or not? What's one soft skill or perhaps A uniquely human skill that professionals should continuously work on mastering. , which is for researchers navigating a very clearly AI driven future. It's becoming an expectation that people are versed in this and it's the nuance of how they do it. It's not just do you have it or not? What's one soft skill or perhaps A uniquely human skill that professionals should continuously work on mastering.

Sasha Mitts 27:18 The first thing that comes to mind, and I really do think this is the most important point, is to build and maintain real relationships with the people who you are making decisions alongside. I mentioned earlier, right, how much the perspective and context and conviction and curiosity, all of these things are what determines the actual value of any tool. Whether it's something as simple as your email or something as powerful as AI, maybe your own opinions about where to steer it. But getting leverage and having a more informed perspective and seeing your work go as far as it might go, and making sure that you're actually doing the work worth doing relies completely on understanding the social and intellectual and economic ecosystem that you're living in. And that doesn't just happen by talking to a language model all day. It doesn't just happen by reading the docs that were posted on your internal workplace. It comes from having the conversations with people, understanding the unspoken priorities or the rapidly changing priorities or what was previously tried. It's something that more and more of this context might get recorded, it might get uploaded, but reasoning over it requires situational awareness and interpersonal awareness and emotional intelligence, which are not things that we've really seen AI be good at yet.

Speaker B 28:35 Yeah, the things not something that are likely to be outsourced effectively.

Sasha Mitts 28:40 Yeah, at the end of the day, even if AI can get good at, say, reading a room visually, maybe we'll have more camera inputs. That doesn't necessarily mean that setting perspective should be abdicated. We do all of these things. All of the businesses that exist have a point of view. Maybe you disagree with it, maybe you like it, maybe you don't. But each business has a thing it is trying to make and sell, and those are human opinions about why they even want to be in that business in the first place. Why a product should be tweaked one way or another. There's not a right answer. We're all just trying to figure out what is available to us and how can we whether it's make money off of it or make someone's life better, you know, whatever your motivating factor is. But even if you can say to an AI, help me do this thing or do this thing for me, it does not mean that every one of those decisions needs to be outsourced.

Speaker B 29:25 Sasha, this was a fascinating conversation. I feel like we have a lot of these different AI related conversations on this podcast, but this was a really refreshing conversation and interesting perspective.

Sasha Mitts 29:37 Thanks for having me. It's really fun to be asked the questions and to think through it with you guys. was a fascinating conversation. I feel like we have a lot of these different AI related conversations on this podcast, but this was a really refreshing conversation and interesting perspective. Thanks for having me. It's really fun to be asked the questions and to think through it with you guys.

Speaker B 29:42 It's been really wonderful to have you on the show today. One thing I'm really going to take away from this conversation is the idea that good research isn't just about finding the answers or finding the fastest road to the answers. It's knowing when a little bit of uncertainty can be valuable.

Stephanie Vance 29:59 Absolutely. I also loved that reminder that while AI excels at recognizing patterns, research often depends on questioning those patterns and remaining open to what hasn't yet been considered right.

Speaker B 30:12 And your perspective on stewardship and how to use these tools really stood out to me too. As AI becomes more capable, the challenge really isn't deciding on what what these systems can do, but what they should do in service of better research and better outcomes. I think your example of you wouldn't ask an artist to fix your car is really interesting because you're right, it's very tempting to outsource everything to an AI tool and instead of an artist that may just tell you, I can't do that. The AI tool will never tell you that it can't do that. It will try its best. So it's up to us to really discern what it can and can't be used for.

Stephanie Vance 30:48 Sasha, thank you so much for sharing such a thoughtful perspective on AI uncertainty and what it means to practice responsible research in a rapidly changing world.

Speaker B 30:58 And to everyone listening, thank you so much for being part of the Curiosity Current. We'll see you next time.

Stephanie Vance 31:05 The Curiosity Current is brought to you by aytm. To find out how AYTM helps brands connect with consumers and bring insights to life, visit aytm. Com and to make sure you never miss an episode, subscribe to the Curiosity Current on Apple, Spotify, YouTube, or wherever you get your podcasts. Thanks for joining us and we'll see you next time.