Stephanie 0:07 Alright. Welcome back to the Curiosity Current. Today, I'm joined on the hosting side by Alana Marmerstein, growth operations analyst on a y t m's marketing team and one of the hosts of the conference series, ways of thinking. Alana, thanks for joining.

Elana 0:24 Thanks for having me, Steph. I'm really excited to be here.

Stephanie 0:27 Me too. And then today's guest is Adam Hagerman. Adam is a researcher and strategist who has spent his career shaping product and user experiences at companies like Indeed and Apple. Adam has led large research teams, built functions from the ground up, and leveraged systems to connect insights to business decisions. His work blends qualitative depth, quantitative rigor, and a systems thinking approach to translate complex challenges into human centered strategy. Before his time at Indeed, Adam worked in investment research at GLG and has a background in history, music, and public affairs, a true renaissance man. His unique perspective allows him to uncover meaningful truths about people and systems and activate those insights to create meaningful impact. In our episode today, we are going to be exploring the evolving role of research and product development, what the next generation of insights professionals needs to succeed, and how organizations can better connect strategy to real user needs. Adam, welcome to the Curiosity Current.

Adam 1:37 Thank you for having me, Stephanie and Alana. I'm I'm excited to talk about these things.

Stephanie 1:42 So were we. Um, so just to jump right in. Adam, you have built this career that's kind of at the intersection of research, product development, behavioral science, strategy. I I would love it if you could just kinda take us back through your career journey and talk about how your education and your experience have shaped your approach to research. mp right in. Adam, you have built this career that's kind of at the intersection of research, product development, behavioral science, strategy. I I would love it if you could just kinda take us back through your career journey and talk about how your education and your experience have shaped your approach to research.

Adam 2:04 Sure. Uh, why not start at the beginning? Uh, my my first job was actually in in retail. I I was that person that was checking you out at at the store going beep beep beep beep. And if anybody who's worked in those roles understand that there's often times limited flexibility to change the way things work. Uh, you kind of do what you have to do. There's a process. There's a procedure so on and so forth. Um, but I realized early on that I'm one of those people who likes to say I I don't think that's very efficient. I I I think there's a better way to do that. Um, and I found that ethos has followed me through my career um and my education. Uh, I I studied music. Uh, I I sometimes joke that I'm a classically trained musician and and that's true but I'm not very good. Uh, I was more interested in music as a cultural artifact than like uh, performing it. And what the the reason that that distinction is important for this conversation is, uh, at the end of the day, the way music is done is an intensely social thing. Uh, it's also like a literature practice. Uh, it's written down. It's done over time. There there there's in fashion out of fashion of style so on and so forth. Um, and I was really captivated with, like, how did that system work? Um, and because I was interested in music as a cultural artifact, I was also drawn to music as in its place in history, uh, as a thing people did. Um, and that led me into more canonical studies of history, uh, wars, empires, so on and so forth. But I was interested in kind of like the anthropological scope of of how music played into that. Uh, there aren't many jobs for that. Uh, when when when I graduated from school, I I was like, well, who who's gonna pay me money to sit and talk about the the the role of, like, opera in the Hobsburg empire? Probably nobody. Um, but what I realized through through my, um, my own curiosities was that what I can do is I can help figure out how the thing works so I can explain it to other people so that the people who are more motivated and able and technically savvy as became the case are able to use those insights to modify the way their product operates in that environment. Um, so I I I mentioned I I had to find work. Um, my first job was at Apple. Um, I the way I stumbled into it was a a chaotic storm of of events that had to do with me speaking German at the time. Um, but what what I got to do was, uh, I I got to work with, uh, software developers as they were trying to get their intellectual property onto the App Store. Um, and while Apple is known for their incredible user experience, that doesn't just happen. It doesn't just manifest from the C phone like Venus. Um, there are people who study that thing. I didn't get to be one of those people but I got to work alongside them. Um so the the role I had was help helping these software developers navigate the Apple bureaucracy. Um how to get their app on the store, how to How to deal with the business once it's running once people are downloading once they start having consumer complaints. All these things. Um and that was also a system that could be improved and my expectation that this can be improved led me to work with those people who could make the thing happen. And I realized that that is a career opportunity. That is something you can do. Um, for various reasons, I've I've moved on to to t l two, uh, which is basically, uh, at at the end of the day, it it it's an expert network, but the group that I worked under was, uh, a kind of a special projects team. Um, they have their survey research team. They have their their more, uh, comprehensive portfolio of consultant like services under under that team as well. So I found myself in another situation where I was doing research to help people make things work better. Um, so in in investment research is where I I really got to dig into my interest in economics. And, uh, economics at the end of the day is, uh, it's a study of how people make decisions, uh, how they choose to allocate whatever resources they have among a group to achieve certain goals. Um, and investment research really helped me lean into that way of systems thinking, uh, using numbers, uh, to to tell the story. Um, and then I moved to Indeed after I I was at GLT for about five years. Did a did a lot of interesting stuff for a lot of interesting clients. Um, but in Indeed came along. Uh, I worked in their marketing research team. Um, I was part of the group that was doing like advertising research. Uh, who is our market? What marketing research. Market and marketing research. Um, and I just got to deploy those skills of like the systems thinking there are multiple things happening at once all in tension with each other and here's how the thing works so that you can create a good message product whatever. Um, and eventually, I made my way over into the product team, uh, because I had worked cross functionally and collaboratively with a lot of different types of teams at Indeed. Um, the the move just kind of made sense. with each other and here's how the thing works so that you can create a good message product whatever. Um, and eventually, I made my way over into the product team, uh, because I had worked cross functionally and collaboratively with a lot of different types of teams at Indeed. Um, the the move just kind of made sense.

Elana 7:55 I I loved hearing that that journey of yours and how all of your interests really ended up leading you to where you are today.

Adam 8:03 Yeah. Um, it it's sometimes it's wild to think how many things I've I've touched, how many different industries, how many, uh, different projects, how many different products within those industries and projects. Um, and and all of that came from somebody who was just kind of frustrated with how the the retail system worked.

Stephanie 8:24 Well, and it sounds like like certainly a theme across all of that is that the system's thinking. But another one that I feel like hearkens back to your education is the role of context in creating meaning. And I wonder, do you still use that framework even within within research?

Adam 8:41 Yeah. Uh, earlier on in your introduction, you you mentioned my my blend of qualitative and quantitative research and, uh, at the end of the day I put that in my LinkedIn profile because those are the words that people need to hear and need to see. But at the end of the day they're just different strategies to understand context. Uh, statistics is pejoratively like line with numbers but what what it is is it it is it's an argument. You're you're you're you're making an argument that something is or isn't you using a a different rhetorical strategy. Uh, much like Tchaikovsky would use, like, the orchestra as a as a rhetorical strategy to convey something, and there would be all this context around those decisions and and how that was done. Uh, the the same is true for a product design. Uh, there there are tools that we can use to interact help people interact with their environment. Um, and we can measure it in numbers. We can talk about it with with, uh, nouns, verbs, adjectives, and and the blending of all of those things is, uh, a triangulation at the end of the day. Uh, it it's it's a picture of what the thing is, how the thing works, and what we can do to, uh, change the outcomes.

Elana 10:05 So at Indeed, you led research programs that touched really every part of the labor market ecosystem. In today's product innovation landscape, how would you say the role of research has evolved from being a feedback mechanism to a strategic driver that really shapes the what and the why of products? So at Indeed, you led research programs that touched really every part of the labor market ecosystem. In today's product innovation landscape, how would you say the role of research has evolved from being a feedback mechanism to a strategic driver that really shapes the what and the why of products?

Adam 10:29 Good question. Um, it's one thing to state what has happened, and it's another thing to try to change the outcomes on the other end. Research helps us figure out how the thing works, uh, and how we can modify how the thing works towards a desired end. Um, and and that's why science exists. Right? Um, it's to tame nature. The the transition for me, moving from validation of, like, yes, this was the right thing to do. Um, I I did something that sometimes gets researchers into trouble. Um, I showed them that we can have a theoretical framework for how the thing works. Um, and that allows iterative thought that like infinite thought experiments. Um, and what what it allowed people to do was, uh, think think more in realm of possibility rather than did it did I get it correct? Um, and changing that that mental framework, uh, the just the framing even, um, I I demonstrated that research could help set the future instead of just recording what had happened because we were able to use all of those those facts and figures that that we that we dredged up in in the primary data collection. And we were able to frame it into a story, a narrative that my stakeholders could engage with much like they do their movie their favorite movie characters, their, uh characters in books, uh plot lines in the news. Um, I was able to give them the same, uh, the the same bits of information and spin them on a way to to work through that narrative so that research was informing what they eventually build rather than validating that what they built was a good idea.

Elana 12:24 Yeah that proactive ness to research rather than the reactiveness.

Adam 12:31 Um, so something I I try to tell my team and I I emphasize when when I'm coaching them is, uh, we we can we get the opportunity to control the narrative. Um, when we're working with with product managers, engineers, etcetera. Uh, go to market people, uh, senior leadership, um, just generally interested parties around the company. Everybody's showing up with an opinion and a bit of information that that they want to uh influence this way of the product. And their motivations are are many, uh, oftentimes well intentioned. Um, what researchers have the opportunity to do is bring an objective lens. But they're not supplanting everything else. They're not saying all of your information is false. Rather what they're doing is they're using that external information to help meld together all the different perspectives and shape that narrative of of what is going to happen. Um, so that that that's that's the coaching I give my team, when I I see that we're stuck in kind of a tactical space, and we need to move it into a more strategic space. It becomes about controlling the narrative rather than giving information. erent perspectives and shape that narrative of of what is going to happen. Um, so that that that's that's the coaching I give my team, when I I see that we're stuck in kind of a tactical space, and we need to move it into a more strategic space. It becomes about controlling the narrative rather than giving information.

Stephanie 13:48 Very interesting. Um, I'm gonna add kinda lean into some of your UX research in this next question. I think some of your answer will be the same, but I'll be curious to to hear if this, uh, uncovers anything else for you. So first, I want to acknowledge that, you know, you really got to shape what UX research looks like at Indeed, which is is rare. Right? That's a rare opportunity to be able to just build a function from the bottom up. And, um, I'm sure it was both challenging and and rewarding if you want to talk a little bit about that. I do wonder though, for organizations, other organizations who are looking to root strategy in those user needs more effectively, what frameworks or processes have you seen work best for turning complex research into actionable business decisions? And I can see that, like, part of your answer is certainly using these storytelling techniques and making sure that, like, from a contextual perspective, that insights has a role that's not to supplant. But are there other things that you sort of practice, um, in this realm?

Adam 14:50 I remember that a decision needs to be made. Um, we we don't just do research because we find it interesting. Like, some sometimes I do. We all go down Internet rabbit holes. We're like, the practical utility of this information we're gathering is low, but we're we find it interesting. In in the business context, there's a decision that needs to be made. Uh, it's oftentimes an investment decision. Uh, there the the team is deciding, do we put people to work on this? Do we reallocate from this other thing? That this is a decision that needs to be made and the research should inform the outcome of that decision. Um, and yes, it's about storytelling so on and so forth, but the the decision is the golden thread that kind of brings all these things together. Yeah. Um, as as we're trying to collate all of the different perspectives and things that could happen with with our new information and our ability to synthesize. Reiterating that there's a decision. What is that decision? Um, how is this research going to inform that decision? Um, it also taking that approach I find makes the right decision inevitable. If that makes sense.

Stephanie 16:03 It does. I like that. with with our new information and our ability to synthesize. Reiterating that there's a decision. What is that decision? Um, how is this research going to inform that decision? Um, it also taking that approach I find makes the right decision inevitable. If that makes sense. It does. I like that.

Adam 16:06 The the way I do that is a mix of logos and a mix of ethos and a bit bit of pathos. But at the end end of the day, um, the way you do it is going to be very, uh, it has to be responsive to how the organization is working. Who has power? Who get who gets to make decisions? How do they make decisions? Um, you you were very kind in saying that I I helped to to to shape the UX culture at Indeed. Um, but I did it in concert with all of these other people. And have that cast of characters been different, the culture of UX research at Indeed would have been different. Um, be because all along the way, there were decisions to be made. And there were there were ways we could choose to approach the research. There were ways we could approach to we there were ways we could choose to approach the stakeholders. Um, but it's all dependent upon that that that cosmic alignment of the cast of characters. Um, so you you may have felt that my answer was not prescriptive and it's because there isn't one framework that will work all the time. Um, but what has helped me is grounding it in that there's a decision that needs to be made and the decision needs to be in the interest of users because that's who we're paid to represent. Um, but also we need to make the thing move forward of we gotta do something.

Stephanie 17:29 Yeah. For sure. And it's it's interesting too because, you know, before when I was asking you about the role of context, I'm thinking about, like, contextualizing insights. But you were really also raising this point that, like, you as a UX researcher, as an insights professional, you sit in a unique context as well. And your success is only going to be as great as your ability to understand that context and know your role within it.

Adam 17:56 And and note, uh, I'll expand upon that. It's not just doing your role. It's also understanding what things you can influence.

Stephanie 18:04 Yeah.

Adam 18:04 What thing what things you can control. Um, and I I I find researchers have a hot take. Um, I I find researchers oftentimes have a diluted sense of of of importance in the decision making process. Um, and that's not to say research isn't important. It's that, uh, we think sometimes that we can show up and because we have the facts and figures, we we get we we've earned the right. But the reality is is it's a social thing and and we have to work with the others and the context is more it's much more than just the user context. It's the business context.

Stephanie 18:43 The business context. Yeah.

Adam 18:44 Because all those things blend together to make the market work the way it does. Um, and I I I don't think we're doing ourselves any favors by ignoring that. and the context is more it's much more than just the user context. It's the business context.

Stephanie 18:43 The business context. Yeah.

Adam 18:44 Because all those things blend together to make the market work the way it does. Um, and I I I don't think we're doing ourselves any favors by ignoring that.

Stephanie 18:56 That makes so much sense. If if I can have a a small follow-up here, I'm curious, uh, if, you know, is there any particular advice, like, given all of this sort of complexity that you would give a a UX research team that is really seeking to build more influence and impact with cross functional partners?

Adam 19:16 When research is involved, good things happen. Um, we're we're collaborative. We're we're not just naysayers. Um, and it's important that we we think about, uh, the simplicity in our delivery. Um, maybe don't show all the stuff along the way, uh, but they do need to be invested from the start. Um, and and, uh, bringing it back to when when research is involved good things happen. There there also has to be an understanding that because research was involved better things happened. Mhmm. Um, and that that goes to under understanding your context. How can I be helpful in this space? What things can I control? What can I contribute to the conversation that shapes it into something that advocates for users? And it it is not dead on arrival. Um, uh, so that that that would be my advice. Or or I don't know that that's really even advice. Um, It's more of a it's more of a desirable outcome. Because the way you achieve that is going to change. Um, the desirable outcome is when research is involved, good things happen.

Stephanie 20:31 And you So rooting rooting your behavior in that understanding or that sort of a mantra. Yeah. I like that.

Adam 20:38 And then as you go about making your decisions about, uh, what to say, when to say it, to whom to say it, um, remember that the the the end goal is when research is involved, good things happen.

Stephanie 20:54 It's confidence boosting too. Never bad.

Elana 20:57 Yeah. I, um, remember listening in another interview, Adam, you actually mentioned how research oftentimes illuminates the consequences of decisions ultimately. And so in that same vein, when you think about organizations that might struggle to connect the strategy to what users actually need. You know, when you push the lever, does something happen? Or if it doesn't happen, why or why not? So from your experience, what would you say are the most common gaps that you've observed, um, in that connecting strategy to what users need? And how can research help bridge those gaps, especially in early stage product ideation and then later on in mature product optimization? r if it doesn't happen, why or why not? So from your experience, what would you say are the most common gaps that you've observed, um, in that connecting strategy to what users need? And how can research help bridge those gaps, especially in early stage product ideation and then later on in mature product optimization?

Adam 21:53 Um, I I find that there's sometimes confusion between usability and usefulness of a product. Um, many product teams are incentivized by things like engagement metrics. Um, uh, but I I think those can be deceiving. Because, uh, those measure usability were able were people able to do the thing. Uh-huh. Uh, rather than usefulness. Um, what was the thing worth doing and did they have a good time doing it?

Elana 22:22 Do they keep coming back to use it again and again?

Adam 22:26 Research helps, uh, explain the the why behind certain observable phenomena, the thing we see, uh, and and and I I I think that the difference between usefulness and use usable, uh, uh, is the gap between a want and a need. Um, I think a a want is demonstrated by somebody clicking the thing. They they were lured into the behavior. Uh, we we created an environment where the behavior we wanted them to do manifested. Great. Wonderful. That's usable. Um, but that's not a strategic decision. That's not solving problems for people. That that's not earning more revenue from the market. Um, earning more revenue from the market is by being useful. Uh, another way I maybe to think about this is like a want explains an action, but a need explains why they want it in the first place. Yeah. Um, I find the jobs to be done framework really helpful in in kind of posturing this difference, uh, because it helps us monitor outcomes rather than activity. Um, and I I think that shift, uh, think thinking about oh, let let's see if I can make this this parallelism work. Usable products demonstrate a want because people did the thing, and that's measured by activity. Number of clicks, sign ups, so on and so forth. Things that things that happened. Um, but the outcome, it a a usefulness, uh, leads to an outcome that is more desirable. And that that's, uh, well that parallel that parallelism fell apart. Uh, maybe maybe we don't put that. Uh, but the difference between usefulness and usability is uh, kind of that gap and and jobs to be done focusing on the outcome we want to affect. It changes the style of thinking. Uh, so product satisfying needs first, that's the strategy. We're satisfying a need, um, and then the wants come later through through the engagement metrics.

Elana 24:50 Right. And research is what is illuminating and revealing and understanding that

Adam 24:55 It explains the to folks. Yep. It it it explains the the the the things we

Elana 25:01 see. Wonderful.

Adam 25:02 What's meaning behind s come later through through the engagement metrics.

Elana 24:50 Right. And research is what is illuminating and revealing and understanding that

Adam 24:55 It explains the to folks. Yep. It it it explains the the the the things we

Elana 25:01 see. Wonderful.

Adam 25:02 What's meaning behind

Elana 25:03 it? So I'm gonna switch gears a little bit, Adam, to everyone's favorite topic, AI. Hope you're not sick about talking about it. Um, but with thinking through the rise of AI and automation, um, they're transforming really how teams gather and interpret user insights. So how can research leaders integrate these tools while ensuring that human empathy and contextual understanding remain central to product strategy? not sick about talking about it. Um, but with thinking through the rise of AI and automation, um, they're transforming really how teams gather and interpret user insights. So how can research leaders integrate these tools while ensuring that human empathy and contextual understanding remain central to product strategy?

Adam 25:41 Oh, wow. Um, Yeah. Okay. So there there are a few ways I could think about this. Um, AI is a tool to help us interact with information. Um, and what it's good at is spotting patterns, not defining truth. Um, in engaging with AI, I I I hear a lot of, uh, fear of inaccuracy. And I I think that comes from a fear of being wrong and this belief that, uh, AI is giving you the answer, so it must be true, etcetera, etcetera. I I think the conversation here is the difference between facts and interpretation. Um, research is here to help us interpret the facts to, uh, help us change reality. Um, I how can research I I I think that's the answer. Research is here to help us, uh, help help understand the difference this is really deeply philosophical question. Um, and I thought I had thought about this really, really well, but I I I think my my answer is changing, uh, really by the day, I suppose. And I I suppose that that's what many of your listeners are going to feel too. Uh, is that, uh, it's really hard to come up with a single answer about how humans need to need to interact in this because we're still figuring it out. Uh, things are changing around us day by day. Yeah. And, I I think we need to have an honest conversation about what AI is good for. Um, what what what does it do well that helps us automate some of our dirty, dull, and dangerous tasks? At the end of the day, that's what it is. It it's a tool to help us automate some things that that maybe take time because, you know, there's a there there's a cost to that time. Um, and it it it it's interesting that this is happening because a lot of this the work my my team did was about how to automate things in in, uh, recruiter workflows. That that's what we did at Indeed. Um, and I I'm I I brought up the the framework of dull, dirty, dangerous, and there there's a fourth one, DEAR. Um, dull tasks, those are the like, no. I I'm I'm quite happy for AI to help me, like, manage those transcripts. I I don't wanna keep tedious notes. I'd rather be present and engaged with the person I'm I'm talking with, and use AI to say, hey. So and so said this thing. I don't remember which person it was, but it's stuck in my head, and I feel it's really helpful. Um, help me find that that that's kind of a control f situation, but it's also helped me contextualize this among this other thing. I'm I'm grappling with this thing that I heard that doesn't quite make sense, and I I really appreciate some extra context and feedback. That's time, that's effort. Um, there's an argument to be made that we should be having that conversation with another human being, and I I I think that that's a conversation to be had, uh, but sometimes the realities of our work environment is that we can't schedule time with the other person to have this thoughtful conversation. And I I I found, uh, AI tools really helpful in unblocking those little moments of of, uh, this just isn't working, help me resolve through this. And it it it helps me move through my my thought process faster. It helps me organize things, um, and it helps me deal with, uh, what I've I've come to call, but I I'm stealing it from somebody else. Uh, cognitive overhead. Just that cloudy funk that that like the cobwebs are there and you just need to swat them out so you can move on. Um, and I I I think that that's the value that I found. Um, it it helps us move through tasks faster, more efficiently. Uh, sometimes it even calls out things that I have overlooked. Um, it uh, LLMs have have a significance tendency. They're they they like to please their their their, uh, counterpart, which is the human. Um, but you you can also accept it as something that may have not been diluted by your your own confirmation bias. So I I I found it useful in in in those ways. Um, and I'm I'm actually not sure that answers your question, Alana. Uh, but that No. ey like to please their their their, uh, counterpart, which is the human. Um, but you you can also accept it as something that may have not been diluted by your your own confirmation bias. So I I I found it useful in in in those ways. Um, and I'm I'm actually not sure that answers your question, Alana. Uh, but that No.

Elana 30:48 I think you did answer yeah. I feel like you from what I heard you say, it sounds like by establishing what you want out of your AI and automation and and understanding what is AI and automation good for in the context of whatever you're doing helps it remain central to I know what AI is going to be doing, and I know what my role as a human and what human empathy and contextual understanding are doing to, like, really get me to the finish line and achieve what I'm trying to accomplish. So I believe you answered the question.

Adam 31:28 Because at the end of the day, there's a decision to be made.

Elana 31:32 Mhmm.

Stephanie 31:33 Got it. I have to say, I did wanna know what the dangerous tasks were. I was waiting on that one.

Elana 31:39 Yeah. And dirty too. And dear.

Adam 31:41 I know. And I was like, dirty

Stephanie 31:43 is one thing, but what are the dangerous tasks?

Adam 31:45 The the dangerous there there are things that that, uh, could could endanger life and limb and emotional, uh,

Stephanie 31:53 emotions So not so much in market research per se, but you're

Adam 31:58 I I would disagree. Um, uh, So what what what are the dangerous tasks in market research? Uh, we're interacting with human beings every day.

Stephanie 32:13 That's fair. Yeah.

Adam 32:14 The the the fact that I'm talking to people and somebody will listen to it and what I have to say will maybe change their perspective on on the way something goes. That's a very dangerous thing. And as that decision about what Adam chooses to say comes out, do we want that to be automated? Because I I could I could say things that could endanger my my life and limb, and I could also say things that endanger the person on the other side's life and limb. Um, and as as we think about what what decisions would would bring that out. It's what kind of research Adam does. It's how Adam chooses to interpret the information around him. It's how he chooses to read the room between Alana, Stephanie, and and the the masses that I don't even know who they are. Those are all decisions I make, and the consequences of those can be quite dangerous. So do you want something automating those decisions that are made along the way? Because they they are dangerous. Just maybe not in the sense that, like, your arm will get cut off.

Stephanie 33:21 Got it. They they require care. Right?

Adam 33:24 Yes. There's a

Stephanie 33:25 yeah. Gotcha. Understood. I'm sorry. Am I I think I'm just, like, a seeker of, like, a a dramatic thing. So when you said that, I was like, oh, what's it

Adam 33:35 gonna It it is dramatic. Right? 00:33:21] Stephanie: Got it. They they require care. Right? Yes. There's a

Stephanie 33:25 yeah. Gotcha. Understood. I'm sorry. Am I I think I'm just, like, a seeker of, like, a a dramatic thing. So when you said that, I was like, oh, what's it

Adam 33:35 gonna It it is dramatic. Right?

Stephanie 33:37 It is. It is. It is dramatic. That's I think that's totally fair. Um, I love that. Uh, I wanna follow on Alana's question just a little bit only to to keep us in the context of, you know, whether it's AI or automation. Either way, I'm I'm really wanna talk about, like, the the rapid pace of insights and and what that sort of forces and requires for those of us who work in the field, uh, you know, to do to accommodate. And in particular, I'm wondering if you ever think about or have addressed how UX researchers can deploy a bit of foresight and anticipate user needs before they manifest. And if there are particular tools, you know, like systems thinking, scenario plan planning that you do, uh, to sort of focus more on foresight? ate. And in particular, I'm wondering if you ever think about or have addressed how UX researchers can deploy a bit of foresight and anticipate user needs before they manifest. And if there are particular tools, you know, like systems thinking, scenario plan planning that you do, uh, to sort of focus more on foresight?

Adam 34:30 Foresight. Um, it it it it really is the system's thinking in the context of what that's the whole point of the the the enlightenment project. Right? It it's it's being able to develop a theory in your head about something works, how something works, test it in a rigorous way, the scientific method, and then use that to update how how you view the world, how how you choose to operate through that world. Like, okay, I thought this was making the system change in a certain way, I tested it, Oh, it worked or, oh, it didn't work or I'm not sure it worked. Um, that that is those are the skills of foresight. Um, and and that that's that's in contrast to maybe reading tarot or, uh, the the assortment of tea leaves at the bottom of your cup. Sure. Um, but in the current philosophy of science that that that's that's how things work. Um, so to answer your question, uh, just keep using skills you've already had with which is the ability to find information, discern the the meaning of that information. Um, what's the provenance? Is is that an unreliable narrator on the other side? Uh, how do I how do I resolve their inputs with what I already know? And you're constantly reevaluating your your understanding of how the system works with with all those new inputs. And I I I I think with all of this stuff that is changing day by day, uh, staying in touch with it, but maybe not obsessing over it. Uh, the the obsession, I I think it just keeps us in a state of perpetual anxiety. Um, because the reality is is it will change tomorrow, So why why invest in the state of now when you you you know that tomorrow it's gonna be a bit different? Um, and going back to, like, what you can control, what you can't control, um, keeping up with what is changing and using that to adapt how you choose to navigate the world. Uh, what are the dull, dangerous, dirty, and dear tasks that that you're ready to to offload to these other tools or incorporate into your workflows to to be more efficient. Um, as you were starting your question, um, I thought, well, yeah, the the days of month month long research projects are coming to an end.

Elana 36:58 Right. ll, dangerous, dirty, and dear tasks that that you're ready to to offload to these other tools or incorporate into your workflows to to be more efficient. Um, as you were starting your question, um, I thought, well, yeah, the the days of month month long research projects are coming to an end. Right.

Adam 37:00 Um, the the reason we were able to get so much time for months long research process is because we had to manual manually manually, uh, move through all of that information, uh, through note cards, uh, affinity maps, data analysis with like, we we we had to do that more manually. Um, now the opportunity is we can move through that information much faster, which means we need to be able to update our understanding of how the whole system works faster. And, um, I I I suppose that's my answer. Uh, keep up with it. Keep keep adapting to your understanding of how the world works, how the system works, how the tools can help maneuver through that system better, faster, cheaper, easier, I suppose.

Stephanie 37:53 So it because you keep coming back to that, I and I really like it because I think this is a unique perspective in in our field. I don't think and I think it probably comes from your background to a large degree. Uh, then is the idea that the more that I can be aware of how the system is changing, the more that I can understand, again, my role in it and where I can make an impact.

Adam 38:17 Yeah. Um, at at the end of the day, there there there's a new technological frontier in front of us. Yeah. Um, and the the the collective Internet has talked about how that's going to change the nature of knowledge work.

Stephanie 38:30 Right.

Adam 38:31 I and and it is changing the nature of knowledge work. Um, just just as, uh, a computer changed the nature of a typist or the plow changed the nature of of, uh, somebody tending a field.

Elana 38:44 Sure.

Adam 38:46 Um, and there there are several ways this can go, and we're we're seeing that debate happen in front of us right now. Uh, some would even say it's a negotiation. Um, there, um, and I don't know. I I guess it is a moment of crisis for for insights professionals. Uh, where where we need we need as as the landscape is evolving, we get to participate in how how that narrative shapes out, how that negotiation shapes out, how how we choose to engage with these tools, what we choose to do with them, with the understanding that we don't get to control how other people use them. Uh, earlier in this conversation, I I talked about controlling the narrative. Now we have now we have another stakeholder we need to we need to talk about which is AI. And the reality of that stakeholder is, um, it's by nature stochastic. It will be different every time.

Stephanie 39:53 Yeah. er people use them. Uh, earlier in this conversation, I I talked about controlling the narrative. Now we have now we have another stakeholder we need to we need to talk about which is AI. And the reality of that stakeholder is, um, it's by nature stochastic. It will be different every time. Yeah.

Adam 39:54 Um, and I think that that raises an interesting philosophical question about like the nature of truth and and beauty in in a world where everything is different and hyper customized. And I I I think there's other literature outside of this podcast that would would agree with you. Um, and I I don't think it behooves us to sit by and just say, well, that that's interesting. I I think we we need to engage in in how this whole thing shakes out, and we as a we as an insights craft, uh, need to be present and aware of how this new stakeholder has entered our our our profession, um, and how we how we manage our stakeholders with that as an option. Um, and I I'm I'm I'm doing my best not not to go into, like, labor economics theory here, but, uh, what what it's going to do is it's going to reconfigure what what our craft looks like. Um, and we're we're responsible for defining how that goes. Um, I guess that's it.

Stephanie 41:05 Yeah. No. That makes so much sense. And I love that, because you said the word crisis and then immediately turned it into opportunity, which I think is important because that is what's I mean, that's all we have. Right? Is mean or we can, you know, wring our hands and and be, you know, and ride off into the sunset. But I've heard of, uh, uh, we've had a couple of other guests come on the show too and talk about, like, a little bit about, like, what does the future then, uh, you know, what are the opportunities for the the insights professional of the future? And I'm curious if either of you either of these kind of strike you. And one is, of course, that expert role. Right? It's, uh, you know, the the still providing the expertise, the lens, the guidance, the strategy. Um, and then another role I've I've heard talked about quite a lot is the data steward, which again requires a whole lot of expertise to ensure that, like, the data that you're compiling from other researchers, from AI, from, uh, generative AI, like produced results from synthetic data, that you are the steward of that data to say, you know, I'm looking across all of this and seeing this. And that that's one role where insights can have an impact. And then again, that other sort of strategic implications, connecting dots kind of role. Do you see those as, like, places where in the future where some of these dirty, dirty tasks and may maybe not the dangerous ones, but the, uh, you know, the the the administrative work, right, gets sort of off loaded in the in the analytics, and and what does that leave for humans is really the the stuff that only humans can do, right, for now? places where in the future where some of these dirty, dirty tasks and may maybe not the dangerous ones, but the, uh, you know, the the the administrative work, right, gets sort of off loaded in the in the analytics, and and what does that leave for humans is really the the stuff that only humans can do, right, for now?

Adam 42:54 Yeah. Um, that that for now was was an interesting, uh, addition. More and more of the road work that that we do will be automated. I I made a big deal about dangerous task. The the reality is is, like, Zoom has has also automated the dangerous task, which is going to a a physical location where we might get hit by car, so on and so forth. So the the automation in in those processes, uh, it is sometimes a good thing. And sometimes that automation re results in more efficient projects uh, contributing to that, like, the the months long research project, uh, work stream is coming coming to an end. I think research researchers will need to focus on more of their high value skills, And we will need to do better than a prompt to an LLM. Uh, what we bring to the table is the ability to think through a problem synthetically and through the systems lens. It's not just about the collection of facts and figures. It it is about the interpretation of them. Um, and as a craft, we'll need to reckon with the value we provide as information gatherers and interpreters. I I think this will shrink the pool of people who get to do insights work. I I I think that's a reality. It will shrink the pool of people who who get to do that. Um, and each of those people will need to be much more productive than they are today. That that that's the pressure of searching for surplus value. Like, that that's what it is. That's what the technology is going to do. Um, so it those who will get to participate in the craft, uh, moving forward will be the ones who have those skills of what is the right way to to gather that information, how do I know this information is reliable, What do I do with this information when I know it's unreliable? Um, and how do I interpret the sum of that? Uh, those who get to continue participating in the insights craft will have to do that in a very compelling way with their stakeholders who are making decisions, which means it has to be a lot less about I'm the trained professional and you should trust what I have to say, and will have to be more of this is how we work collaboratively, uh, to do what is in the user's best interest or the customer's best interest. And we, as as those critical thinkers, have to earn the the right to sit in that room. Um, that that's the when good things happen or when research is involved, good things happen axiom that I said earlier. And we have to do it in a way that is better than had we not been there. And that's our critical thinking. We we have to be able to process all of that information coming at at us, make sense of it in a way that aligns with with the consortium of stakeholders value, and then also what the market values. Um, bringing all of that information together, it it's much like an LLM, it is also stochastic. Yeah. We we we kind of have this belief that if we just measure it we we get we get to control it. But the the reality is is we're we're always re adapting and and shifting. And the the the realities are changing day by day. And if you just ask an LLM, yeah, you you get the collective sum of the in Internet, but it's also stuff that already happened. Um, as as as we, the critical thinkers, are going through the the the the the process of of changing a product, which will change the way people interact with their environment, which will change the nature of the environment they're interacting with. It's just constantly playing whack a mole with a system that that's constantly changing. Um, and we as insights professionals will those who get to continue and and and and do that will be able to, uh, adapt with those changing times at a much much more rapid pace. he nature of the environment they're interacting with. It's just constantly playing whack a mole with a system that that's constantly changing. Um, and we as insights professionals will those who get to continue and and and and do that will be able to, uh, adapt with those changing times at a much much more rapid pace.

Stephanie 47:19 Makes a lot of sense.

Elana 47:21 Absolutely. And I think now more than ever, there's this, um, additional prominence of research not being just discovering truths, but it's activating them, working with your stakeholders to, um, you know, actually go beyond uncovering user insights, but using those insights to reshape organizational priorities or product strategies. Um, Um, and I was wondering, Adam, if you could share an example where your research, uh, went beyond the research truths and insights to, um, reshape organizational priorities or product strategy in a measurable way.

Adam 48:04 Yeah. Uh, I I I'm talking about how the era of months months long research projects is coming to an end, and here I'm gonna talk about one of those months long research projects.

Stephanie 48:16 Alright.

Adam 48:18 We we did a a rather extensive jobs to be done study at at Indeed. Um, and the the point of doing that was to uncover new new opportunities.

Stephanie 48:29 Right. Yeah. s coming to an end, and here I'm gonna talk about one of those months long research projects. Alright.

Adam 48:18 We we did a a rather extensive jobs to be done study at at Indeed. Um, and the the point of doing that was to uncover new new opportunities.

Stephanie 48:29 Right. Yeah.

Adam 48:30 As as this as the space is growing, um, as, there becomes an opportunity to organically grow your user base through new features and functionality that solve a real user need. That usefulness. When when we did our jobs to be done study, uh, we we were looking for for that white space. We we were looking for those outcomes that we would we would try to manage and and influence and how that fits within the larger story of what people were trying to do when they come to Indeed. Um, obviously, like user growth. I'll I'll use a more specific example. So it was a big study that that had many applications in in several different product spaces. Um, one one of them was our, um, the the virtual interview product. Um, of with with COVID, a lot of things went went virtual. Um, and there there was an opportunity for Indeed to take up more space in the HR ecosystem. Uh, rather than just being the the receptacle for, uh, for job applications, it could actually be the next point of contact of, okay. Let let's have a screening interview. So on and so forth. Um, the the product was initially, uh, mired in in decision and complexity. Uh, there there there were competing visions about what it what it was even meant to do. And what that meant is a lot of product teams were working on usability things. So, yes, people were clicking. Yes, people were doing the interview. Yes, people were, like, doing whatever out with whatever activity was being monitored. But there there was a lack of purpose. And and a lack of connection to how that thing fit into the HR ecosystem. And Indeed's right to play and so on and so forth. Um, with the jobs to be done work, we were able to talk about the these are the jobs that people hire an interview platform for. And that interview platform isn't just Zoom. It isn't just Indeed's interview platform. It isn't just a phone call. It's a it's a it's a a panoply of things. But they're they're hired to do those jobs for different reasons. And if we understand what those reasons are. And we innovate on that with a virtual product that is connected to the user's resume. Of the candidate pipeline so on and so forth. Here's the additional value that it unlocks. Um, and when I mean the product had to struggle for a while but eventually like we we were able to start participating in the narrative loop, uh, start shaping, uh, the way the product was thinking about iterating and expanding. And once they did that, the engagement metrics went up because we we unlocked the usefulness aspect.

Stephanie 51:26 Right.

Adam 51:26 So that the usability just follows. So instead of just constantly iterating on like the size of the button or do we put the button at the top or the bottom.

Elana 51:35 Right. ey did that, the engagement metrics went up because we we unlocked the usefulness aspect.

Stephanie 51:26 Right.

Adam 51:26 So that the usability just follows. So instead of just constantly iterating on like the size of the button or do we put the button at the top or the bottom.

Elana 51:35 Right.

Adam 51:35 Um, we we were able to get it. Well, what are people actually doing here and how do we cons intentionally create a workflow that helps them do that better, faster, cheaper, easier? The insights we collected were not just those facts and figures. The team already knew the facts and figures. What what they needed was the cohesion with the story and the strategy and, um, how the how the interview process works fitting within that larger context. Um,

Elana 52:05 and

Adam 52:05 that's what unlocked the the the growth in user base, retention, repeat users, so on and so forth.

Stephanie 52:12 Makes sense. Um, Adam, we're gonna switch topics just a little bit as we're sort of getting closer to to the end of our interview. But I read that at Indeed I'm gonna move this over here so I can

Adam 52:29 give

Stephanie 52:29 me one sec. I'll start over on this part. Okay. Um, Adam, I read that indeed you kind of spearheaded a move away from NPS to more actionable measures of success. And and I love this. Um, I think a lot of us have similar experiences with NPS, both conceptually, but also just not having it pan out over time as a meaningful driver of any particular outcome, particularly in the b two b context. Um, what kinds of metrics have you found are more appropriate for understanding the impact of of your work?

Adam 53:07 Yeah. Um, I think NPS is useful. Um, I it's just not useful for everything. Um, I what what we did at Indeed by by deemphasizing NPS was we we asked the team to consider what are you actually measuring, and how do you think your your your inputs to the product experience are going to change that measurement? And, uh, that that's a hard question to answer, especially with NPS. Because NPS is it it it's asked far away from the the experience. Oftentimes days, sometimes weeks later, sometimes months later. Um, the person who responds to it isn't always the person who had the experience. Mhmm. Uh, we we found that to be true. Um, and as a result, what we were measuring was not telling us what we thought it was telling us. What we were measuring was essentially brand loyalty, Which, like, that's great. It's nice to know your brand loyalty, but how does changing the way, uh, an inbox is organized? How how does that change brand loyalty?

Stephanie 54:25 Right. t we were measuring was not telling us what we thought it was telling us. What we were measuring was essentially brand loyalty, Which, like, that's great. It's nice to know your brand loyalty, but how does changing the way, uh, an inbox is organized? How how does that change brand loyalty? Right.

Adam 54:26 It it's a much more complex thing than what the product itself can can affect. Um, and when when I asked that question to my stakeholders, I what what I got generally was a nihilistic answer of, like, oh, well, like, I'm I'm I'm held accountable to NBS, but I can't do anything about it. So I whatever. But that was oftentimes the only user experience metric that people used that that weren't transactional. Like, did they click it? Did they return? So on and so forth. Um, I had a boss at the time who was really supportive of big swings. Um, I I I worked in an environment where I had, uh, kind of the institutional, uh, support to challenge a very dear metric, something that had been measured over time that was discussed, that was well understood and well socialized, even though there was kind of this nihilistic bent on it. Um, so we engage with the spirit of NPS, which is we want people to give us a rating from this side to this side that we can track over time, and that when we change something in the product that metric will move. Um, we had a hypothesis that, uh, if we take that measurement in situ while the thing is happening

Stephanie 55:55 Yeah. the spirit of NPS, which is we want people to give us a rating from this side to this side that we can track over time, and that when we change something in the product that metric will move. Um, we had a hypothesis that, uh, if we take that measurement in situ while the thing is happening Yeah.

Adam 55:56 Uh, we will get a more sensitive measurement. So something when when something changes, it changes. Um, and that it it would parallel other other desirable business outcomes, like revenue engagement, so on and so forth. We were still solving for this usefulness thing. We were measuring usefulness. So Gotcha. I I mentioned earlier, NPS is measuring brand loyalty, which is a complex thing. Usefulness is also complex, but usefulness is very narrow in its in its scope. So we were measuring it in situ, and we were able to prove we had to prove this. Um, I I had to come up well, it it wasn't just me. Um, I had a brilliant researcher working lead researcher, uh, working on this at the time, and I I was just the guy kind of making making sure that all all the edges were were, uh, kind of sanded down. Um, but what we did is we came up with a system to demonstrate that, yes, user sentiment measurement can be very helpful, but you need to do it in situation. You need to be very closely tied to the usefulness. Why did somebody come here? Their job to be done. Um, and there needs to be some intentional, uh, sampling plan over how you're going to do it so it is a reliable metric. Um, we did it through trial and tribulation. Uh, we didn't always get things right, but when we didn't get it right, we use that as a simp an input to, like, okay, maybe we're thinking about this the wrong way. How do we change this so that the measurement becomes more reliable, um, over time? And once we're able to demonstrate that, uh, the measurement is reliable and that when product teams listen to it, when they listen to the signal that's coming and they do something about it, something good happens. And and and that's where

Elana 57:47 the scope you're making it more meaningful that the outcome and the results more meaningful rather than you like us?

Adam 57:54 Yeah.

Stephanie 57:55 Maybe?

Adam 57:55 Yep. Uh, we we we were intentional about, uh, their job to be done, where they're doing it, and uh, we we had to play around with the methodology so that we were getting reliable measure. Mhmm. Um, because that that reliability is important to the credibility.

Elana 58:12 Definitely.

Adam 58:13 And one once people realize that when when they respond to the feedback that this this this measurement system is giving them, good things happen. Um, and then that that's a self reinforcing environment.

Elana 58:25 Absolutely. Thank you for that, Adam. I'd like to wrap things up with one final question, uh, which is what is one piece of advice you would offer to insights professionals or organizational leaders? We won't exclude them. Um, but what what's a piece of advice you would offer to those who want to make their work truly transformational? you for that, Adam. I'd like to wrap things up with one final question, uh, which is what is one piece of advice you would offer to insights professionals or organizational leaders? We won't exclude them. Um, but what what's a piece of advice you would offer to those who want to make their work truly transformational?

Adam 58:57 Remember that this is not a standardized test. Um, we're we're working in the real world that is messy, and, uh, it's not happened yet. So we we can't really control the Um, that that's our desire. Our our desire is to to do good in the world. But sometimes we need to give ourselves grace. Um, this is not a standardized test where there's an objectively correct answer. We're all just trying to figure it out along the way. Um, and the giving yourself grace to be incorrect and also the the the ability to fail gracefully, um, and the ability to be wrong, I guess. Fail gracefully and and be wrong. Uh, I I guess that would be my advice.

Elana 1:00:00 Okay. I think that's great. I'm a a big fan of failing to learn failing to use that as a learning experience to be explicit. Um, I I I guess I want a takeaway for anybody.

Stephanie 1:00:16 You can be feeling your pain.

Adam 1:00:18 I I I don't think it's really about failing to learn. I think it's, uh, our our objective should not be to fail. Sure. Uh, and and I don't think you were implying that. And, however, when it does happen, it's because there was something we misunderstood. And that's okay. There we we misunderstood something about our theory of how the system works. Uh, we tried something out. It didn't work. But now we know that's not a lever we should use. Um, and that by by learning that it it should also help preclude other things we could do that save us time ultimately. So the intentionality behind failing for me is yes about learning but also what does that learning tell us about what paths are available to us so we can be more efficient in our decision making process moving forward. So I'm not disagreeing with you. I'm just clarifying.

Elana 1:01:17 Clarifying. Absolutely. And we and we

Stephanie 1:01:19 do talk about that a lot here too, Alana. I mean, I say Alana because we we work together, but about, um, about just being willing to experiment is how we talk about it. Right? And and if you are going to sort of, uh, engender and sort of foster that culture at your company, you have to give people permission to fail. Otherwise, how are they incentivized to experiment? So I think those really go hand in hand.

Elana 1:01:46 The curiosity that we we love to talk about.

Adam 1:01:49 Yeah. Um, the one of the one of the things I I use when coaching my team on on this very topic is, um, don't don't ask if it if it's correct. Rather, how would you know it's correct? And that that that shifts from that standardized test of, like, I gotta get the answer correct. Like, the methodological rigor

Elana 1:02:12 in Show your work. e of the things I I use when coaching my team on on this very topic is, um, don't don't ask if it if it's correct. Rather, how would you know it's correct? And that that that shifts from that standardized test of, like, I gotta get the answer correct. Like, the methodological rigor in Show your work.

Adam 1:02:13 Yeah. Show show your work a a little bit. Um, but it it it it's how would you know you've you've gotten there? What what are the signals that would need to improve? Um, and how can we be sensitive to those things, um, as opposed to thinking we have to get the answer correct.

Stephanie 1:02:32 Makes makes great sense. Um, well, Adam, this has been an absolutely illuminating conversation with you today. We we really appreciate it. Um, I think one of the takeaways for me is is almost just, uh, so much of how you think about research feels a bit more conceptual than I think we often we'll you know, we sit in a very it's strategic, of course, but, like, we're you know, it's so easy to get caught up in the things that you're doing without kinda stepping back and saying you know to that very point you made how would you know how would you know if it was successful and keeping our minds on the higher order conceptual things that we're actually trying to accomplish

Adam 1:03:12 and the decision to be made.

Stephanie 1:03:14 Yes. It's been so it's been a really a real treat to have you kind of ground us in all of that in this conversation. So really appreciate it. Thanks so much for your time today.

Adam 1:03:24 Thank you for your time.

Elana 1:03:25 Thank you, Adam.