Alisa Hamilton 0:01 AI is going to make it much easier for us to create realistic test environments. And I've always been a huge fan of choice modeling exercises. I just think it's way more realistic than asking people to, like, just rate things and to say, oh, yeah, that rating of the frozen pizza is going to predict what they're going to buy. But I think that shelf testing and even testing of physical space and all those kinds of things used to be prohibitively expensive, take a long time. And I really think now AI is going to bring us to a point where we can start testing things in semi realistic environments much easier than what we could have in the past and for much less money than we could in the past.

Speaker B 0:51 Hello, fellow insight seekers, Molly. I'm your host, Molly, and welcome to the Curiosity Current. We're so glad to have you here.

Stephanie Vance 0:59 And I'm your host, Stephanie. We're here to dive into the fast moving waters of market research where curiosity isn't just encouraged, it's essential.

Speaker B 1:08 Each episode, we'll explore what's shaping the world of consumer behavior. From fresh trends and new tech to the stories behind the data, from bold

Stephanie Vance 1:17 innovations to the human quirks that move markets, we'll explore how Curiosity fuels smarter research and sharper insights.

Speaker B 1:25 So whether you're deep into the data or just here for the fun of

Stephanie Vance 1:29 discovery, grab your life vest and join us as we ride the Curiosity Current. Today on the Curiosity Current, we are joined by Alisa Hamilton, founder of Harvest Insights and a researcher with more than 20 years of experience across both, both qualitative and quantitative research.

Speaker B 1:49 Elisa has worked through lots of shifts in this industry, but the one that's happening right now feels particularly significant. AI and new methodologies are changing what researchers can do, how quickly they can do it, and potentially where their value comes from.

Stephanie Vance 2:03 At Harvest Insights, Elise has been figuring out how to bring those newer capabilities into the work without losing the judgment and human understanding that good research depends on.

Speaker B 2:14 So today we're asking what research should embrace, what it should be careful not to lose, and what the next generation of great researchers may need to do a little differently.

Stephanie Vance 2:23 Elisa, welcome to the show.

Speaker B 2:25 Thanks.

Alisa Hamilton 2:25 I'm excited to be here.

Stephanie Vance 2:27 Well, we're so excited to have you. As we just mentioned, you've been doing qual quant research for more than 20 years. Same.

Speaker B 2:35 Yeah.

Stephanie Vance 2:36 Of all the changes that you've seen in that time, I'm curious to kind of kick us off here. What feels genuinely different about the moment that we're in right now?

Speaker B 2:46 Yeah, that. ou've been doing qual quant research for more than 20 years. Same. Yeah.

Stephanie Vance 2:36 Of all the changes that you've seen in that time, I'm curious to kind of kick us off here. What feels genuinely different about the moment that we're in right now?

Speaker B 2:46 Yeah, that.

Alisa Hamilton 2:46 That's a great question. Because we have been through lots of shifts from, you know, mail surveys to online. And I think what feels different now is that we, we're not making one shift, we're making like 100 shifts all at the same time. Because it's not just changing sort of one type of methodology, we're sort of changing all the methodologies at the same time. And so it's creating, I think, a lot more challenges, a little bit of chaos, and also a lot of opportunities, depending on how you look at it. On any given day, I feel we're going in any of those three directions or all at once.

Stephanie Vance 3:32 Very fair. And I think we'll probably unpack a lot of that as we kind of go along through the conversation today.

Speaker B 3:38 Yeah, I was just going to say let's start on the topic of responsible AI in research. Right. Because there's lots of pressure right now, top down, from clients to adopt AI. Oftentimes it's simply because it's available or someone said that we had to use it. So what's something in that lens? What's something that researchers are using AI for right now that makes you think, actually we really don't need AI for this?

Alisa Hamilton 4:04 One thing that people are using AI for that, I think, is not turning out the way a lot of people had. Hope is creating PowerPoint reports from AI. And it's funny because I was feeling like they were all starting to look the same. And then I had opportunity to pitch an idea. And the pitch decks, there were five of them and four of them all looked the same. And I was like, oh, okay. So everybody used Claude for these.

Speaker B 4:40 And was the language very similar in it as well, like the structure of the language?

Alisa Hamilton 4:45 Yeah, it really was. And the. There's always a slide that has like, box, box, box, box, box. And the bullet points being off. I mean, it's uncanny how similar they are. So it's not that I think that humans can always do a better job. I just, I don't think AI is there yet and. Or I don't think we've trained it to be there yet.

Stephanie Vance 5:07 That's a fun example, I think. And I say fun only because, get it, you know, it's. There aren't a lot of situations in which you get to kind of compare several different things at once and be like, well, these are the ones that were clearly created by AI and clearly a human sat down and thought about this for the fifth one that's fun to think about. Kind of in that same vein of what is AI good for and what Is it not good for as more of the research process, which undeniably research execution, I think is becoming more automated faster. A lot of that is due to LLMs and other types of automation. What parts of the role of the researcher, the human, become more valuable maybe than less in this world? is AI good for and what Is it not good for as more of the research process, which undeniably research execution, I think is becoming more automated faster. A lot of that is due to LLMs and other types of automation. What parts of the role of the researcher, the human, become more valuable maybe than less in this world?

Alisa Hamilton 5:50 I actually think that kind of the front end of the research process. So the. What questions are we going to ask? How are we going to ask them? What methodologies are we going to use? It's almost even more important now because methodologies are not as black and white as they used to. We don't just do surveys and focus groups anymore. There's like all, you know, combinations of things that you can do and even to new methodologies. So I think really putting the thought into what questions are we trying to ask and then how are we going to answer them is really important. And then on the back end, having seen reports that, you know, AI just spits out, I really think that a lot of the nuance and the curiosity that humans bring in the back end, looking at that data, whether it's qual or quant, and, and asking, well, why this, why that, and sort of continuing that line of questioning even after the survey's over or the focus groups are over, is super important.

Stephanie Vance 6:57 Yeah. So you're talking about really both upfront, Right. When we're deciding how are we going to address this business question. And then also on the back end when it comes to interpreting human behavior.

Alisa Hamilton 7:07 Right.

Stephanie Vance 7:07 Like a stochastic model is it's going to tell you something, but it's not the same as a human looking at human behavior.

Alisa Hamilton 7:14 Yeah, absolutely.

Speaker B 7:16 I do want to dig a little bit deeper into something you said at the beginning of that, which was that there are methods that are changing. There are new and emerging methods to address things that maybe weren't around or something that wasn't measured when you started your career. So what new approaches are you seeing that are genuinely changing how you do work today versus other types of things that are maybe just giving old methods a new rapper or a new name? are new and emerging methods to address things that maybe weren't around or something that wasn't measured when you started your career. So what new approaches are you seeing that are genuinely changing how you do work today versus other types of things that are maybe just giving old methods a new rapper or a new name?

Alisa Hamilton 7:41 Yeah, I mean, I think the, the obvious one and the one everyone always talks about is AI moderation. You know, let's talk about that. I think we can go beyond that. I think we can think of that not as a methodology, but as a tool that we can use to plug into other methodologies. And I think it's really great for that. I think it's great for collecting upfront information about a topic and then weeding through the responses that you go through and picking out, you know, the people that you want to dive deeper into and have a real human conversation with. You know, I think that's a pretty easy example of that's not necessarily a new methodology, but it's definitely a different way to sort of think about collecting the data that you want to collect and how to ensure that you're qualified. Quality as good as it can be. I think that another thing that AI can do, which probably like, not a lot of people think of, but AI is going to make it much easier for us to create realistic test environments. And I've always been a huge fan of choice modeling exercises. I just think it's way more realistic than asking people to like just rate things and to say, oh yeah, that rating of the frozen pizza is going to predict what they're going to buy. But I think that shelf testing and even testing of physical spaces and all those kinds of things used to be prohibitively expensive, take a long time. And I really think now AI is going to bring us to a point where we can start testing things in semi realistic environments much easier than what we could have in the past and for much less money than we could in the past. So I feel like we're running the gamut from just improving what we could do before to pushing the envelope of getting much closer to real human behavior.

Speaker B 9:42 So understanding that the what you're trying to answer might be fundamentally the same, but you have better access to different ways of achieving that.

Alisa Hamilton 9:52 Yes, absolutely, I think so.

Stephanie Vance 9:54 Well, and to your point about choice modeling too, that's different than kind of in context research, which I totally agree with you about. And I'm already seeing where Molly and I work at AYTM where we're able to sort of use these methodologies and it's so much fun. But with choice modeling, I think one thing that AI does, I don't feel like it's introducing a lot of new methodologies, but it's making those methodologies more accessible. And so, you know, used to, you had to be a choice modeling expert. You had to be somebody with a statistics background. But now if you have a conceptual understanding of what you're doing, you can have AI make a design sheet, for instance. a lot of new methodologies, but it's making those methodologies more accessible. And so, you know, used to, you had to be a choice modeling expert. You had to be somebody with a statistics background. But now if you have a conceptual understanding of what you're doing, you can have AI make a design sheet, for instance.

Alisa Hamilton 10:30 Absolutely. And think about how that sort of democratizes good research. You don't just have to use a Google form or a survey monkey or ask, just ask 10 of your friends, like you can actually get in and do some real scientific testing without having your PhD in statistics. Well, let's talk a little bit about

Stephanie Vance 10:50 two of everybody's favorite topics, speed and cost. As we know they've always shaped research decisions in the sense of, you know, what are we going to invest in? What can we reasonably invest in given timing constraints, cost constraints, et cetera. But one of the things that AI is doing is really pushing the envelope on both of those things. It's definitely kind of a curvilinear situation where suddenly we're moving so much faster than we ever have and in a more cost effective way. Where do you think that efficiency starts becoming maybe a threat to the quality of either the research or the thinking, either at the front end or the back end?

Alisa Hamilton 11:28 I think that's a great question because clients always been pushing the envelope on how, how fast can we get it and how cheap can we get it. Oh, and also we want it to be perfect, right? Always.

Stephanie Vance 11:40 Yeah.

Alisa Hamilton 11:40 The three magic things. Okay, you can have two, but not three.

Speaker B 11:43 I was just gonna say you can pick two.

Alisa Hamilton 11:45 I think that we can do things faster. I think a lot of people right now maybe still slightly disappointed by how still sort of manual things feel at this point and how you can do things fast and you can do things cheap. But I think a lot of people using AI right now feel like it's actually the third thing that's missing. It's the quality. I don't know that there's a place anywhere in the process where you can just throw something into AI and get an answer back. That would be quality under anybody's measure.

Speaker B 12:26 This podcast is brought to you by aytm. We built AYTM to help research and marketing teams get real answers fast without cutting corners on the research itself. From quick concept tests to bigger strategic questions, it's made to keep up with how your teams actually work. Check out more@aytm.com we talk about this r B: This podcast is brought to you by aytm. We built AYTM to help research and marketing teams get real answers fast without cutting corners on the research itself. From quick concept tests to bigger strategic questions, it's made to keep up with how your teams actually work. Check out more@aytm.com we talk about this

Stephanie Vance 12:46 a lot internally with the researchers I work with where we sort of struggle with, like, automation is good. It's important. There are steps within the research process that absolutely can be automated. I think there are things in the QA process, for instance, that can be automated. Not the full stack, but some pieces of it. Everything doesn't have to be automated through AI, though. There are all sorts of different types of automation. And I think there's a little bit of this singular focus in probably every industry right now where it's like, AI is the solution to all our problems and that's not necessarily the case, right?

Alisa Hamilton 13:21 Oh, Yeah, I agree 100%. And I think that sometimes it takes longer to train AI to be really good at A task if it's not a task that you're doing repeatedly and sometimes trying to train AI. Some people think of AI as like a high school intern, okay? Not a college intern, a high school intern. So you have to really explain every little thing and iterate with it until it learns what you want. That doesn't always make sense if it's something that you don't do very frequently. But like you said, for repetitive tasks, even for those, maybe there's other answers that aren't, aren't necessarily AI.

Speaker B 14:06 And it's also hard too because we've always in the research industry had to justify the reason that we exist. But there is an additional justification that goes along with that of there's a difference now between I can't just slap a raw data sheet and put it directly into the LLM of my choice and it actually give me back something useful. But what's difficult is that you might not be able to tell that. Do you not have the background in research to understand that maybe this is not correct and you do still have to guide it in that way. But to your point, if I'm only going to be doing this a handful of times, do I really need to sink the time into getting this perfectly right using this system or should I just use a different system? So there's definitely a trend that we're seeing too of let's just do everything in AI and then whatever messes up, we'll pull it back.

Alisa Hamilton 14:54 Yeah. Which I kind of like the reverse which is let's do it to the best of our ability. Then let's put it in AI and see if. Did we miss anything? Could I enhance this some way? Does it give me some additional ideas? Can it just clean up because I use six different fonts, Things like that.

Stephanie Vance 15:15 Right? Yeah, it's great at fonts.

Alisa Hamilton 15:18 It's better than I am at fonts. en let's put it in AI and see if. Did we miss anything? Could I enhance this some way? Does it give me some additional ideas? Can it just clean up because I use six different fonts, Things like that.

Stephanie Vance 15:15 Right? Yeah, it's great at fonts.

Alisa Hamilton 15:18 It's better than I am at fonts.

Speaker B 15:20 It's also great. We had a guest on and it stuck with me a while back. He was talking about how AI was probably the best probability calculator that it couldn't pull out true human impactful findings. But it can do a really excellent job of pointing out where things happen most often and what's most likely to happen. Instead of just taking it as like that it's actually thinking that it's actually intelligent. Instead it's just using the past to be a very good indicator of the future.

Stephanie Vance 15:50 In that same vein, I think things like throwing a cross tabs into AI and being like write up all of the significant differences.

Speaker B 15:58 Do you Know what I mean?

Stephanie Vance 15:59 Because when we do that manually, that is this extremely time consuming task and we're sometimes going to miss stuff. And the thing about AI is that it's got the eyes of a hawk. I'm just going to say that, right? Like it catches the little tiny things that our human eyes miss. But that's not a deliverable, that's like a thought starter.

Alisa Hamilton 16:16 Yes, exactly.

Speaker B 16:17 Let's switch gears a little bit and talk about the humans that are going to be entering this amazing industry of ours. You are currently mentoring younger researchers who are entering the field build a very different time than when you entered. What is something that you are encouraging them to build now or to prioritize skill set wise now that you may not have had to work so hard at or prioritize 10 to 20 years ago. industry of ours. You are currently mentoring younger researchers who are entering the field build a very different time than when you entered. What is something that you are encouraging them to build now or to prioritize skill set wise now that you may not have had to work so hard at or prioritize 10 to 20 years ago.

Alisa Hamilton 16:44 That's a great question because we talk about this all the time. How is the next generation going to learn what we learned by doing everything hands on? I can't remember who I was talking to that told me one of the bigger companies was not letting their first year interns or first year analysts use AI. I can't remember which company it was, but I was like, whoa, that's crazy, but interesting. But I think that to answer your question, one of the things that I've always been struck by when working with people that are, especially those that are coming out of like the MMR programs is they have such a good grasp of the theory of what we do. They don't always have the best skills for true execution. And that comes with years and years and years of doing this. So I think now when I'm talking to them, in the past I would have said you should go learn how to program in Qualtrics, because once you learn how to do it, you can pretty much program in anything else. Well, I don't think those programming jobs are going to be there anymore. There's just too many alternatives that you don't need. All of these certifications to program. AI is going to help with that. So now I'm kind of like, I feel more that it's sort of a breadth rather than a depth of one thing. Go out and get free subscriptions to a bunch of different services and try programming something in each of those so you can understand sort of logically what's going on and know how to understand or skip pattern and things like that. But you're not going to have to know all of the little tiny nuances of your survey flow and all that. I, I don't think in the future. So that's what I've been telling people. But I still think like the human. I still encourage people, if they're interested in qual, to go get certified as a moderator. I think we're still going to have human moderators 100%.

Stephanie Vance 18:53 No, I agree with you. We could both be wrong, but I just agree with you wholeheartedly. been telling people. But I still think like the human. I still encourage people, if they're interested in qual, to go get certified as a moderator. I think we're still going to have human moderators 100%. No, I agree with you. We could both be wrong, but I just agree with you wholeheartedly.

Speaker B 18:59 I'm stuck on what you said about that, you know, somebody whose interns or first years are not allowed to use AI for things. And what's fascinating about Matt is that that's probably, I would assume, almost a detriment to the company if they need to just be doing things really, really fast. Like, it's easier to just say, go, do AI and do all these GR tasks and just do. Check it off really, really fast. But there's an onus on organizations now to put effort into the training of the next generation of researchers. That just happened when? 10, 20 years ago, or even, I don't know, just a few years ago before AI, that was still happening. But now it's actually taking a little bit of discipline to say, actually no, we're going to take a step back and invest in you and invest in the future of the industry. That is a topic that we've heard some educators talk about on, on the show about how that's going to end up being really important for businesses to maintain that sort of discipline and ethic moving forward.

Alisa Hamilton 19:59 I agree. I haven't really formulated a fully baked idea of whether I think it's. I, I think it's good, but I'm like, sounds like a fun experiment. Let's see how it goes.

Stephanie Vance 20:12 Totally. It reminds me too of this idea of, of reps, right? And when are reps important? And I think it's what you're saying about programming is so true. I don't think those programming reps are going to be all that necessary going forward. But when I think about things like questionnaire design, because I'm more of a quantitative researcher, when I'm reviewing something that somebody more junior has done and there are times where I'm like, oh, don't do that. And the reason I know that is because I've done it before and gotten burned by it, right? It's like, you'll get no differentiation on that. Don't even bother. And it's like, I don't know that anybody can teach you that. I think sometimes you just have to have the experience, experience getting burned a few times to sort of build that knowledge base of what Works well and what doesn't?

Alisa Hamilton 20:55 It's really, I think, an open question. How are people going to make those mistakes? Are younger folks going to make those same mistakes or will they be caught if they run those same things through AI? I don't know. We'll be retired by then, so we'll

Stephanie Vance 21:13 be able to watch from, from outside. Elisa. We have a segment that we run on every episode of the Curiosity current called current 101, and we ask all of our guests the same question. So we're going to ask you now, in your experience, what is one trend or practice in market research that you would like to see stop and what is one thing that you'd like to see more of? ve a segment that we run on every episode of the Curiosity current called current 101, and we ask all of our guests the same question. So we're going to ask you now, in your experience, what is one trend or practice in market research that you would like to see stop and what is one thing that you'd like to see more of?

Alisa Hamilton 21:35 I have always been annoyed. This is going to be a, this is a little bit of a weird one, but I really think that the lines between qual and quant are going to blur. I've always thought that for years and years and years, because I really do think that the best way to answer most questions is through a combination of them. And I don't think that qual and quant, it's just such industry speak. And so one of the things that I would like to see stop is us constantly drawing a line and being like, well, Zach Ball or Zach want is that one. Because I just don't think that it means anything to the rest of the world. And it's eventually not going to mean much to us. Us either, because all these new methodologies are going to be some kind of blend.

Stephanie Vance 22:27 That is a good one. I love that one. Okay. And what's something you want to see more of?

Alisa Hamilton 22:31 I want to see more of using AI for new methodologies, not just let's all have AI moderators. Like, we've been there, we've done that, we've tried it, it works. They'll get a little better. Whatever, it's fine.

Speaker B 22:48 But like, I just feel there's so

Alisa Hamilton 22:51 many more opportunities out there for how we can use this tool. And we've kind of just dug ourselves into this hole where this is what we offer. And I just think that we can do a lot of more and we should be doing a lot more. And if we don't, then just like in other times where analytics or big data or all those things came in and took a piece of market research from us, we will be overcome by some other field that finds a way to use it in a more creative way. Those are pretty high level theoretical things.

Speaker B 23:29 Well, I think the start that you mentioned was very much like asking for A faster horse. We want AI to just. You can have a robot in a room with people asking questions. That seems very odd versus maybe utilizing a new way of thinking about that, utilizing that technology to get something similar. I think that that's going to, like you said, sort of a higher level theoretical thing, but like a mind challenge to figure out what's the new way to get to the answer. That's not just brute forcing robots into the existing methods.

Alisa Hamilton 24:00 That is a great way to put it.

Stephanie Vance 24:01 But we should brute force the robots to do whatever we want them to do.

Speaker B 24:05 Of course.

Alisa Hamilton 24:05 Well, if not, they'll brute force us, apparently.

Stephanie Vance 24:07 Exactly.

Speaker B 24:09 Hey, that's why I'm always nice to my, my LLMs anything. I'm like, thank you so much, Claude. I really appreciate your time.

Alisa Hamilton 24:17 You'll make it longer than the rest of us.

Stephanie Vance 24:19 Yeah, you will. force us, apparently. Exactly.

Speaker B 24:09 Hey, that's why I'm always nice to my, my LLMs anything. I'm like, thank you so much, Claude. I really appreciate your time.

Alisa Hamilton 24:17 You'll make it longer than the rest of us.

Stephanie Vance 24:19 Yeah, you will.

Speaker B 24:20 Yeah, I hope so. Well, Elisa, thank you so much for joining us today. I just have one final closing question for you. With methodologies and technology changing so quickly, as we've been talking about, what's one thing if you were to distill it down to one thing about being a great researcher that you don't think is ever going to change?

Alisa Hamilton 24:41 I think the curiosity piece, I mean, not just to play on the name of your podcast, but yeah, I was

Speaker B 24:46 going to say we did not pay her to say that. Promise.

Alisa Hamilton 24:50 I think actually it's coming to mind because I'm doing some moderating right now and going to be helping a younger researcher do some moderating soon. And I keep thinking to myself, I need to remember to tell her to just stay curious. It's okay to go off script a little bit if you are curious about something that is related to the objectives of the project. And that's one thing that I don't know that the AI will ever quite take over, I think, is that sense of. Well, why is that?

Speaker B 25:22 Yeah, the human. Wondering.

Alisa Hamilton 25:23 I could have said wondering and then I wouldn't have.

Speaker B 25:26 Oh, it's appreciated. Well, thank you so much, Elisa. You left us with a lot to think about today. We really appreciate you joining us.

Stephanie Vance 25:34 Absolutely.

Alisa Hamilton 25:35 Thank you, ladies.

Speaker B 25:37 Well, Elisa, thanks so much for joining us today. I think what I'm going to be coming back to in this conversation is that a lot of the new technology, specifically AI, that we're seeing doesn't necessarily make the fundamentals of research any less important. And even in some ways, it puts more pressure on researchers to take a closer look to know what good work actually looks like.

Stephanie Vance 25:58 Absolutely. And I think as more of the mechanics of research become easier to automate, which we just keep hearing over and over again. Judgment becomes a much more interesting and honestly critical part of the equation. Knowing which question to ask, which method fits, and what really deserves our attention.

Speaker B 26:17 And I also really appreciated the perspective from someone who has worked across both qual and quant, although we're not dividing those anymore, that she's been doing this for so long and that the tools may change but that understanding people, staying curious and asking those questions is still the job underneath everything.

Stephanie Vance 26:35 Elisa, thank you so much for joining us and giving us your perspective on where research is changing and where its foundations continue to matter. g and that the tools may change but that understanding people, staying curious and asking those questions is still the job underneath everything. Elisa, thank you so much for joining us and giving us your perspective on where research is changing and where its foundations continue to matter.

Speaker B 26:44 And to everyone listening, thank you so much for being part of the Curiosity Current. If you want more content from this podcast, be sure to visit the CuriosityCurrent.com and listen in to all of our past conversations. We'll see you next time.

Stephanie Vance 26:58 The Curiosity Current is brought to you by aytm. To find out how AYTM helps brands connect with consumers and bring insights to life, visit aytm.com and to make sure you never miss miss an episode, subscribe to the Curiosity Current on Apple, Spotify, YouTube, or wherever you get your podcasts. Thanks for joining us and we'll see you next time.