Karine Pepin 0:00 When you think about the ecosystem and the neighborhood, for that to change, we need to fundamentally change the incentive models. Because the way it works today is that both suppliers and respondents are profiting from volume, not from quality. So unless we can like change that completely, I don't think the neighborhood can fundamentally change.

Stephanie Vance 0:27 Hello fellow insight seekers. I'm your host Molly, and welcome to the Curiosity Current. We're so glad to have you here. And I'm your host, Stephanie. We're here to dive into the fast moving waters of market research where curiosity isn't just encouraged, it's essential. Each episode we'll explore what's shaping the world of consumer behavior. From fresh trends and new tech to the stories behind the data, from bold innovations to the human quirks that move markets, we'll explore how curiosity fuels smarter research and sharper insights. So whether you're deep into the data or just here for the fun of discovery, grab your life vest and join us as we ride the Curiosity Current. Today on the Curiosity Current, we're joined by Karim Pepin, co founder of the Research Heads, a Qualcomm agency focused on helping brands transform insights into bite sized research assets and a fierce advocate for data quality and the participant experience in market research.

Speaker C 1:30 Karine spent more than two decades on the researcher side of the industry where data quality is not an abstract problem. It is something she has to see in the data and ultimately defend to clients.

Stephanie Vance 1:40 Today we're exploring where data quality really stands in our industry, why catching bad respondents only solves part of the problem, and what researchers can actually do when the data and their tools tell them different things.

Speaker C 1:53 Karim, welcome to the show.

Karine Pepin 1:55 Thank you. Thank you both for having me.

Stephanie Vance 1:57 Well, we're really happy to have you and excited about this conversation today. If you don't mind, we would really love to start with kind of the 30,000 foot view. If you read LinkedIn, and I know you do because you post a lot on LinkedIn, I read a lot of what you post, but also sometimes if you even just read the news, data quality kind of feels like a five alarm fire. Everyone is sounding the alarm about bots, about fraud, about attention being in a race to the bottom. You're someone who works with this hands on every day and who's very plugged into data quality at the industry level. So where do we actually stand as an industry? Is it as dire as the feeds make it out to be or do you see it differently? the alarm about bots, about fraud, about attention being in a race to the bottom. You're someone who works with this hands on every day and who's very plugged into data quality at the industry level. So where do we actually stand as an industry? Is it as dire as the feeds make it out to be or do you see it differently?

Karine Pepin 2:38 Well, I think it's pretty bad to be honest. But I also think that if you hear more about it on LinkedIn than you used to is because there are more companies trying to monetize data quality now, which makes it sound worse. But as far as I'm concerned, compared to the past five years, I don't really think it's gotten worse. I think our biggest threats are not AI agents. They continue to be people misrepresenting themselves and click farms. So I think, yes, we need to, to work a lot on data quality, but I don't think it's actually even worse than five years ago.

Speaker C 3:17 And that's, that's a great place to start because it sets up some of the tension that I want to dig into. At the 30,000 foot level, we can talk about the state of the industry, but for a working researcher, data quality isn't abstract. It's something you have to spot in the actual data and then defend to a client. So can we get into the weeds a bit? Sometimes data looks perfectly reasonable at first, but starts falling apart once you dig into the crosstabs. What tells you that you might be looking at a quality problem rather than a genuine finding? . It's something you have to spot in the actual data and then defend to a client. So can we get into the weeds a bit? Sometimes data looks perfectly reasonable at first, but starts falling apart once you dig into the crosstabs. What tells you that you might be looking at a quality problem rather than a genuine finding?

Karine Pepin 3:46 It's a good question. So I think there are a couple of things, but first is how much of the data is surprising to you. I think you need to have a good ratio of things that you're confirming and things that are new. If there's too much new, then I think that's a problem. But assuming that you're just seeing this in a few spots, the way that I handle this is that I try to find a ground truth. I think that's, that's something that's super hard for researchers, is that we don't know what. There's, there's nobody that's going to come at the end and give you a report card and said you were, you were right. Right. Like you, you just go with your instinct. So what I like to do is create myself a ground truth. So, so I identify in a data set a subset of the sample that I'm confident they're real people, they're attentive and that's my ground truth. Whatever they say is the truth. And the way that I find these people in the data set, there are different ways. So sometimes when you blend sample, you could have a subset that was done face to face and a subset that was done online. It's like, well, the face to face people are verified. Right. So, so what are they saying? It could be a LinkedIn recruit, it could be a panel that you have more confidence in, or it could even be markers from a fraud detection software, if that's all you have. So, for example, there is a fraud detection software out there that follows respondents across the ecosystem and computes a Persona. So some of these Personas are really bad, and some of these Personas are really good. So you can take the good Persona data and see how they responded compared to the rest. So in an ideal world, the two align, and you're like, okay, well, that's a real finding, because the good people are also saying that if it doesn't align, then this is when you start digging more into the data. And I think what's super challenging is that you can identify these problems in an aggregate, at an aggregate level, because the data looks wrong or it just looks like there's noise. But you can only fix that problem at a respondent level. So then that's why you have to start digging into the data and find, like, is it completely random, or is there like a group of people, a panel, demographics, like, something that is causing

Stephanie Vance 6:09 that to happen, Kind of related to that. Talking about digging in at the individual level, you use this great metaphor of data quality being a house and a neighborhood problem. Individual researchers at the individual survey level keep adding better logs, better alarms, but the wider ecosystem, the safety of the neighborhood is not necessarily improving. What would it actually take to start fixing the neighborhood? this great metaphor of data quality being a house and a neighborhood problem. Individual researchers at the individual survey level keep adding better logs, better alarms, but the wider ecosystem, the safety of the neighborhood is not necessarily improving. What would it actually take to start fixing the neighborhood?

Karine Pepin 6:38 Yeah, so I like this analogy because researchers in the past few years have been building better or just more secure surveys. Right. More secure houses. We had door locks, you know, trapped in the surveys, but now we've added a security system. We've added, like, security guards, like, you know, but if you. If you live in an unsafe neighborhood, you still don't feel safe, even if your house has barbed wire all over it. So I think when you think about the ecosystem and the neighborhood, for that to change, we need to fundamentally change the incentive models. Because the way it works today is that both suppliers and respondents are profiting from volume, not from quality. So unless we can, like, change that completely, I don't think the neighborhood can fundamentally change. And, you know, we can have better pockets of neighborhoods. You know, some panels I know are trying hard, you know, to clean their neighborhood, but because of the way exchanges work and how people are blending sample from many suppliers, sometimes it doesn't feel like you're in a safe neighborhood at all. Even though there might be pockets that are safer than others.

Stephanie Vance 7:56 No, that absolutely makes sense. And I think this might be a good opportunity to talk a little bit about the work that you do with case for data quality. Would you can you talk about that and how that relates to you.

Karine Pepin 8:08 So case for quality. A few years ago, before I joined, I joined them, they were one of the first to actually do studies about data quality to really understand what the state in the industry was. And they also did not only just look at quality across suppliers and across sources, but they also tested fraud detection software. So do they actually work? So it was, it was a really big study. And one of the key findings of the study was that the average respondent attempts 21 surveys in a 24 hour period. And they are people that complete just more surveys than humanly possible. So this is a, this is something we've known for five years now. Right. But we haven't really acted on that. So what case for quality advocates for is first of all, transparency. Transparency about where is the sample really coming from, you know, how many people are blocked, how many people are clean on the back end, all this information that suppliers should be providing so that especially if you use the same supplier for programming, sampling and doing data processing and all you get is a clean data set, you have no idea what happened in between.

Stephanie Vance 9:27 Right. ocked, how many people are clean on the back end, all this information that suppliers should be providing so that especially if you use the same supplier for programming, sampling and doing data processing and all you get is a clean data set, you have no idea what happened in between. Right.

Karine Pepin 9:27 So transparency and the other part of this puzzle is, is I guess validating people's identity because at the end of the day that's sort of like the only thing that is going to, to help improve this is making sure that people are who they say they are. And the only way to do this is to verify who the respondents are. So you know, historically you have double obtained panel and you still have them. But it's so easy to create yourself a fake email account and join, you know, 50 different, use 50 different, you know, email address to join the same panel, that sort of thing. So that's another big thing that we've been advocating for.

Stephanie Vance 10:07 Makes good sense.

Speaker C 10:08 Yeah. So I'm sure that you've been in these situations where you are faced with a bunch of respondents and multiple fraud tools flagged respondents that you thought were legitimate. So when the technology and your own reading of the data disagree, how do you decide what to trust?

Karine Pepin 10:26 Yeah, it's, that's pretty much the story of my life is to look at all these fraud detection flags and try to make sense of it. And I think at the end of the day, you know, we're all trying to do the right thing. Right. You know, there's going to be false positives and there's going to be false negative. As long as we don't identify, we don't validate people's identity. This, this is going to Happen. Right. We don't know who they are. So we're just going to have to use all these signals and, and try to figure out if they're good or fraudulent. So personally, and I know that there's like a bias there, but if, to go back to the neighborhood analogy, if I'm using a panel or a sample source that I'm very confident in, you know, I'm walking into a good neighborhood here, I will be more likely to give the benefit of the doubt to the respondent because I'm like, well, you know, I'm in this good neighborhood. It's a stranger on the streets, but I'm sure they're fine.

Stephanie Vance 11:26 Right.

Karine Pepin 11:26 If on the other end, I know that this source has a lot of fraud to begin with, I'm in this bad neighborhood now. I think all the strangers are out there to get me. And it's very difficult to give people benefit of the doubt when you're in that situation where it's like, well, if there's already 40% that's fraudulent, they probably are. But. But at the end of the day, we're just all trying to do the right thing for the data for the respondents, you know, who has spent time doing the survey and, you know, for the suppliers, because we don't want to reconcile people for no reason. But this is the kind of stuff that really keeps me up at night. ably are. But. But at the end of the day, we're just all trying to do the right thing for the data for the respondents, you know, who has spent time doing the survey and, you know, for the suppliers, because we don't want to reconcile people for no reason. But this is the kind of stuff that really keeps me up at night.

Stephanie Vance 12:04 Yeah, absolutely. You know, me too. And kind of in the same vein, you know, when we think about an individual data set where, you know, you talked earlier about the role of exchanges, we know that there are different panel sources in a lot of our surveys. Right. So when that's the case, I'm curious, like, what tells you that the source itself may be affecting the quality of the data? And really where this question is coming from for me is I think about it like, in the context of, like, sampling strategy. Like, there are times where if I divide my data by, or do a cross tab with source as the banner variable, they may look wildly different, but that could be a reflection of the sampling strategy that was used because I know that I can go and get high income, become niche respondents from a particular panel, and that's the group that I have left. Right. And so that was intentional, but it's not always easy for an end user to have access to that information. So how do you think about that?

Karine Pepin 13:06 Yeah, so for me, because I track quality metrics, you know, I use my own, not my own, but I manage my own, like fraud detection system software and processes, I don't I don't rely on the suppliers to do that. I can see how the data is coming in. I can see the fraud, you know, markers and flags. But I think what, you know, part of it is understanding what is the sampling strategy. Right. Because one of the thing that I think is actually more telling than just purely looking at, okay, percentage of virtual machine percentage, this percentage that is actually looking at your incidence rate. So if you. If there is a source that has a vastly different incidence rate and you're not aware they're targeting, then it can only be two things, right. A leading invitation or people are lying. But either way, like, you know, this is a problem. So it. It's kind of important to understand how the sampling is done. But I think it is about kind of combining all of the signals that, that, that you're. You're getting because, you know, you know, I'm sure that it will easily blend into your survey. You know, you might not realize that incidence is 50%, you know, and that's a problem. But the data looks fine. You know, you can't really spot that after the fact. So it's kind of monitoring throughout the process, you know, while you're sampling, you know, the screener data, all of that, not just look at the completes at the end. might not realize that incidence is 50%, you know, and that's a problem. But the data looks fine. You know, you can't really spot that after the fact. So it's kind of monitoring throughout the process, you know, while you're sampling, you know, the screener data, all of that, not just look at the completes at the end.

Stephanie Vance 14:36 Yeah, that's a great tip. And I mean, it sounds like a lot of what you personally do and probably advocate for is if we're going to be living and walking through these neighborhoods that are not so safe, there's a level of personal responsibility that just has to be taken right now. Like, it's. The onus is. Is sort of on us to be aware of the sampling strategy, to demand a view into sample sourcing and things like that.

Karine Pepin 15:02 Yeah. And it is something that Case for Quality has been advocating for, just like we talked about a few minutes ago.

Stephanie Vance 15:10 Yeah, makes sense. Well, Karim, we have a segment that we run on every episode called the current 101, and we ask all of our guests the same questions. In your experience, what is one trend or practice in market research or data quality that you would like to see? Stock or you could do? Sampling. Whatever works. And what is one thing you would like to see more of?

Karine Pepin 15:33 I think stop would be less false marketing claims. Nobody has 50 million people on their panel. That's just not possible. And they are certainly not verified or validated, however you use that word. And I think that's really doing a disservice to the industry and to smaller sample providers who might not have, you know, that's 5 million. But the size really doesn't matter. It's how active are people on the panel, how responsive are they, how attentive are they? That that's what matters. Not really that, you know, if people say the 50 million is probably because they have 50 million people who have ever signed up the past 35 years.

Stephanie Vance 16:12 Yeah, exactly. oesn't matter. It's how active are people on the panel, how responsive are they, how attentive are they? That that's what matters. Not really that, you know, if people say the 50 million is probably because they have 50 million people who have ever signed up the past 35 years. Yeah, exactly.

Karine Pepin 16:13 That's irrelevant today, right? Yeah, yeah, exactly. And what I would like to see more of is, you know, that's a dream because, I mean, that seems like it will never happen. But I would really like if we got together and agreed on what are the targets that you cannot do online. And I'm using online research because there's so much traffic that goes through the online ecosystem, obviously, but there are some target audience that are not online, just sitting there waiting patiently for a survey. But we never say no. Right. Because the supplier says we're feasible, the agency says we're feasible, and the client thinks we're feasible. Because we will always get data. Like getting data is not the problem. Right. Like there will always be fraudsters who will qualify for these low incidents. But we know, like, we know better. And so we can keep giving the impression that everything is possible. And I think that's really not setting ourselves up for success, for quality, because we know that, you know, the lower the incidence, the worse the quality is going to be. So I think if we eliminated a lot of that kind of, of work or audience or target or whatnot, it would actually solve some of the quality problems that we have.

Stephanie Vance 17:37 And so for you, it's, it's. I mean, because I was going to ask you. Well, how do you define that? Like, what audiences are you talking about? But you really just said, like low incidence is you probably don't want to be doing online. Is that fair to say?

Karine Pepin 17:49 Well, that's exactly why we'll never agree because we'll start discussion. But you know, I think B2B, for example.

Stephanie Vance 17:56 Oh, yeah, yeah.

Karine Pepin 17:57 Like, like, let's be realistic. Like, sure, you can find IT decision makers, but really when you understand the ecosystem, right, and how much people are getting paid and the routing system, them and all of that, how many legit IT decision makers will just sit there and wait for a survey? Right, right. So if you want to do this right, we gotta do a LinkedIn, you know, recruit and all of that. But I think what becomes difficult is if you're the one proposing the LinkedIn recruit at 200 CPI and your competitor is proposing online at $45 CPI.

Stephanie Vance 18:36 Such a good point.

Karine Pepin 18:37 How do you explain this that they can do it, but you can't.

Stephanie Vance 18:40 Right. It creates a really challenging situation.

Karine Pepin 18:43 Yeah, yeah. And I think this is a, this is a. Not something that will be fixed by one agency or one supplier, but it's really kind of trying to get together and be like, okay, let's be honest about what is possible here. But like I said, like, we are so far from anything, from anything of that nature. But we're just talking here, right? think this is a, this is a. Not something that will be fixed by one agency or one supplier, but it's really kind of trying to get together and be like, okay, let's be honest about what is possible here. But like I said, like, we are so far from anything, from anything of that nature. But we're just talking here, right?

Stephanie Vance 19:04 We are, yeah. Love it.

Speaker C 19:08 Okay. Well, we've covered both the 30,000 foot view and the practical realities of what good data quality work actually looks like. So to bring it home in your view, Corinne, is the industry actually heading in the right direction? Where are we still missing the mark? And of everything we've talked about, what is the most important work still ahead of us?

Karine Pepin 19:26 Well, I think it's fixing the neighborhood. Right. As we talked about, I think there's been a lot of work done to kind of improve security around the surveys themselves. You know, I counted 23 different fraud detection tools out there. We know the main ones, but there are other ones too. So I counted 23 of them and on my end, all I can do is curate the traffic. So this is what I will continue to do as, as best as I can, dealing with my false positives and my false negatives and trying to use better sources as much as I can, that sort of things. But I realized that this is just a band aid solution. Right. This is not actually fixing the problem for everyone. But I think we're just at this stage, I guess, of the problem that you can, you know, if you can fix your own house and make your own house secure, well, that's kind of half the battle. So I don't know how we tackle the bigger ecosystem issue of how just like structurally we're incentivizing volume enough quality. And unless, until that changes, it's really not going to change.

Stephanie Vance 20:34 So until that time, it sounds like we all need to focus on home security as much as we can.

Karine Pepin 20:40 Fair to say, I think so. I don't want to be negative Nancy here, but let's just be honest.

Stephanie Vance 20:47 No, I think that's really fair and good advice, quite frankly, for anybody who is doing research and really cares about data quality. Crane, thank you so much for joining us. I think what's especially useful here is the reminder that data quality doesn't begin and end with catching bad respondents. It shows up in the sourcing and the tools we use and ultimately in whether the data itself holds together. When a researcher starts asking harder Questions and really drilling in.

Speaker C 21:15 Yeah. Thank you, Karim, for joining us today and for sharing the practitioner's perspective on a problem the entire industry is still working to solve.

Karine Pepin 21:22 Thank you for having me. itself holds together. When a researcher starts asking harder Questions and really drilling in.

Speaker C 21:15 Yeah. Thank you, Karim, for joining us today and for sharing the practitioner's perspective on a problem the entire industry is still working to solve.

Karine Pepin 21:22 Thank you for having me.

Stephanie Vance 21:23 Yes. And on that note, now we want to shift from Karine's perspective as a researcher working directly with data to what we are seeing in the data itself. Now we want to shift from Karine's perspective as a researcher working directly with data to what we're seeing at a more macro level. Jonathan Goodbred, who's been interviewing with me today, is AYTM's head of data Polity. And John's now going to unpack our newly released Data Polity benchmark report.

Speaker C 21:52 Thanks, Stephanie. I'm really excited about this report. You know, as I came into aytm, there was a real need for us to make a statement in the market on data quality. And as we evaluated methods to do that, we thought the best way we could do that in full transparency was by reporting out the same statistics that the GDQ reports out every single quarter, which is where this report came from. So we take the gdq, we take the GDQ values that they report, we find our own internally from our data and we report out so that people can compare us to the rest of the industry. So the analysis is grounded in our 4P framework of data quality. Prevent, protect, purify and prove. And I'm really excited to share the details of it here with you today.

Stephanie Vance 22:38 Awesome. Well, let's get into it then. So, Jonathan, looking across the benchmark report, what findings tell us the most about where data quality problems are entering the research process? ent, protect, purify and prove. And I'm really excited to share the details of it here with you today. Awesome. Well, let's get into it then. So, Jonathan, looking across the benchmark report, what findings tell us the most about where data quality problems are entering the research process?

Speaker C 22:51 Right. Well, the thing is the data quality problems can really enter at any phase of the research process. And that's why we have a framework of the four Ps prevent, protect, purify, improve. And so prevent tells us that we need a great survey instrument that people can really engage with. And so it's a measure of how much cognitive load and the kind of survey design we're doing in order to ensure we get great quality respondents who are paying attention. And in that sense, We've got a 5 minute and 44 second median LOI here at AYTM, which I'm really happy to see because we know that respondents are willing on average to give us about 10 minutes of their time. So a five minute survey on average, and that does include IR checks and things like that, that tells us that we're well within that window and that we are being good custodians of people's time. And so once you've built that instrument, you got protect yourself from the fraud. Right. And so there's a whole chunk of people and you don't even want them to see your survey. And we've got 46% removal for that from our third party providers. But if you add in paid viewpoint, which is our panel, you get 47% and I think that's the more fair number. And that was a really gratifying number to see because if you look at the research right now, a lot of the research is saying for high incidence work you should be pulling out 30 to 40 at the gate. And we do. But when the incidents on the audience goes lower, that number has to go higher. And so the fact that we're hitting 47% tells me that we're doing the work we need to do to pull the fraud before it hits the link at all. We then have our purify step, which is okay, we protected ourselves, we created this great survey instrument. Now what are we going to do about whatever fraud is left over? And we see a 5.4% post survey cleanout rate. So we get the right people in and out of all of those human beings that answered our survey, about 5.4% of them we toss out for data quality challenges. And so we're finding the fraud that's left over, which is so great. And then there's prove. And prove is what's really, really important in all of this because you can make all the data quality claims that you want, it doesn't matter if you can't back it up. And we put a data quality report on every single survey link showing you who, who got removed at what stage and why. And that way you know that the first three stages have given you great data quality.

Stephanie Vance 25:15 I love that. It makes a ton of sense. And I guess one thing that I would wonder if you could unpack a little bit is what are some of the signals that we see in the data that let us know that these are respondents we should move once they're remove once they're in the survey? at data quality. I love that. It makes a ton of sense. And I guess one thing that I would wonder if you could unpack a little bit is what are some of the signals that we see in the data that let us know that these are respondents we should move once they're remove once they're in the survey?

Speaker C 25:29 Yeah, absolutely. And it's going to depend on what's in the survey in the first place? Right. But when we remove from a survey, what we're looking for is inattentive people who got looking for bot farms or groups of people working together. And then there's AI agents and those aren't a threat yet when agents are fast and cheap, we'll have a different conversation. But they are slow and expensive. So agents taking Your surveys is not yet a big thing. And so we look at indicators that show us that these kinds of people are in our survey. And so we look for people with extraordinarily long Lois on a survey that does not require it. We look for people who give us poor open ends, or in the worst case, open ends that we can literally tell came from.

Stephanie Vance 26:19 Right.

Speaker C 26:19 And so we look at some of those behaviors of the respondent, we look across respondents, and we see if there's patterns in how they answered and the way in which they answered oe verbatim fields. Right. And so we use the data that they provide us in order to get a sense of how they're interacting with our survey. And as you note bad behaviors in the data, those are the individuals that you are going to want to remove.

Stephanie Vance 26:41 That makes sense. You know, the report also raises questions about study design itself, and we haven't talked very much about that. What do researchers need to rethink about survey length, survey content, and respondent experience if they want better quality data? re going to want to remove. That makes sense. You know, the report also raises questions about study design itself, and we haven't talked very much about that. What do researchers need to rethink about survey length, survey content, and respondent experience if they want better quality data?

Speaker C 26:58 Oh, yeah. You know, since 1991, we've known from Krosnick that the more cognitive load that's in a survey, the more people are going to satisfy going through the survey. And so it is our job as researchers to actually put out a survey instrument that people are going to engage with and that people are going to feel compelled to give us their honest answers. And that means accounting for respondent cognitive load. And there's many ways to do that. One we use is the length of interview. Right. So we, we again know that respondents are willing to give us about 10 minutes of their attention for a survey, but we also need to look at the cognitive load of the individual components of the survey instrument. So if you have something that's very cognitively taxing, like a maxdiff exercise or perhaps a conjoint, those belong at the beginning of the survey because that's when the people have the most cognitive energy to put towards your exercise. You put them at the end. You're looking for satisfy satisfying. So you also need to make sure that you are not just stacking these hard exercises one after another after another, because people's eyes will glaze over and they will stop participating. So what you really need to do is design a survey of an appropriate length of interview, 10 minutes or less. You need to put your cognitively loaded exercises up front, the easier questions in back, so as they get more tired, they have easier questions to answer. And then you're going to come out with a survey instrument that People really want to pay attention to and work with.

Stephanie Vance 28:20 It really highlights that everybody has a role in making data quality better. Right from the panel providers all the way through to the researchers that are boots on the ground working with the data to tell the story. rvey instrument that People really want to pay attention to and work with. It really highlights that everybody has a role in making data quality better. Right from the panel providers all the way through to the researchers that are boots on the ground working with the data to tell the story.

Speaker C 28:34 Yeah, absolutely. Data quality has to be an event in which everyone participates, and I do mean everyone from across the industry, be you a panel provider or a consultancy or a tech firm that services the broader research and insights community. We all have to be involved in this because at the core of what we do is our research. And if our research does not talk about real world scenarios, if it's just a puff of insights in the wind instead of being grounded in real world views, it's not useful to anyone. And that hurts your dynamo. That hurts our personal reputations, it hurts our organizational reputations. If everyone doesn't participate and demand that our that our data quality gets better, we're going to be in some trouble. So my viewpoint on this is that we need transparency from every player in the game. Sometimes that transparency isn't going to sound great, but we have to accept that we are in a data quality challenge. Let's all work together, become transparent and figure out ways to get out of this.

Stephanie Vance 29:35 Absolutely. Well, Jonathan, I think this brings us full circle. Karine gave us that view from inside the data set and the benchmark report really gives us another way to see what where these quality challenges are actually showing up.

Speaker C 29:49 Yeah, absolutely. And reporting like this is so important. I love that we're doing this at aytm. I would love to see other organizations start doing this too. And quite frankly, I would love to see us all compare numbers. And it's not that I think that it's important to know who's best and who's worse according to which metric. It's that I think as we come together and we understand our panels together, that we're going to be able to handle this data quality challenge more effectively.

Stephanie Vance 30:15 Yeah. Jonathan, thanks for helping us break down the report. And to everyone who's listening, thank you for being part of the Curiosity Current. We'll see you next time.

Speaker C 30:24 Thank you so much, Stephanie. It was a pleasure to be here.

Stephanie Vance 30:28 The Curiosity Current is brought to you by aytm. To find out how AYTM helps brands connect with consumers and bring insights to life, visit aytm.com and to make sure you never miss an episode, subscribe to the Curiosity Current on Apple, Spotify, YouTube, or wherever you get your podcasts. Thanks for joining us and we'll see you next time.