Questioning Everything We Think We Know About AI in Law ft. Daniel Schwarcz

The impact of AI on law has been studied through the narrow lens of productivity and profitability. But the reality is much broader than that. In this episode of Between the Briefs by Steno, Adrian Cea and Joe Stephens welcome back Daniel Schwarcz, Professor at the University of Minnesota Law School to discuss groundbreaking empirical research on how AI impacts lawyer performance, legal reasoning and long-term professional development.

What You’ll Learn:

How to move beyond benchmarking studies to measure what actually matters
Why reasoning models and retrieval-augmented generation (RAG) platforms deliver measurable improvements
The counterintuitive finding that using AI for research strengthens independent legal reasoning when AI is unavailable
How litigation work benefits more measurably from current AI tools than transactional work
The critical distinction between confidentiality risks and genuine professional concerns
Why high-valuation legal tech firms like Harvey and Legora face competitive pressure from accessible alternatives

This episode challenges everything we think we know about AI’s cognitive impact on lawyers and their practice.

Highlights:
00:00 Introduction & Meeting Professor Daniel Schwarcz 2.0

02:09 How AI Progress Has Defied Earlier Predictions

03:34 Getting into the Research: New Findings & More

05:56 Why Randomized Controlled Trials Matter for Lawyers

10:47 Reasoning Models vs. RAG: Choosing the Right AI Architecture for Your Practice

16:40 How and Why AI Research Skills Persist Even When Technology is Removed

23:00 Legal Tech Vendors, Study Citations & Navigating Selective Evidence

26:43 Why It’s Critical to Use Foundation Models Responsibly

30:02 AI's Role in Access to Justice & Pro Se Litigants and Future Research

34:00 The Long-Term Effects of AI in Law and Emerging Research Questions

36:19 Human vs. AI: Context-Dependent Futures in Legal Practice

39:05 Why Empirical Evidence Shouldn't Slow Adoption

42:31 Daniel’s Hot Take: High-Valuation Legal Tech Firms Face Significant Market Pressure

44:12 Key Takeaways & Closing Thoughts

Quotes:

"I'm excited about this new paper I have that tries to look at not only what the effect on quality and speed is of using AI on legal tasks, but also what the effect is on the ability of people who are using AI to reason when the AI is taken away from them. I think that's an important step in trying to understand the broader scenario that all lawyers and members of the legal academy more generally are facing right now."
"The mere fact that an AI can do well on an exam or on a task doesn't really answer the question most lawyers want answered, which is: how will I perform when I use AI compared to how I perform when I don't use AI? You can't extrapolate from the fact that an AI performs well on its own to the fact that if I use an AI, I'll start performing well on my own."
The basic interpretation is that AI helped our participants understand the legal source material better - they understood the case law better, they understood the law better. That produces benefits that persist even when they no longer have AI."
"The study made me more confident than I would otherwise be that using these tools can be beneficial in many ways, even in ways that aren't fully intuitive.”
"In some contexts, the practice of law will absolutely look like AI judgment being primary with humans handling exceptions - like in will drafting where an AI handles the initial draft and a human checks it. But in other settings with massive human elements like counseling clients or talking to judges, AI won't be a substitute because the human element is essential."