The Mit List | Episode 2: Ronuk on the AI Hallucination Snowball

Hello. Welcome. I’m here with Ronuk Raval. He is our chief technology officer here at Encircle. Ronuk, you recently sat on a panel of some powerhouses in the restoration technology space, where some of the leaders were talking about AI and restoration. Can you tell me what your main takeaways were from that panel? Yeah. I mean, it was at the Paul Davis conference. I was on there with someone from Cotality, someone from DocuSketch, and someone from Verisk. And we were presenting to various Paul Davis owners and attendees in that ecosystem about AI and restoration. And so in terms of main takeaways, I think the overall panel, all four of us were more aligned than not. And certainly, AI is becoming ubiquitous across software and restoration is no different. And so each of those vendors had an AI story. But also, was interesting to see where the differences were in both culture and opinion and how that reflected into the products that they were bringing to market. So tell me about where was everyone on the same page in terms of AI and technology? So a couple of funny moments in there as well of like, one of the questions the moderator had asked us was, should restorers be building their own AI? And everyone was empathetically, no. Building AI at scale is a hard problem, especially as you generalize across inputs the AI might have never seen before. Restoration is a liability heavy professional service. Doing this correctly requires things that general off the purpose shelf AI shouldn’t do. And by and large, building software systems is not in the wheelhouse of restoration companies. And so it’s largely a distraction. So all four people are well aligned on like, hey, don’t do that. But also, we’re biased because we’re software vendors. Yeah, of course. Was there anything where Encircle or your position on technology differed from the other folks on the panel? Absolutely. And I didn’t intend it to, but it kind of came off like I was throwing shade at some of the other vendors, but not my intention. But certainly, there was one of the questions or a couple of questions followed along a theme of that the other vendors answered it with, hey, AI can make mistakes, which it actually can. You are responsible for verifying that the AI is producing accurate, meaningful responses, but it can hallucinate and it can go off rails. So that’s on you. But I think the position I was trying to take is that we at Encircle care a lot about field documentation and the signals that it can get and producing reliable, robust results at scale. And also having the AI be able to cite back why it believes certain things and having multiple guardrails and defense in-depth preventing the AI from inventing details because once that cascade happens, like, the AI just built on itself and it creates a bit of a hallucination snowball. So I framed it then off the cuff as your job isn’t to beta test our software in the field. We’re working really hard to make sure that AI doesn’t do that. And I think some of the audience might have interpreted that as a dig, not my intention, but that’s one of the points of differentiation. Great. So this leads me into one of my next questions because you, as chief technology officer at Encircle, you’re the one the buck stops here making the architectural calls on what Encircle AI does and doesn’t do. So from an engineering perspective, where do you and your team draw the line on what AI should fully own versus what should stay with a human? And how does that decision actually get made as you’re building Encircle AI products? Totally. So a few ways to answer that. I’ll start directly on the question you asked, which is the like, where does AI live and where does the human shine? And I think there’s no replacing taste and judgment. And so there are certain aspects that an AI just can’t tell based on pixels alone. It doesn’t know that a dark carpet, whether it’s dark or if it’s wet. And it’s up to us building responsible AI systems to ensure that an AI doesn’t guess and start building a narrative that isn’t supported by the reality. So I absolutely believe that the human in the loop and the taste and judgment that goes into restoration is absolutely essential. Where AI belongs is in automating administrative and tedious tasks, in making sure being sort of a belt and suspender system to make sure that things aren’t being missed and that rules are being applied consistently. And it’s sort of a, hey, do you think about this? That you can kind of prompt the user into ensuring that everything is happening consistently in compliance, so on and so forth. So that’s really where AI is thriving and shining. Great. And from an engineering, you talked about how Encircle is sort of in the field documentation world. That’s where where we’ve we’ve been for fourteen years. But from an engineering, standpoint, where does field documentation or field data capture still fall short? And what does the what does Encircle AI or AI tools fundamentally need from the field that it might not be getting today? Like, what what more could we be feeding the the AI engine from from a field documentation standpoint? Oh, I mean so, of course, like, pictures, videos, voice, those are, like, what do I consider primary signals. And Encircle is a great tool for that because it’s it’s right there. It’s it’s sort of in the workflow with you in the field, And so we can capture those primary signals. But the other part that I think is equally important that really, really helps our AIs is all of the metadata that comes with that, timestamps and geolocations and other things that we also capture while you’re capturing those primary signals. We can use that to infer how long it’s been since the homeowner reported the loss to when you showed up on-site to the pictures that you started taking. And especially in mitigation, things can deteriorate over time and that time component is vital. And if you’re having disconnected systems, let’s say, where you’re capturing the photos in the system A and then uploading them in system B, those timestamps and other signals can often get missed or may not be fed into the AI model in a way. And context and data is king, And so AI really thrives on those secondary signals. And that’s where we shine with bringing AI into the field. Super interesting. You used the term hallucination snowball, I think, earlier. I’m gonna give that up in the moment, but No, that’s super interesting. And people talk about AI hallucinating, and like, what does that mean and all of that. So we know that AI in restoration is now influencing estimates and claim outcomes and claim and reviews. One of the things that we’ve talked about a lot at Encircle is accuracy, is prioritizing accuracy. So how do you, from an engineering standpoint, decide when an Encircle tool, an AI tool, is production ready? And how you stop it from confidently filling in gaps that it shouldn’t be doing, which we know a lot of generic LLMs are somewhat guilty of? Totally. In fact, that’s what these models are trained on, right? Like if we think about, without getting overly technical about it, the models are rewarded for filling in those gaps, for taking a question and producing a response regardless of how accurate or not accurate those responses. And then we reward it. And so they’re incentivized to produce a response. At Encircle, we’ve taken great care and spent a lot of effort in productionizing our AI evaluation system. So all of our products go through a very rigorous sort of definition of what good means and what kind of failure modes we see. And we build good classifiers and other things to detect those at scale while we’re building the actual product itself. So we’re building two things in parallel. We’re building a product that we serve our users, and then we’re building these guardrails that we can use to ensure that correct answers are always coming out or at a good enough rate that we could productionize. And then secondarily, we often build our products in layers because we found, and the general research will support this, that if an AI confidently gives you an answer, you can ask it to find citations or try and be adversarial and work backwards as well as forwards to make sure that it’s not producing a response that it’s made up. It will find out that something isn’t supported in primary documentation, and then we can add that as a data quality note and produce pass that downstream. It sounds like a justification for why you you and the other panelists said, don’t build this yourself. Absolutely. Yeah. Like, doing this properly for other people. I and I didn’t say this on the on the panel, but, like, I say this internally all the time. It’s one thing to use AI to for yourself to where you are the primarily the builder as well as the user. You can use AI and you know where the limitations are and you can make yourself go faster. It’s another where you’re the builder and someone else is a user, and there’s a lot of risk that comes along with that because you’re no longer in that loop. And so doing that properly requires real engineering. And if that’s not your primary forte, I would not tell you to go do that. Good. We’ve actually been hearing a lot lately about, in the insurance world, about fraud detection and how AI can generate documentation and even, you know, generate photos that are not real. So how do you or at Encircle, how do you ensure that the output that we’re producing holds up to scrutiny of carriers, auditors, and legal liability? Yeah. And so philosophically, would say that we will absolutely adopt authenticity measures where possible. But I do think that people that are not in circle are playing this cat and mouse game of trying to do detection after the fact. And, you know, OpenAI and Google and they’re working together to create fingerprints that can be, detected. But, really, it’s like there’s a lot there that that’s gonna go back and forth over time. I truly believe that that’s not a game that, on its own, solves the overall problem. Chain of custody is a very important thing. And so I go back to those secondary signals I talked about, timestamps, geolocates, what application version and operating system was running when the picture was captured, and was it captured in our app versus uploaded on a desktop computer that could have been altered. All of those, once they get into our systems and have this long sort of audit trail, we can provide that trail of authenticity that can help mitigate fraud as well. So I think there’s a it’s a it’s a nuanced complex story, and each of these is a piece of the puzzle So we’re thinking of it as field first and addressing it at the capture the source of capture rather than implementing measures to detect it That makes sense. So there’s a couple more questions, and then before we wrap up, there’s some real anxiety in the industry about what AI is going to do and how it’s going to change things. What does it look like when it’s actually making the best people better instead of replacing your cheapest labor, I guess, if that makes sense? Yeah, or like, I think the crux of your question is like, there are people that are deploying AI sort of as a bolt on of, hey. Here is this capability. I’ve heard about it. Like, let’s maybe use it here and there. And then there’s people that are, like, investing in their best people and saying, hey, how can I get you to do more of the things only you can do and take away the rest? So for restoration contractors, it’s like, how can I get you doing more restoration, more time with the customer, more things that are unique to that job rather than texting people or coordinating schedules or all of the other administrative, making reports, other things? So that’s sort of how I would answer that question. So is it it’s a paradigm a shift from this paradigm of, like, we want people doing more with less. It’s not it’s not that. It’s we want we want more with more almost. Yeah. Or we want to enable our best to do more of their best. Right. By doing less of the stuff that they don’t need to be doing. Exactly. Yeah. That makes sense. Okay. Okay. Last question, and then we’ll wrap up. I know I personally like, AI is everywhere right now, and AI anxiety or AI fatigue or whatever we wanna call it. What are you seeing real ROI with right now in terms of AI? And if you look ahead twenty four months, what’s going to separate specifically in restoration, what’s going to separate the high performing restoration shops from everyone else? Yeah. It’s similar to the previous question. Think it’s like we know AI is going to become ubiquitous. Everyone’s going to claim that they’re using AI. But I think that what’s going to separate the successful ones from the non successful ones are the ones that have truly been able to integrate AI in ways that helps their best people perform the best, do more in the things that are unique to them, apply their taste and judgment and character to their job, and let AI take care of the rest, whether it be compliance or other administrative details. Great. Last question, and then we’re gonna wrap, I promise. This is a question for me. What everybody’s talking about becoming AI native. What does that mean? Oh, I’m yeah. Okay. So AI AI native means different things to different people. To me, it means it’s a paradigm shift. So okay. This could this could get very deep, very fast. I’ll keep try and keep it simple. When systems change, when the core assumptions of how you’ve operated your business or how you go about your work, when you’re challenging those with AI and taking that lens to it and asking yourself, all of these things that used to be hard, that I believed used to be hard, I can now use AI for and focus on these other things. That’s a paradigm shift. It’s one of the biggest leverage points you can make in any sort of system. That’s where I think AI native starts to be defined. So you’re taking that lens to the core workflow of whatever problem you’re trying to solve or whatever business that you’re in and then rebuilding it on top of that. Great. Thank you so much, Ronuk, for sharing your knowledge and wisdom. And I’m sure that everybody on that panel and the audience was super appreciative of hearing from you as well. That was a great panel. All right. Thanks so much, Ronuk. Thank you. Bye.