RIMScast: Genuine Generative AI Talk with Tom Wilde of Indico Data

Genuine Generative AI Talk with Tom Wilde of Indico Data

Jul 18, 2023

Welcome to RIMScast. Your host is Justin Smulison, Business Content Manager at RIMS, the Risk and Insurance Management Society.

How does Generative AI fit into modern business practices? What are the best practices before any implementation of automation and AI? What are the benefits and pitfalls of ChatGPT and other AI tools? Tom Wilde, CEO of Indico Data, discusses his experiences with risk professionals and how risk managers are in an advantageous situation. Tom gives examples of blindspots and risks with ChatGPT. He shares his thoughts on the trend of CROs becoming CEOs.

Key Takeaways:

[:01] About RIMScast.

[:27] About today’s episode, where we will talk about technological innovation and risk with Tom Wilde, the CEO of Indico Data.

[:36] All about exciting, upcoming RIMS events! Registration is open for the RIMS Canada Conference 2023, which will be held September 11th–14th in Ottawa! Visit RIMSCanadaConference.ca for more information.

[:57] On September 14th, the Spencer Educational Foundation returns to New York City for its Annual Funding Their Future Gala. The event will be held at the Cipriani on 42nd Street. A link is on this episode’s notes. You can also visit SpencerEd.org.

[1:14] The RIMS Western Regional Conference will be held October 4th–6th in Vail, Colorado. Visit RIMSWesternRegional.com for more information and to register.

[1:26] Head to the RIMS.org/Advocacy page to find information about The RIMS Legislative Summit, which is returning to Washington, D.C. on October 25th and 26th.

[1:40] We are very excited about the RIMS ERM Conference 2023, which will be held November 2nd and 3rd in Denver, Colorado! The theme is Elevate and Evolve. Registration will open soon as will a call for nominations for the ERM Award of Distinction. Visit the events page on RIMS.org for more information.

[2:06] The prominence of ChatGPT and AI in business has all sorts of upside and downside risks. Here to discuss them and lend his insight is Tom Wilde, the CEO of Indico Data. He is based in Boston, Massachusetts.

[2:21] “Has the future arrived?”
[2:30] Justin welcomes Tom Wilde of Indico Data to RIMScast. Tom has been in the software space for his entire career, beginning on the consumer internet side and then moving into enterprise software, focused on content technology.

[3:18] Tom has been fascinated with trying to sort out the human-machine challenge between the way we produce and consume digital content and the way machines make sense of it. That’s been a common thread through all the businesses he has built.

[4:01] Tom tells about being a CEO. Tom is a builder CEO. He finds disruptive technologies and turns them into something the market finds useful and valuable.

[4:52] The building blocks for ChatGPT, Deep Learning, Large Language Models, and Generative AI, have been in development for the past decade. Everything suddenly came together at the right time to create this “iPhone moment” for artificial intelligence where we all can see the value.

[5:40] The invisible tipping point was that the Large Language Model underpinning ChatGPT, GPT 3 (now GPT 4), had reached a critical mass and a set of performance capabilities that suddenly made it viable.

[6:28] We’re in a classic phase of ChatGPT now, with incredible expectations and excitement about it. This is a moment as big as a browser or smartphone. This is a fundamentally new software programming paradigm. It’s not a Swiss Army knife application that can solve everything.

[7:10] For the first time, we have a programming language that is based on data and that uses the English language as the programming interface. There are about 25,000,000 software engineers in the world. There are several billion English language speakers on Earth. Suddenly, all of them, conceivably, can program using ChatGPT.

[8:23] ChatGPT introduces some challenges to the enterprise, such as having to hold employees back from pushing code directly into production.

[8:52] There suddenly needs to be some thought and structure around who can create prompts, what data can be fed to these cloud Large Language models, and what is the audit trail for what prompt was used to create what outcome.

[9:09] Using software development paradigms makes sense in how to govern this and deal with compliance. Regarding Generative AI, while we have the huge benefit of programming with data, we have to be aware of what that means.

[9:28] The quality of data becomes important in understanding how this new programming language behaves. It’s only as good as the data it was trained on. If you take all the English-language content on the internet, what does that represent in terms of our understanding of human knowledge and behavior?

[10:03] The English-language content on the internet carries flaws and biases. It has inappropriate and dangerous content. We have to be aware of what this programming language was taught to do and think about.

[10:24] You should not use this technology as a black box. Microsoft suggests thinking about this as a co-pilot. The pilot flies the plane and has the final judgment as to what is safe and unsafe operation. The co-pilot reduces the workload as an extra set of hands and works the radio. ChatGPT is a co-pilot, not a pilot.

[11:13] You have to have application frameworks in place to deal with hallucinations, to understand what data was used to make that decision at what time, who has been empowered to use this technology to perform part of their job, and what are the guardrails for that. Where can the enterprise’s data be sent and stored?

[11:45] Indico Data considers the three S’s: Safely, Securely, and Scalably. To use this technology, you need a framework for understanding the three S’s.

[11:58] RIMS plug time! Sponsor an episode of RIMScast! Contact us at pd@rims.org. For upcoming virtual workshops visit RIMS.org/virtualworkshops for the calendar. Applying and Integrating ERM will be held on August 1st and 2nd by Ken Baker of Enterprise Risk Consulting.

[12:42] Optimizing Risk Management with Artificial Intelligence will be led on September 28th by Pat Saporito. More information about these sessions and others is on the Virtual Workshops page. See the full schedule and register.

[13:02] The RIMS-CRMP Exam Prep is July 19th and 20th, 9:00 am–4:00 pm EDT. August 15th through 17th is the RIMS-CRMP-FED Exam Prep 9:00 am–4:00 pm EDT. See the links at Rims.org/certification for these and future workshops. A link is also in this episode’s show notes, as is a link to the full virtual workshop calendar.

[13:57] As a technology CEO, you have to learn to think through the way the enterprise thinks of risk because they’re not going to buy or implement your solution if it creates unacceptable risk, regardless of the benefit.

[14:22] As a technology CEO, Tom thinks a lot about how to minimize the risks of deploying their product from a data protection standpoint, information security standpoint, and change management standpoint. Those things have to be top-of-mind or you can’t engage with the enterprise. They are risk-averse, as they should be.

[14:49] In many cases, you’re dealing with PII, health data, and other sensitive information, so there are serious data protection requirements. GDPR and regional or national regulations on data security create additional layers of thinking to protect your client from being exposed to regulatory bodies.

[15:17] Insurance and banking risk managers are using Indico Data to help them identify and quantify risk. Auditors and risk managers try to consume and make sense of vast amounts of unstructured data, like contracts, policies, documents, and emails, to find the needles in the haystack. What are patterns that suggest risky behavior or exposure?

[16:02] Tom has seen a huge uptick in risk management wanting to embrace technology as a “bionic arm,” not as a robot, to allow a much faster and broader perspective on potential threat vectors and exposures.

[16:53] Tom tells of a court case where attorneys used ChatGPT to create prior case representation for the court but ChatGPT fabricated the cases. It didn’t understand it wasn’t supposed to make them up. There’s a gap between the way AI and humans work. AI has blind spots and until we find them and fix them, they expose us to risk.

[18:12] If you’re using AI to make decisions, how are you sure that those are quality decisions, that they’re not fabricated or hallucinated, and that they’re complete? Generative AI struggles with precision and recall. Precision means that all the answers it gives are correct. Recall is that it found all the possible data it should have presented.

[18:51] Generative AI will make recall mistakes and leave stuff out, not knowing that it’s supposed to be comprehensive. When you ask ChatGPT why it did not include something in its summary, it says, “Sorry, you’re right, I should have included that.” We need to continue to be suspicious of Generative AI’s confidence in the answers it brings.

[20:03] The challenge with automation and AI is it allows you to make a bad process even worse, more rapidly. Ask yourself if you are automating a good process. Is it a process that should be automated or is the process itself the problem? Like using a power saw, you need to know what you’re doing with ChatGPT. It’s a very powerful tool.

[21:41] A CEO is a storyteller. To be a technology CEO, with the pace of innovation in the market today, your customers crave for you to help them make sense of it and think about it rationally. They’re desperate to do that for their constituents. You become a valuable partner CEO if your product helps them make sense of things and adds value.

[22:29] Your storytelling also helps your employees and your investors. Your job as an early-stage CEO is to create that narrative and bring it to light.

[23:00] Justin and Tom discuss the trend of CROs becoming CEOs. Being good at understanding risk makes you good at taking bets. As a CEO, you have a key responsibility to make bets where you have incomplete information, understanding the downside and the upside.

[23:27] Risk professionals have an advantage in being able to measure the risks in front of them and make bets more confidently and with perhaps better outcomes.

[23:42] We are in a world of high beta factors: climate, economic, and social, that have to be understood so that you don’t inadvertently bet the whole company. Understanding these factors also puts you in a position to take advantage of opportunities that emerge from the macro elements surrounding us as business leaders.

[24:15] “Special thanks to Tom Wilde for joining me today. That was a lot of fun! I’ve got more links to RIMS coverage of AI and automation on this episode’s show notes.”

[24:27] You can sponsor a RIMScast episode for this, our weekly show, or a dedicated episode. Links to sponsored episodes are on our show notes. RIMScast has a global audience of professionals, students, business leaders, and more. Contact pd@rims.org for more information.

[25:13] Become a RIMS member and get access to the tools, thought leadership, and network you need to succeed. Visit RIMS.org/membership or email membershipdept@RIMS.org for more information. The RIMS app is available only for RIMS members! You can find it in the App Store.

[25:37] Risk Knowledge is the RIMS searchable content library that provides relevant information for today’s risk professionals. Materials include RIMS executive reports, survey findings, contributed articles, industry research, benchmarking data, and more.

[25:53] For the best reporting on the profession of risk management, read Risk Management Magazine at RMMagazine.com and check out the blog at RiskManagementMonitor.com. Justin Smulison is Business Content Manager. You can email Justin at Content@RIMS.org.

[26:16] Justin thanks you for your continued support and engagement on social media channels! Listen every week! Stay safe!

Mentioned in this Episode:

NEW FOR MEMBERS! RIMS Mobile App

RIMS ERM Conference 2023.

RIMS Canada 2023 — Sept. 11–14 in Ottawa!

Spencer Educational Foundation — Funding Their Future Gala — Sept. 14, 2023

Contribute to Risk Management Magazine
RIMS Western Regional — Oct 4–6, Vail Colorado

RIMS-Certified Risk Management Professional (RIMS-CRMP)

Dan Kugler Risk Manager on Campus Grant
Upcoming Virtual Workshops:

Applying and Integrating ERM | August 1

Optimizing Risk Management with AI | Sept. 28

See the full calendar of RIMS Virtual Workshops