Shows
Algorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingGuest 1. Ryan Carrier: Executive Director of ForHumanityMitigating AI Risks Ryan Carrier is founder and executive director of ForHumanity, a non-profit focused on mitigating the risks associated with AI, autonomous, and algorithmic systems. With 25 years of experience in financial services, Ryan discusses ForHumanity's mission to analyze and mitigate the downside risks of AI to benefit society. The conversation includes insights on the foundation of ForHumanity, the role of independent AI audits, educational programs offered by the ForHumanity AI Education and Training Center, AI governance, and the development of audit certification schemes. Ryan a...
2025-01-2044 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 15. Algorithm Integrity Documentation - Getting StartedSpoken (by a human) version of this article.Documentation makes it easier to consistently maintain algorithm integrity.This is well known.But there are lots of types of documents to prepare, and often the first hurdle is just thinking about where to start.So this simple guide is meant to help do exactly that – get going.About this podcastA podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla.
2024-11-1905 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 14. External data - use with careSpoken (by a human) version of this article.Banks and insurers are increasingly using external data; using them beyond their intended purpose can be risky (e.g. discriminatory).Emerging regulations and regulatory guidance emphasise the need for active oversight by boards, senior management to ensure responsible use of external data.Keeping the customer top of mind, asking the right questions, and focusing on the intended purpose of the data, can help reduce the risk.Law and guideline mentioned in the article: Colorado's External Consumer Data and Information Sources (ECDIS) law Ne...
2024-11-1206 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 13. Bridging the purpose-risk gap: Customer-first algorithmic risk assessmentsSpoken (by a human) version of this article.Banks and insurers sometimes lose sight of their customer-centric purpose when assessing AI/algorithm risks, focusing instead on regular business risks and regulatory concerns.Regulators are noticing this disconnect.This article aims to outline why the disconnect happens and how we can fix it.Report mentioned in the article: ASIC, REP 798 Beware the gap: Governance arrangements in the face of AI innovation.About this podcastA podcast for Financial Services leaders, where we discuss fairness and accuracy...
2024-11-0507 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 12. Risk-Focused Principles for Change Control in Algorithmic SystemsSpoken (by a human) version of this article.With algorithmic systems, an change can trigger a cascade of unintended consequences, potentially compromising fairness, accountability, and public trust.So, managing changes is important. But if you use the wrong framework, your change control process may tick the boxes, but be both ineffective and inefficient.This article outlines a potential solution: a risk focused, principles-based approach to change control for algorithmic systems.Resource mentioned in the article: ISA 315 guideline for general IT controls.About this podcastA...
2024-10-2912 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 11. Deprovisioning User Access to Maintain Algorithm IntegritySpoken (by a human) version of this article.The integrity of algorithmic systems goes beyond accuracy and fairness.In Episode 4, we outlined 10 key aspects of algorithm integrity.Number 5 in that list (not in order of importance) is Security: the algorithmic system needs to be protected from unauthorised access, manipulation and exploitation.In this episode, we explore one important sub-component of this: deprovisioning user access.Link from article: U.S. National Coordinator for Critical Infrastructure Security and Resilience (CISA) advisory. About this podcastA p...
2024-10-2209 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 10. Fairness reviews: identifying essential attributesSpoken (by a human) version of this article.When we're checking for fairness in our algorithmic systems (incl. processes, models, rules), we often ask:What are the personal characteristics or attributes that, if used, could lead to discrimination?This article provides a basic framework for identifying and categorising these attributes.About this podcastA podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla.Produced by Risk Insights (riskinsights.com...
2024-10-1506 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 9. Algorithmic Integrity: Don't wait for legislationSpoken (by a human) version of this article. Legislation isn't the silver bullet for algorithmic integrity. Are they useful? Sure. They help provide clarity and can reduce ambiguity. And once a law is passed, we must comply. However:existing legislation may already applynew algorithm-focused laws can be too narrow or quickly outdatedstandards can be confusing, and may not cover what we need"best practice" frameworks help, but they're not always the best (and there are several, so they can't all be "best").In short, they are helpful.But we ne...
2024-10-0810 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 8. A Balanced Focus on New and Established AlgorithmsSpoken (by a human) version of this article.Even in discussions among AI governance professionals, there seems to be a silent “gen” before AI.With rapid progress - or rather prominence – of generative AI capabilities, these have taken centre stage.Amidst this excitement, we mustn't lose sight of the established algorithms and data-enabled workflows driving core business decisions. These range from simple rules-based systems to complex machine learning models, each playing a crucial role in our operations.In this episode, we'll examine why we need to keep an eye on established algorit...
2024-10-0108 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 7. Postcodes: Hidden Proxies for Protected AttributesSpoken (by a human) version of this article.In a previous article, we discussed algorithmic fairness, and how seemingly neutral data points can become proxies for protected attributes.In this article, we'll explore a concrete example of a proxy used in insurance and banking algorithms: postcodes.We've used Australian terminology and data. But the concept will apply to most countries. Using Australian Bureau of Statistics (ABS) Census data, it aims to demonstrate how postcodes can serve as hidden proxies for gender, disability status and citizenship.About this podcast
2024-09-2411 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 6. Balancing Security and Access for increased algorithmic integritySpoken (by a human) version of this article.When we talk about security in algorithmic systems, it's easy to focus solely on keeping the bad guys out.But there's another side to this coin that's just as important: making sure the right people can get in.This article aims to explain how security and access work together for better algorithm integrity.About this podcastA podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted...
2024-09-1705 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 5. Equal vs Equitable: Algorithmic FairnessSpoken (by a human) version of this article.Fairness in algorithmic systems is a multi-faceted, and developing, topic.In episode 4, we explored ten key aspects to consider when scoping an algorithm integrity audit.One aspect was fairness, with this in the description: "...The design ensures equitable treatment..."This raises an important question. Shouldn't we aim for equal, rather than equitable treatment?This episode aims to shed light on the distinctions between equal and equitable treatment in algorithmic systems, while acknowledging that our understanding of fairness is still...
2024-09-1013 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 4. Structuring the Audit Objective: 10 Key Aspects of Algorithm IntegritySpoken (by a human) version of this article.In Episode 1, we explored the challenges of placing undue reliance on audits.One potential solution that we outlined is a clear scope, particularly regarding the audit objective.In this episode, we focus on algorithm integrity as the broad audit objective.While it’s easy to assert that an algorithm has integrity, confirming this assertion is a bit more complex.To help simplify this, this episode breaks it down into a set of key areas to consider.About this po...
2024-09-0312 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 3. Navigate Algorithm Audit Guidance: some aren't relevant to your contextSpoken (by a human) version of this article.AI and algorithm audits help ensure ethical and accurate data processing, preventing harm and disadvantage.However, the guidelines are not yet mature, and quite disparate.This can make the audit process confusing, and quite daunting - how do you wade through it all to find the information that you need, in deciding how to commission your audit?Fortunately, there is a solution - narrowing the guidelines down, based on relevance.Not all existing guidelines are universally applicable. This can vary based...
2024-08-2708 min
Auditing with data: for Performance Auditors and Internal Auditors that use (or want to use) data58. The final episodeThis is the final episode of this podcast. It includes:an explanation - why the podcast is coming to a close.snippets from a selection of prior episodesa brief introduction to Algorithm Integrity Matters - the new Risk Insights podcast.About this podcastThe podcast for performance auditors and internal auditors that use (or want to use) data. Hosted by Conor McGarrity and Yusuf Moolla. Produced by Risk Insights (riskinsights.com.au).
2024-08-2744 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 2. Choice vs obligation: motivation shapes the effectiveness of your reviewSpoken (by a human) version of this article.The motivation(s) for commissioning a review can determine how effective it will be.Consider a personal health check-up:Sometimes we undergo medical check-ups because we don’t have a choice. We need to - for example for workplace requirements or for insurance.At other times, we choose to undergo such check-ups. We want to maintain optimal health and catch any potential issues early.Often, our approach differs depending on whether we are forced to, or choose to.The mo...
2024-08-2006 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processingArticle 1. How reliable is the algorithm review that you have commissioned?Spoken (by a human) version of this article.One common issue with audits is undue reliance.Can you rely on the audit report to tell you what you need to know?Could you be relying on it too much?https://riskinsights.com.au/blog/reliable-auditsAbout this podcastA podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla.Produced by Risk Insights...
2024-08-1310 minAlgorithm Integrity Matters: for Financial Services leaders, to enhance fairness and accuracy in data processing0. IntroductionA brief intro to the podcast.If you have suggestions for topics you'd like me to cover, feel free to reach out to me via email. yusuf@riskinsights.com.auAbout this podcastA podcast for Financial Services leaders, where we discuss fairness and accuracy in the use of data, algorithms, and AI. Hosted by Yusuf Moolla.Produced by Risk Insights (riskinsights.com.au).
2024-08-0601 minAuditing with data: for Performance Auditors and Internal Auditors that use (or want to use) data57. Michael Drechnowicz, Head of Internal Audit, TK ElevatorEmbracing Internal Audit Evolution: A Conversation with Michael Drechnowicz, Head of Internal Audit at TK ElevatorIn this episode, Michael shares his insights on the evolution of IA in his role as Head of Internal Audit with a major player in the global elevator and escalator industry. His work demonstrates a shift from traditional finance-focus towards extensive business process coverage, ranging from sales to purchasing and fraud investigations. We discuss the increasing importance of diversity in global audit teams and how technology, particularly Artificial Intelligence (AI) an...
2023-12-1335 min
The Performance Audit Report25 | NYC Deputy Comptroller for Audit - Maura Hayes-ChaffeMaura Hayes-Chaffe is the Deputy Comptroller for Audit in New York City.Maura joined us to discuss the Bureau of Audit, including:the challenges with recruiting and attracting performance auditorsnew approaches to conducting PAsthe changing environment (e.g., economics)equity in PA, citizen powered audits, and a recent housing related auditthe use of data, including open data, by the audit team.LinksNYC Office of the ComptrollerThis episode's web pageAbout this podcast The podcast for performance auditors. Hosted by Conor McGarrity and Y...
2023-03-0735 min
The Performance Audit Report24 | Cathy O'Neil, Algorithm Auditor, Author of WMathDOur guest is Cathy O'Neil, author of Weapons of Math Destruction: how big data increases inequality and threatens democracy.Cathy joined us to discuss critical aspects for auditors to consider, including:The human element: why understanding 'for whom an algorithm might fail' is more important than the detailed technical design of an algorithm.Algorithm testing: the importance of post-deployment testing to ensure there are no unintended adverse outcomes. Setting standards: the need for industry-wide standards so that regulators can ensure there is no bias or discrimination in how algorithms operate. Laggers and leaders: the four big in...
2023-02-2844 minAuditing with data: for Performance Auditors and Internal Auditors that use (or want to use) data56. Cathy O'Neil, Algorithm Auditor, Author of WMathDOur guest is Cathy O'Neil, author of Weapons of Math Destruction: how big data increases inequality and threatens democracy.Cathy joined us to discuss critical aspects for auditors to consider, including:The human element: why understanding 'for whom an algorithm might fail' is more important than the detailed technical design of an algorithm.Algorithm testing: the importance of post-deployment testing to ensure there are no unintended adverse outcomes. Setting standards: the need for industry-wide standards so that regulators can ensure there is no bias or discrimination in how algorithms operate. Laggers and leaders: the four big in...
2023-02-2844 min
The Performance Audit Report23 | North Dakota Audit Office and a vaccine PAThis episode featured guests from the North Dakota State Audit Office:Allison Bader, Performance Audit Manager and Claudia Kuball, Performance Auditor.In this episode we discuss:Team diversity – the value of recruiting performance auditors from different backgrounds, incl. how specialist expertise can help focus on major risksImpact – their communications team is instrumental in engaging citizens and maximising audit impactSignificant audit – their audit of Storage and Handling of COVID Vaccines including its design, execution and findingsCitizen engagement – how North Dakotans can get involved, and even speak to the State Auditor directly on radio every Wed morning
2023-02-0730 min
The Performance Audit Report22 | Engaging citizens in performance auditingIn this episode, we discuss how audit offices are engaging citizens in performance audits. The show notes here and the full article here contain further details and examples.We delve into how audit offices are engaging citizens:Forward work plans – consulting broadly for focus areasRefining the broad focus areas for specific citizen concerns Audit planning – citizen input on audit scope Audit execution – seeking agency staff views during fieldwork Reporting – tailoring reports for citizens and civil society organisations Links: This episode's web page with further details and examples - https://pareports.com/podcast/22 Th...
2023-01-2411 min
The Performance Audit Report21 | Planning with dataThis originally aired as episode 50 of The Assurance Show.In this episode, we discuss the benefits of adopting a data focused approach during audit planning.Link: This episode's web page - https://pareports.com/podcast/21About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices from audit offices.
2023-01-1018 min
The Performance Audit Report20 | Work plan balanceThis originally aired as episode 49 of The Assurance Show.In this episode, we discuss the importance of balance in your work program - covering each of the four objectives:EffectivenessEfficiencyEconomyComplianceLink: This episode's web page - https://pareports.com/podcast/20About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly newsletter with a selection of PA reports and practices f...
2022-12-1316 min
The Performance Audit Report19 | Forward Work PlanningThis originally aired as episode 48 of The Assurance Show.In this episode, we discuss strategic audit planning (a.k.a. forward work planning).Understanding your community / public expectationsUnderstanding your elected body / parliamentariansUnderstanding the entities within your audit office's jurisdictionLink: This episode's web page - https://pareports.com/podcast/19About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Performance Audit Research.Get updates New episodes, and a fortnightly n...
2022-11-2916 min
The Performance Audit Report18 | Michael Pickup, Auditor-General of British Columbia (Canada)Michael Pickup is the Auditor General of British Columbia in Canada.In this episode we discuss:the changing role of performance auditrecruitment and retentionworkplace flexibilitypeople and product focused innovationthe use of video and other media to increase reachLinks:Michael Pickup on LinkedIn - https://www.linkedin.com/in/michael-pickup-fcpa-fca-a1ab91168/ Auditor General of British Columbia - https://www.bcauditor.com/ This episode's web page - https://pareports.com/podcast/18 About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Prod...
2022-11-2244 min
The Performance Audit Report17 | Four QA Steps - data in auditThis originally aired as episode 39 of The Assurance Show.In using data for audits, quality assurance of our work is important.In this episode, we discuss: 1. The four main QA stepsTechnical QA Functional QA Business QA (validation) Audit QA 2. Why we need a QA PlanLink: This episode's web page - https://pareports.com/podcast/17About this podcast The podcast for performance auditors.Hosted by Conor McGarrity and Yusuf Moolla.Produced by PA Reports - Perform...
2022-11-1510 min