Summary
Highlights
The panel is introduced, and each member shares their background and how they got involved in the field of AI compliance and ethical AI. Luke Vain of UBS discusses his transition from governance to data ethics, highlighting the growing importance of the field. Paul, with a PhD in AI from the '90s, recounts his return to AI due to the ethical risks and now leads data and AI ethics at Lloyds. Carol from a bank's data and analytics team explains her journey through privacy functions, GDPR, and the realization of the dynamic regulatory landscape, emphasizing the convergence of various laws like the EU AI Act and the Equality Act.
Carol elaborates on how data protection laws, primarily GDPR and the UK Data Protection Act, significantly impact AI development and deployment. She lists numerous overlapping regulations including the ePrivacy regulation, Consumer Duty, and the EU AI Act, emphasizing that 100% compliance with all of them is virtually impossible, pushing organizations into an ethical space. She highlights Article 5 of GDPR, focusing on lawfulness, fairness, and transparency, and how privacy teams are now expected to cover broader areas. Fairness metrics and the challenge of using special category data for bias detection are discussed, noting that the EU AI Act mandates this. The importance of data quality, control over proprietary information in generative AI prompts, and the need for self-regulation are stressed, advocating for an organizational culture of good data behaviors.
Paul discusses how financial institutions can proactively detect and address algorithmic discrimination. He explains that fairness is a contested concept with no universal definition, and statistical fairness can lead to irresponsible practices, such as lending to ineligible individuals. He advocates for analyzing the trade-off between statistical fairness and model accuracy, bringing these considerations to ethics councils for human-led decision-making. Paul also touches on bias, noting that some biases are reasonable with proper business cases and consumer duty adherence. He emphasizes the role of AI ethics councils in addressing ethical dilemmas that go beyond legal compliance, fostering 'ethical muscles' within organizations through discussions about core values.
Luke discusses effective communication of AI systems to stakeholders. He references a legal case against Experian regarding opaque data privacy notices, suggesting that companies should be more upfront about surprising details of data usage. He argues that trust is paramount, and companies that invest in clear, timely explanations of AI workings will gain a competitive advantage. Paul adds that demonstrating accountability involves people, process, and technology. He advocates for corporate-wide ethics training, improving model validation processes, and building ethical considerations directly into the machine learning development pipeline to prevent issues and adjust system behavior proactively. Carol discusses liability for AI failures, emphasizing that existing legal frameworks like contract and tort law are largely sufficient. She expresses concern about the concept of granting legal personhood to AI, citing Joanna Bryson's view that the costs outweigh the moral gains. Paul strongly asserts that accountability for AI systems must rest with the deploying company, not individual developers, holding boards responsible for balancing shareholder value with customer care and safety. He highlights the importance of diversity in tech teams and practices like 'consequence scanning' to identify and mitigate risks from the outset.
Luke addresses the contingency plans for generative AI potentially using stolen or copyrighted material. He observes the surge in legal cases against AI developers for copyright infringement, particularly for using copyrighted literature without compensation. He discusses the legal argument of 'transformation' of data into model weights and the difficulty of proving it given AI's ability to reproduce specific content. Luke speculates that capitalism will likely prioritize economic growth over strict copyright enforcement, but warns of 'model collapse' and the need for robust backup plans if AI quality deteriorates. Paul adds concern about synthetic data leading to a narrower distribution of outputs and homogenized results. Both emphasize the ethical responsibility of tech giants to address these issues, such as watermarking AI-generated content, noting the profound societal impact on fields like policing and the judicial system due to questionable evidentiary reliability. The panel then debates whether ethical considerations cost competitive advantage. Luke argues against this, referencing the Apple and Goldman Sachs credit card case where reputation was damaged despite legal vindication. He asserts that trust is everything in banking and that ethical practices often lead to increased model resilience and robustness, ultimately benefiting profits. Paul agrees that ethics, safety, and ESG are integral for an organization's survival and profitability, stating that it's not a binary choice but a strategic imperative for boards to decide where to draw the line.
An audience member asks about the legal personhood of AI, drawing parallels to corporations. Carol reiterates her preference for the current legal system, believing that the company behind the AI should remain liable, though she acknowledges the evolving societal view on machine status. Luke ties this to the historical shift in liability from individuals to corporations, which fostered innovation, but questions the benefit of extending liability to AI itself, as AI lacks human autonomy and accountability. Another audience member raises concerns about AI hallucination in reports and who is responsible for errors when users know the risks. Carol stresses internal controls, advising users to apply AI only in areas where they can vet the output, citing a judge who used ChatGPT but could verify its legal reasoning. She warns against 'mental muscle' loss from over-reliance. Luke offers a philosophical closing thought, comparing AI's reduction of creative costs to the Industrial Revolution's impact on labor. He questions whether society truly desires the automation of all work and creativity, pointing out that decisions shaping humanity's future are often made by a narrow group without democratic input, urging for collective action to guide technology's direction.