If the machine helps make decisions, who is responsible?

Ethics and Compliance Switzerland is hosting a conference in Zurich on October 21 and 22, 2026. Following “Third-Party Risks 2022” and “Future Trends 2024,” this year’s conference will focus on […]


↧
According to Patrick Wellens, a board of directors must be prepared to draw the line between what AI is allowed to decide on our behalf and what it is not.

According to Patrick Wellens, a board of directors must be prepared to draw the line between what AI is allowed to decide on our behalf and what it is not.

According to Patrick Wellens, a board of directors must be prepared to draw the line between what AI is allowed to decide on our behalf and what it is not.

Ethics and Compliance Switzerland is hosting a conference in Zurich on October 21 and 22, 2026. Following “Third-Party Risks 2022” and “Future Trends 2024,” this year’s conference will focus on ethics and artificial intelligence. ECS President Patrick Wellens explains why compliance cannot be delegated to algorithms.

Since 2014, Ethics and Compliance Switzerland (ECS) has been building a cross-industry compliance community in Switzerland. The Basel-based nonprofit association focuses on working groups, roundtables, and exchanges among practitioners rather than glossy brochures. Members come from industry, academia, consulting, and law firms, as well as, to a lesser extent, the financial services sector. The flagship event is the biennial conference in Zurich, held at the Sheraton Zurich Hotel.

Patrick Wellens brings over 30 years of experience in governance, risk, and compliance, primarily in the pharmaceutical, consumer goods, and industrial sectors. He served as Country Compliance Officer, Data Protection Officer, and Global Divisional Compliance Officer at a pharmaceutical company in Basel. He also served as a senior manager in the forensic division of one of the “Big Four” firms, where he supported executive boards, general counsels, and chief compliance officers with services related to compliance, fraud prevention, detection, and investigation. He is a member of the editorial advisory board of the Society for Corporate Compliance and Ethics and of the editorial advisory board of the magazine “Risk & Compliance,” a board member of the Association of Corporate Investigators, a guest lecturer at numerous universities, and chairman of the board of Ethics & Compliance Switzerland.

Why is this topic also of interest to the insurance industry?

Although most members come from the industry, issues such as fairness, transparency, and insurer liability are also moving from the seminar room into day-to-day business. This is exactly where the conference comes in.

Mr. Wellens, after discussing third-party risks in 2022 and future trends in 2024, the focus is now on ethics and AI. Why now, in particular?

AI is ubiquitous and is fundamentally transforming entire industries, and the compliance sector is no exception. But the opportunities opened up by AI come with profound ethical and governance challenges: How can we ensure accountability, transparency, fairness, data protection, and responsible decision-making when increasingly important decisions are supported by, or even made entirely by, algorithms?

Policymakers and regulatory agencies are imposing an ever-increasing number of regulations on companies. To meet these new requirements, additional resources are needed in the area of compliance. At the same time, many board members and executives believe that ever-rising compliance costs are not the solution. Their response is to focus on integrity, ethics, and personal accountability rather than on further expanding compliance resources. Whether this approach is effective in ensuring regulatory compliance is another question.

What should participants see differently on the evening of the second day compared to the morning of the first?

After the conference, participants should have a better understanding of how the compliance function remains effective despite geopolitical and technological upheavals (AI), how AI can be better utilized to support the compliance function, and which factors and elements must be taken into account in AI governance, but also how companies can create an environment in which integrity is chosen of one’s own free will rather than being imposed from the outside.

Which panel or presentation should you definitely not miss, and why?

Different companies have varying levels of maturity in the area of compliance, different resources, and varying degrees of AI integration; therefore, I don’t want to single out any specific presentation or panel. Many of the presentations and panels are highly relevant.

Many companies are using AI before they have rules in place for it. Is compliance a brake or an airbag in this context?

When companies implement AI without rules, they’re like driving a car without seat belts or brakes on a foggy highway. It may feel fast at first, but the first unforeseen obstacle, such as data breaches, copyright lawsuits, or reputational damage, will result in a total loss. Compliance acts as an airbag because it mitigates risks and ensures legal certainty, reputation protection, and IP protection.

AI regulations (such as the EU AI Act) are not a nuisance, but a legal reality. Compliance protects against fines that could threaten a company’s very existence. If an AI makes racist or erroneous decisions, a sound governance structure mitigates the damage before it destroys the brand. Compliance is the airbag that saves the company from ruin when mistakes are made. Compliance doesn’t slow down innovation, it mitigates risk.

If an algorithm rejects a claim or misclassifies a customer, who bears the ethical and legal responsibility?

In the case of erroneous decisions made by an algorithm—such as the wrongful rejection of an insurance claim or an incorrect customer classification—the ethical and legal responsibility lies with the company using the AI, not with the algorithm itself.

The law always seeks to hold a natural or legal person accountable. An algorithm is not a legal entity and cannot be sued. The company that deploys it is liable to the customer. Executives have a duty of care and oversight. If they deploy AI systems to customers without sufficient quality control and human oversight (human-in-the-loop), they may be acting with organizational negligence.

If the algorithm systematically discriminates against certain customer groups (e.g., based on place of residence, age, or gender), the company has an ethical obligation to clean up the training data. Companies are ethically, and increasingly legally, obligated to disclose why an AI made a particular decision. A non-transparent “black-box system,” in which no one can understand how a rejection came about, is ethically unacceptable.

AI can uncover corruption and antitrust risks, but it can also conceal them. Where do you see the greatest opportunities, and where are the blind spots?

When it comes to detecting irregularities, AI offers invaluable advantages by analyzing massive amounts of data in real time. For example, when detecting bid-rigging cartels (collusion in public construction projects or procurement), AI sifts through historical data and immediately identifies instances where bids from supposedly competing companies show suspicious similarities in their calculations or systematically take turns winning contracts. Forensic data analysis: AI systems scan millions of emails, chat logs, and expense reports within a company. They no longer search solely for rigid keywords like “bribe,” but instead understand the context. They raise the alarm when tone, communication patterns, or payment flows deviate from the norm.

The most dangerous blind spot occurs when two competing online retailers use sophisticated, self-learning pricing algorithms. Although the managers do not communicate with each other, the algorithms “learn” through constant interaction that it is more profitable for both sides to keep prices high rather than undercut each other. If many competitors in the same industry use the same AI-powered pricing or optimization software, they feed the same algorithm with data, often confidential. The software then acts as a central hub that sets prices for all participating spokes. blends in seamlessly with the market without competitors having to communicate directly with one another. It is also dangerous when decisions are hidden behind the complexity of an algorithm (black box). If the compliance department asks why order X went to company Y, the answer is: “The system (AI) calculated it that way.”

The EU AI Act is in effect, while Switzerland is taking a sector-by-sector approach. How should Swiss companies with business in the EU handle this?

I would not recommend that Swiss companies with business in the EU wait for Switzerland’s own AI regulations (scheduled for 2027). The pragmatic approach is to treat the EU AI Act as the relevant compliance standard, even if the company is based in Switzerland. The reason is its extraterritorial scope: Among other things, the EU AI Act covers providers outside the EU if they make AI systems available on the EU market, as well as providers or operators outside the EU if the output of their AI system is used in the EU. For this reason, I would advise Swiss companies to conduct an “EU exposure assessment.”

Where do we use or provide AI? Which AI systems are used by Swiss employees? Which ones are offered to customers in the EU? And where are AI-generated results used in the EU? Which risk category applies to us?

One should carefully examine prohibited AI practices, high-risk AI, transparency requirements, documentation, governance, and control obligations, etc. Which existing compliance systems (data privacy, IT security, risk management) can we utilize? A Swiss company does not necessarily need a completely separate “AI compliance program.”

You describe a resilient compliance program approach as forward-looking, data-driven, and embedded in day-to-day operations. How can this be achieved without turning compliance into a bureaucratic machine?

The true art of a good compliance program: It should reduce complexity, not create additional complexity.

Risk over rules: Don’t create a new process for every theoretical risk. First, understand where the truly relevant risks lie, and focus resources there.

Integrate into existing processes: Compliance should take place, as much as possible, where decisions are made anyway, such as in procurement, business travel, third-party management, or product launches. Employees should not have to initiate a separate “compliance procedure” for every decision.

Using Data for Prevention: Training rates or the number of completed audits say relatively little about actual compliance. More interesting, for example, are recurring “speak-up” topics, investigation patterns, audit findings, exceptions, or unusual transactions. From these, the compliance function can identify where risks are developing before they turn into incidents.

Regularly question controls: Every compliance measure should be able to withstand the following counterquestion: What risk does it address, and what would happen if we eliminated it? If there is no longer a convincing answer to that question, the measure should be simplified or eliminated.

What role does a speak-up culture play if, in the future, poor decisions are made based on the code rather than on judgment?

When wrong decisions are no longer caused by human error in the mind but by flawed logic or distorted data in the code, the role of the speak-up culture changes fundamentally.

It transforms from an ethical oversight mechanism into a technological early-warning lifeline for the company. In the AI era, the speak-up culture must take on a completely new dynamic, as the way errors arise and are concealed is changing drastically:

A strong speak-up culture encourages employees to explicitly question the results of an algorithm. If the AI rejects a damage report or a specialist feels that the system is systematically disadvantaging certain customer groups, they must set aside any fear of being labeled “anti-technology” or “incompetent.” Today, “speak up” means openly addressing the discrepancy between human intuition and machine output.

Mistakes in our heads often happen suddenly and are obvious. Errors in code develop quietly and gradually: Over the course of months, an algorithm becomes less accurate or more discriminatory without anyone noticing, because the real world is changing but the training data is outdated. Front-line employees (e.g., in customer service) are the first to notice these subtle anomalies, long before IT sees them in the dashboards. A functioning speak-up culture ensures that these subtle observations are reported immediately, and without bureaucratic hurdles, to the tech and compliance teams.

Programmers and data scientists are under enormous pressure to get projects up and running quickly. When they realize that an AI model is mathematically unstable or has ethical flaws, it takes courage to pull the plug just before go-live. A “speak-up” culture must be firmly established within the IT team as a guarantee of psychological safety. Developers should not fear reprisals or project delays when they point out risks in the code.

Until now, whistleblowing has mostly revolved around a boss paying bribes or a cartel meeting at a hotel. In the future, it will be about deliberately turning a blind eye to data quality (e.g., when management approves substandard or biased data to cut costs). Speak-up channels and whistleblowing systems must be technically understood and staff must be trained in such a way that they can also be used to report data manipulation and code violations (e.g., deliberate non-compliance with the EU AI Act).

The role of the Chief Compliance Officer is changing. What skills will he need in five years that he does not yet possess today?

The CCO of the future will need less additional knowledge of regulations than today, but significantly more understanding of technology, data literacy, business acumen, strategic thinking, and ethical judgment. Some skills will become significantly more important:

Data and Technology Proficiency: A CCO doesn’t need to know how to code. But he must understand how AI, data analytics, and automation identify risks, influence decisions, and transform compliance processes. Those who cannot interpret data will become increasingly dependent on others.

AI Governance and Algorithmic Thinking: The CCO must be able to assess how an AI system could pose a compliance or ethical risk, for example, due to bias, a lack of transparency, data protection issues, manipulation, or autonomous decisions, and how to incorporate these risks into governance frameworks and processes.

Systemic Thinking: Compliance risks will increasingly occur in an interconnected manner. Cybersecurity, data protection, supply chains, human rights, AI, corruption, and reputation can all be interrelated. The CCO must understand these interconnections rather than simply managing individual areas of law.

Business and Transformation Expertise: Compliance must understand how the company makes money, how decisions are actually made, and where change processes take place. Only then can compliance be integrated into the business process rather than operating on the sidelines.

The ability to deal with uncertainty: Regulatory developments will not follow a linear path everywhere. The CCO must develop scenarios and enable decision-making even when not every legal issue has been definitively resolved.

Ethical judgment. The CCO of the future must therefore not only know what is permitted, but also help management decide what is responsible.

What can different industries learn from pharmaceutical compliance, one of the most heavily regulated industries of all?

The pharmaceutical industry has been operating under intense regulatory pressure for decades. Other industries can learn two fundamental principles in particular from pharmaceutical compliance.

“Validation” instead of blind trust. Before software is deployed in production, it must be “validated.” This means that it is scientifically and comprehensively proven that the system does exactly what it is supposed to do in every situation.

An uncompromising culture of documentation (“If it wasn’t documented, it didn’t happen”). In pharmaceutical compliance, there are no verbal agreements or informal approvals. Every change to a process, every deviation from a standard (Deviation Management), and every corrective action (CAPA – Corrective and Preventive Action) must be precisely documented, justified, and signed off by the responsible parties. The lesson for other industries: In cases of liability (e.g., following an incorrect decision by an algorithm or a data breach), a thorough documentation history can save a company from ruin. It proves to regulators and courts that management did not violate its duty of care.

What advice do you give to small and medium-sized businesses that have neither a compliance department nor an AI governance budget?

Not having the funds to maintain a dedicated compliance department is a valid limitation. Not having the funds for compliance itself, however, is not. Small businesses do not need a lesser commitment to compliance, but rather a simpler compliance structure. They need to know which rules apply to them and have a credible method for managing their material risks. Compliance should be risk-based, not based on company size.

The guiding principle should be risk-based, not resource-based. The absence of a compliance department does not negate the underlying legal or ethical obligations. I would break down the message into five steps:

First: Understanding Risks – Conducting a pragmatic compliance risk assessment that focuses on the laws and risks that are actually relevant to the company.

Second: Assign responsibility – Someone must be explicitly responsible for compliance, even if there is no compliance officer or dedicated department.

Third: Establish fundamental controls – a manageable number of clear guidelines, procedures, approval processes, reporting channels, and escalation mechanisms that are tailored to the greatest risks.

Fourth: Raise awareness – train employees on the rules relevant to their work and, most importantly, explain why these rules are important.

Fifth: Review and adapt – A compliance program does not have to be highly complex, but it should evolve as the company, its markets, its workforce, its customers, or its risks change.

An SME doesn’t need an AI governance department simply because it uses AI. However, it should know what AI it is using, what risks are associated with it, who is responsible, and what control measures are necessary.

ECS thrives on the exchange of ideas among practitioners. Why is membership also worthwhile for insurers and brokers?

For an annual membership fee of 200 Swiss francs, ECS offers more than 30 events each year through ten different working groups (covering topics such as anti-corruption, competition law, AI and compliance, and third-party risk management). In addition to professional networking, members have the opportunity to exchange ideas with peers about how they address specific compliance issues. Membership also provides discounts on continuing education programs offered by universities. For insurers and brokers, membership in Ethics and Compliance Switzerland is particularly worthwhile because it looks beyond the confines of their own highly regulated industry.

If you could ask a board member just one question about their use of AI, what would it be?

Where are we willing to draw the line between what AI is allowed to decide on our behalf, and what it isn’t? This question forces the board of directors to grapple with several fundamental issues at once:

Accountability: Who bears ultimate responsibility if an AI-driven decision causes harm? Ethics: Are there decisions that should never be delegated to an AI? Human oversight: Where must sound human judgment be maintained? Trust: Would we be prepared to explain the role of AI to employees, customers, regulatory authorities, and the public? Governance: Are our control mechanisms keeping pace with the speed of AI adoption?

What kind of decision should a machine never make?

The EU AI Act defines a whole list of uses where AI should not be employed.

These questions were posed by Binci Heeb.

See also: The Blind Spot in Acquisitions: How a Lack of GRC Makes Life Difficult


Tags: #AI #Airbag #Algorithm #Antitrust Risks #Brake #COO #Corruption #EU AI Act #Have a Say #Insurance industry #Machine #Nonprofit Association #Resilience #Responsibility #Speak-up Culture