Calvin Risk: Innovative AI governance and risk management for companies

For almost three years, the ETH spin-off Calvin Risk has been helping companies transition to high-quality, high-performance and low-risk AI. Syang Zhou and Julian Riebartsch ventured into self-employment with their insurtech to create something bigger and do nothing less than change the world for more humanity.

Find out whether they succeed in doing so in an interview with thebrokernews. Binci Heeb talks to Julian Riebartsch, CEO and co-founder of Calvin Risk.

Julian Riebartsch, who had the initial idea for Calvin Risk?

The idea for Calvin Risk arose from the shared conviction of Syang Zhou and myself that there is an urgent need for solutions to systematically understand and manage the risks of artificial intelligence. Especially in companies, we often see a tension between the urge to innovate and a lack of risk awareness – this is exactly where we wanted to start. Syang had previously supported banks and insurance companies such as UBS, ABN AMRO, Generali and ZKB at PwC in bringing AI systems into production – whether through their development or through tech audits. It became clear how high the hurdles are for these companies to actually use AI systems productively, as the risks are difficult to grasp. I myself worked for a deeptech venture capital fund at the time and saw similar challenges with startups with AI-based products: Even with promising pilot projects, the step into productive use was often almost impossible to master. Together with Syang’s research at ETH Zurich during his PhD on robust and secure AI in finance, the idea for Calvin Risk developed.

At ETH Zurich, you focused in particular on the development of AI-based forward modeling software for astronomical surveys and dark matter experiments… Can you explain this in simple terms and tell us how this supported you in co-founding Calvin Risk?

Basically, it was about modeling complex astrophysical phenomena with the help of AI and predicting what the associated properties will look like – for example, how galaxies will develop. This involves vast amounts of data that need to be analyzed and modeled. This taught me how powerful, but also how sensitive, AI systems are. These experiences were central to Calvin Risk, because companies also face the challenge of using AI systems in a secure and comprehensible way. This is exactly what we are analyzing today – using methods that are deeply rooted in science.

Many insurtechs come and go. What do you do differently from those that (have to) give up again?

We focus very strongly on the concrete added value for companies. For us, it’s not about buzzwords, but about tangible, quantitative risk measurement, as well as automated quality assurance and requirements management. What’s more, we actively listen to our customers – from DAX-listed companies to SMEs – and develop our tools together with them.

Last year, Calvin Risk was named Insurtech of the Year 2024 at the Swiss InsurTech Hub Summit and Awards, congratulations from thebrokernews! What does this award mean to you?

Thank you very much! The award was a huge honor for us – and a nice confirmation that our topic is relevant. It also shows that risk analysis in AI is not just a niche topic, but belongs at the center of corporate strategy right next to the use of AI.

You say that Calvin Risk has the most comprehensive quantitative solutions for assessing and managing the risks of AI algorithms. What makes yours different from other solutions?

Our solutions are data-driven, comprehensible and scalable. We do not offer a “black box”, but measurable metrics and structured reports that also convince regulators. In addition, we work independently of specific models – whether GPT-4, our own machine learning models or others: we always analyze at the level of risks such as fairness, bias, accuracy, transparency and security.

What are the biggest risks?

Some of the greatest risks lie in the area of unintentional bias, security gaps, lack of traceability and lack of robustness. It becomes particularly dangerous when organizations are unaware of these risks or underestimate them – then there is not only the threat of ethical problems, but also legal and economic damage.

How do you help companies to reduce and manage their AI activities and the associated risks?

We offer a toolset with which companies can test, evaluate and continuously monitor their AI models. We also provide strategic support – for example in the development and digitalization of internal practical governance guidelines and processes or in preparing for regulatory requirements such as the EU AI Act.

How does a comprehensive risk assessment work at Calvin Risk?

Our platform guides companies through a structured analysis: from the model architecture to training data and specific application scenarios. We measure various risk indicators, compare them with benchmarks and provide clear recommendations on how to minimize risks.

How do you deal with the rapid technological development in the field of generative AI – especially when models such as GPT-4 or Claude are rapidly developing new capabilities? Can governance frameworks keep pace at all?

That is definitely a challenge. We therefore rely on flexible, adaptive and modular frameworks that can grow with the technology. It is important that governance is not rigid, but flexible enough to accommodate new risks – and this is exactly what our dynamic analyses make possible.

To what extent can your tools and methods also help smaller companies that may not have dedicated compliance or risk departments but still want to use AI?

For smaller companies in particular, we offer out-of-the-box requirements for compliance, customized, easy-to-understand dashboards and automated recommendations. Our goal is to make AI risk management accessible – not only for large corporations, but also for start-ups and SMEs.

Are there certain ethical principles that you have integrated into your software architecture – for example with regard to fairness, non-discrimination or sustainability?

Yes, these principles are central. For example, when analyzing non-discrimination, we follow the anti-discrimination laws relating to protected groups in the EU and the USA. However, there will always be aspects that have to be decided by the companies themselves – for example, how important the role of fairness towards individuals versus fairness towards groups is. In any case, our software helps to systematically analyze fairness and discrimination and identifies potential measures for improvement. Transparency is also important to us: our customers should understand why a risk exists and how it can be addressed.

How do you experience the exchange with regulators – for example in the context of the EU AI Act? Are start-ups like yours involved in the discussion, or is regulation lagging behind practice?

The exchange is increasing, but there is still room for improvement. We are actively involved in working groups to bring in the perspective of young tech companies. The EU AI Act is an important step – but it must remain practicable. Start-ups like ours can contribute a lot here.

Finally, could you imagine your platform becoming interesting for government or non-profit organizations in the long term – for example in the health or education sector?

Absolutely. Responsible use of AI is essential, especially in the healthcare and education sectors. We are already holding initial talks with public authorities – in the long term, we would also like to use Calvin Risk in areas where AI decisions affect particularly vulnerable groups.

Julian Riebartsch, CEO & Co-Founder – With a background in computational physics and venture capital, Julian combines technical expertise with practical experience in risk and governance. He is a graduate of ETH Zurich and the University of Tokyo and was previously an investor at b2ventures and Matterwave Ventures.

Read also: Ledgertech: Is embedded insurance the future?