Disclosure control of AI/ML models

Introduction

Artificial Intelligence/ Machine Learning (AI/ML) models have introduced new demands of TRE providers. As discussed in this article, whether a model is being brought into or being taken out of the TRE, it is crucial to assess the risk of any individual-level data in the model. Our primary goal is to ensure the safety of all data within the TRE and to mitigate risk.

Whilst we have established processes for models being released from our TRE, the landscape is rapidly evolving and we would encourage Users to clearly communicate their requirements. At the project initiation stage, we aim to understand more about your project, and we may ask you to complete our AI/ML triage form. This form may be revisited to faciliate your work.

Our current processes focus on assessing risks related to the environment, data, and governance. As such, these requests often involve multiple teams within HIC, which likely leads to longer process times.

Requesting AI/ML model into the TRE

Working with Users, we aim to understand project requirements and will have already asked you if you intend on requesting a trained AI/ML model into our TRE. This request should be made through HIC Support. We focus on three key elements:

1. Governance Assessment: We ensure that we are authorized to host the model within our TRE. This involves asking for a licencing agreement for the model, ensuring it covers the intended use within the TRE project. If Users intend on requesting the TRE out of the TRE they should also refer to the next section at this stage. We may also review any Data Sharing Agreements or Contracts that explicitly detail the model’s use.

2. Model Assessment: This technical assessment ensures the model is free from embedded viruses or malware. Only files that pass this check will be provisioned in the TRE.

3. Data Assessment: We verify that we are legally allowed to host the model. For trained models, we must evidence that the model was trained using only data HIC are allowed to host. We may ask Users for evidence that the model has no embedded data. We may also review any Data Sharing Agreements or Contracts that allow HIC to host the data.

Once the appropriate actions have been completed, the model will be released into the TRE ready for use.

1. We will work with you to understand your project requirements using AI/ML methods. If these change part-way through the project, then the project governance may need to be re-assessed, which may lead to additional resources and costs.

Requesting AI/ML model out of the TRE

Just like other output requests, you must undergo our ‘egress’ process to take your model out of the TRE. There are 2 elements we focus on:

1. Risk Assessment: which is primarily concerned with ensuring the TRE User is educated in the risks, and that appropriate documentation is in place.

2. Model Assessment: which is the technical review of the trained AI/ML model.

We will expand on these areas below.

Disclosure control of AI/ML models

Risk Assessment

There are various factors that we will consider when assessing the risk impact:

1. Safe People: We expect Users working with these methodologies to understand the associated risks. We will discuss the process and risks with you to ensure you understand the reason behind this risk impact assessment. It is crucial that you have read our Guidance on Artificial Intelligence and Machine Learning Models in our TRE and have an up-to-date signed TRE User Agreement.

2. Data Minimisation: We validate the data used to train the model. We will check the pseudonymisation and derived data.

3. Contractual Agreement: If a third party is involved, appropriate documentation may be required.

4. Ethics and transparency: We ensure that appropriate information governance is in place from the User Organisation, especially when a third party is involved. Depending on the TRE User and information governance pathway, we will check compliance with all relevant ethical standards.

5. Data Protection Impact Assessment (DPIA): A DPIA helps to identify and reduce risks to project data. Project-specific DPIA can be crucial for managing data security effectively.

Model Assessment: Disclosure control of AI/ML

As TRE providers, we perform a ‘white box attack’ on models. This means that we have full access to the model and its parameters, allowing us to perform a direct assessment and range of different attacks. When assessing the model, we will consider the following:

1. AI/ML Triage Form: This document should be familiar from the start of the project, but it can evolve throughout a project lifetime and must be reviewed for accuracy.

2. Model Attack Report: A technical expert will conduct an ‘attack’ of the AI/ML model. This process relies on a growing body of evidence such as SACRO, GRAIMATTER, DARE UK recommendations and scientific papers such as https://arxiv.org/abs/2212.01233. As TRE providers, and a Safe Haven, we conduct these checks, which can be thought of as a form of ‘ethical hacking’. We use an AI model specifically designed to attack other models to identify the risk of disclosing sensitive data. The attack model generates a report with numerical values showing the risk of recreating the source data (i.e. release individual-level data). Different types of attacks and processes are used, but we will apply the worst-case scenarios to quantify the risk and make informed decisions.

3. Model Risk Assessment: We ask Users questions relevant to the model, including information and statistics about the model, such as area under the curve (AUC) and model file sizes.

 Related articles