Disclosure control of Artificial Intelligence and Machine Learning models

Introduction

Artificial Intelligence/ Machine Learning (AI/ML) models have introduced new demands of TRE providers. As discussed in this article, whether a model is being brought into or being taken out of the TRE, it is crucial to assess the risk of the model (to the environment, data, and governance). As such, these requests often involve multiple staff members within HIC, with specialist input that can lead to longer process times.

The pace of change in AI/ML is significant, and we would encourage TRE Users to clearly communicate their requirements to us as early as possible. We aim to understand more about your project, and we may ask you to complete our AI/ML triage form.

If you want to transfer an AI/ML model from Project A to Project B within the TRE (i.e. between segregated workspaces), it will be treated as 2 separate requests: one output request and one input request. This approach helps maintain data security in both TRE projects.

Requesting AI/ML model in to the TRE

We aim to understand your project requirements and will have already asked you if you intend on requesting a trained AI/ML model into our TRE. Our focus is on three key elements:

Governance Assessment: we check that we are authorized to host the model by asking for licencing agreements, and ensuring it covers the intended use within the TRE. If the model is also required to be taken out of the TRE, users should refer to the next section. We may also review any Data Sharing Agreements or Contracts that explicitly detail the model’s use.
Model Assessment: All submitted models undergo a technical assessment to ensure they are free from embedded viruses or malware and comply with our disclosure control processes. Only models that pass this assessment will be provisioned within the TRE.
Data Assessment: We assess whether we are legally permitted to host the model. For trained models, Users must provide evidence that only data approved for hosting by HIC were used during training. We may request additional documentation to confirm that the model contains no embedded data. This may include examining relevant Data Sharing Agreements or Contracts that authorize HIC to host the data.

Once the appropriate actions have been completed, the model will be released in to the TRE.

We will work with you to understand your project requirements. If these change - for example, requesting the model out of the TRE - governance may need to be reviewed, and could involve additional resources or costs.

Requesting AI/ML model out of the TRE

You must undergo our disclosure control process for any file requested out of the TRE, including your mode. This output process is also known as ‘egress’, which is the terminology used in your TRE workspace. There are 2 elements we focus on:

Risk Assessment: which is primarily concerned with ensuring the TRE User is educated in the risks, and that appropriate documentation is in place.
Model Assessment: which is the technical review of the trained AI/ML model.

We will expand on these areas below.

We reserve the right to not permit the AI/ML model to be provisioned in the TRE and/or released from the TRE

Disclosure control of AI/ML models

Risk Assessment

There are various factors that we will consider when assessing the risk impact:

Safe People: We expect Users working with these methodologies to understand the associated risks. We will discuss the process and risks with you to ensure you understand the reason behind this risk impact assessment. It is crucial that you have read our Guidance on Artificial Intelligence and Machine Learning Models in our TRE and have an up-to-date signed TRE User Agreement.
Data Minimisation: We validate the data used to train the model. We will check the pseudonymisation and derived data.
Contractual Agreement: If a third party is involved, appropriate documentation may be reviewed.
Ethics and transparency: We ensure that appropriate information governance is in place from the User Organisation, especially when a third party is involved, which may involve reviewing ethical standards and other documentation.
Data Protection Impact Assessment (DPIA): A DPIA helps to identify and reduce risks to project data. Project-specific DPIA can be crucial for managing data security effectively.

Model Assessment: Disclosure control of AI/ML

As TRE providers, we perform a ‘white box attack’ on models. This means that we have full access to the model and its parameters, allowing us to perform a direct assessment and range of different attacks. When assessing the model, we will consider the following:

AI/ML Triage Form: This document should be familiar from the start of the project, but it can evolve throughout a project lifetime and must be reviewed for accuracy.
Model Attack Report: A technical expert will conduct an ‘attack’ of the AI/ML model. This process relies on a growing body of evidence such as SACRO, GRAIMATTER, DARE UK recommendations and scientific papers such as https://arxiv.org/abs/2212.01233. As a Safe Haven, we conduct these checks, which can be thought of as a form of ‘ethical hacking’. We use an AI model specifically designed to attack other models to identify the risk of disclosing sensitive data. The attack model generates a report with numerical values showing the risk of recreating the source data (i.e. release individual-level data). Different types of attacks and processes are used, but we will apply the worst-case scenarios to quantify the risk and make informed decisions.
Model Risk Assessment: We ask Users questions relevant to the model, including information and statistics about the model, such as area under the curve (AUC) and model file sizes.

As with all HIC processes, documentation, communication and evidence are all stored on our project management system. This creates an audit trail and is crucial for our compliance with our security standards.