Create a Catalogue from file

Page Content

Prerequisite

In order to successfully create a new catalogue from a file you will need a delimited flat file (*.csv, *.txt) or Excel (*.xls).

Delimited flat files are easier to load than Excel

It is key that the file loaded doesn’t already exist as a catalogue in RDMP. If the new catalogue is specific to a project please rename the file in relevance with the topic, before starting this process.

This process is sensitive to failure due to:

Column type incompatibility
Fields size capacity
…

RDMP system tries its best to cover the errors which can occur at any point of the loading process. Please refer to section - When something goes wrong… of this page for help.

Steps

Step 1 - Access to the Creation view

Click on "Catalogues (button)" in RDMP top menu

Right click on the folder where the catalogue needs to be created and select "Create new catalogue from file…"

Step 2 - Data Source

Pick the file which contains the catalogue. At this stage the format of the file selected doesn't matter as long as it is a file which is data compatible. (Excel or flat file).

Step 3 - RDMP database Destination

Fill in the database details

UserID / Password (optional): When connecting the database to test if a catalogue can be created in it, the system will use these credentials. If not informed, RDMP will use windows credential by default.
Server: by default, the SQL server which has been setup when RDMP has been installed. It can be changed at this stage if another server exists.
Database: List of all database on the server.

If the server has been changed, click on the refresh button to update list of databases

Step 4 - Pipeline

Select a pipeline

If the file imported is a flat file, select: BULK INSERT: CSV Import File.
If the file is in Excel, select: BULK INSERT: Excel File.

RDMP will automatically display the compatible pipeline to the file selected.

Click on Execute

Step 5 - Configure Extractability view

Once the catalogue has been imported with success, a new window pops up to configure the level of confidentiality for each item of the new catalogue.

Catalogue Name (mandatory): By default, it's the name of the file and Table create in the database to store data related to the catalogue. It can be changed without impacting the table name
Table Name (mandatory): When created the table has been named by the file name. It can be modified at this stage, but it doesn't change the Catalogue name. Unless necessary, there is no specific reason to change it.
Acronym: Used for the metadata catalogue title. It can also be used to name the extracted file / folder, as a way of shortening the name down
Description: Used for the metadata catalogue description
Project link: If a project has been created previously and the catalogue is specific to this project only, click pick and select the project.

Each catalogue item needs to be setup with a specific extractability level respecting the following baseline:

Core: Doesn't require approval to extract, always in the list of items extracted by default.
Supplemental: not a routinely extracted column, but does not need special approval either
SpecialistApprovalRequired: Can be extractable but requires specific governance approval
Internal: Internal use only (aggregation etc…)
Deprecated: Obsolete

Please note, only CatalogueItems setup as Core will be extracted without requiring additional action.

Individual configuration: each item is also configurable one by one in the main window.

Mass configuration: under Set All, at the bottom right of the window, select the extractability level you want all items to be at.

When there is a lot of items with the same extractability level, it could be useful to first set them all with the same type of extraction using the mass configuration and then change the one which have specific levels of extractability using individual option.

Patient Identifier Column: this field establish the patient identifier for the catalogue. This is mandatory and will be anonymised automatically when the catalogue is extracted

Filter: When the catalogue contains a lot of Catalogue Items, the Filter as the bottom of the window allows a quick search in the items

Final Step

Once all parameters for Catalogue extractability is in place, either:

press Ok, job done
or Add to existing Catalogue to JOIN with another table and create only one Catalogue (the linkage will be created automatically based on the Patient Identifier)

The number of columns added to the existing catalogue will depend on the extractability configuration just setup. Only Core items will be added.

or Cancel (Do not create a Catalogue)

At this stage the table with data has already be imported to the database suggested at the first screen. The system then asks if the table needs to be kept or not. If kept, the catalogue won't be created but it can be re-created later using the option Create Catalogue from table.