...
| Table of Contents | ||||||
|---|---|---|---|---|---|---|
|
...
Use Case
You have data in a file on disk that you want to load into your database and use with RDMP for cohort building and/or project extractions.
File Types
In order to successfully create a new catalogue from a file , RDMP accept only specific files formats: you will need a delimited flat file (*.csv, *.txt) or Excel (*.xls).
| Info |
|---|
Flat files are easier to load than Excel |
It is key that the file loaded doesn’t already exist as a catalogue in RDMP. If the new catalogue is specific to a project please rename the file in relevance with the topic, before starting this process.
| Note |
|---|
This process is sensitive to failure due to:
RDMP system tries its best to cover the errors which can occur at any point of the loading process. Please refer to section - When something goes wrong… of this page for help. |
Steps
Step 1: Click on "Catalogues (button)" in RDMP top menu
...
Step 2: Right click on the folder where the catalogue needs to be created and select “Create new catalogue from file…”
...
Delimited files (e.g. csv) must be properly formatted (e.g. quoting fields with commas, newlines etc). Where there is a problem RDMP will indicate the line number and nature of the problem. For more information see When something goes wrong… |
...
Steps
Step 1 - Open the Catalogues Collection
Click on Catalogues button in RDMP top menu
...
Select New->From File…
...
Step 2 - Data Source
Pick the file which contains the catalogue. At this stage the format of the file selected doesn’t doesn't matter as long as it is a file which is data compatible. (Excel or flat file).
...
...
Step 3 - RDMP database Destination
Fill in the database details
...
UserID / Password (optional): When connecting the database to test if a catalogue can be created in it, the system will use these credentials. If not informed, RDMP will use windows credential by default.
Server: by default, the SQL server which has been setup when RDMP has been installed. It can be changed at this stage if another server exists.
Database: List of all database on the server. This should be where you store your clinical data, not the RDMP database itself.
| Info |
|---|
If the server has been changed, click on to the refresh button to update the list of databases |
Step
...
If the file imported is a flat file, select BULK INSERT: CSV Import File.
If the file is in Excel, select: BULK INSERT: Excel File.
...
4 - Pipeline
Select an appropriate pipeline for reading your file type.
...
| Info |
|---|
RDMP will automatically display the compatible pipeline |
...
based on the file selected. |
Step 6: once all setup, click on Execute
Step 7: Configure Catalogue items extractability
Once Click Execute to begin the upload process.
Step 5 - Configure Extractability view
Once the catalogue has been imported with success, a new window pops up to configure the level of confidentiality for each item of the new catalogue.
...
Catalogue Name (mandatory): By default, it’s it's the name of the file and Table create in the database to store data related to the catalogue. It can be changed without impacting the table name
Table Name (mandatory): When created the table has been named by the file name. It can be modified at this stage, but it doesn’t doesn't change the Catalogue name. Unless necessary, there is no specific reason to change it.
Acronym: Used for the metadata catalogue title. It can also be used to name the extracted file / folder, as a way to shorten of shortening the name down
Description: Used for the metadata catalogue description
Project link:
If a project has been created previously and the catalogue is specific to this project only, click pick and select the project.Configure the level of extractability of items:
...
Each catalogue item needs to be setup with a specific extractability level respecting the following baseline:
Core:
...
Doesn't require approval to extract, always in the list of items extracted by default.
Supplemental: not a routinely extracted column, but does not need special approval either
SpecialistApprovalRequired: Can be extractable but requires specific governance approval
Internal: Internal use only (aggregation etc…)
Deprecated: Obsolete
| Info |
|---|
Only Please note, only CatalogueItems setup as Core will be extracted without requiring additional action. |
There are 2 ways of configuring the extractability level:
Individual configuration: each item is also configurable one by one in the main window.
...
Mass configuration: under Set All, at the bottom right of the window, select the extractability level you want all items to be at.
...
When there is a lot of items with the same extractability level, it could be useful to first set them all with the same type of extraction using the mass configuration and then change the one which have specific levels of extractability using individual option.
Patient Identifier Column: this field establish the patient identifier for the catalogue. This is mandatory and will be anonymised automatically when the catalogue is extracted
...
4- Filter: When the catalogue contains a lot of Catalogue Items, the Filter as the bottom of the window allows a quick search in the items.5-
Final Step
...
Once all parameters for Catalogue extractability is in place, either:
press Ok, job done
or Add to existing Catalogue to JOIN with another table and create only one Catalogue (the linkage will be created automatically based on the Patient Identifier)
The number of columns added to the existing catalogue will depend on the extractability configuration just setup. Only Core items will be added.
...
or Cancel (Do not create a Catalogue)
At this stage the table with data has already be imported to the database suggested at the first screen. The system then asks if the table needs to be kept or not. If kept, the catalogue won’t won't be created but it can be re-created later using the option Create Catalogue from table.
When something goes wrong
...