Create a Catalogue from file
- Magalie Guignard-Duff
- Thomas Nind (Unlicensed)
Use Case
You have data in a file on disk that you want to load into your database and use with RDMP for cohort building and/or project extractions.
File Types
In order to successfully create a new catalogue from a file you will need a delimited flat file (*.csv, *.txt) or Excel (*.xls).
Excel files with multiple tabs, formulas, sprawling layouts etc may not be machine readable. In such cases you may wish to export Excel data as csv first and load that.
It is key that the file loaded doesn’t already exist as a catalogue in RDMP. If the new catalogue is specific to a project please rename the file in relevance with the topic, before starting this process.
Delimited files (e.g. csv) must be properly formatted (e.g. quoting fields with commas, newlines etc). Where there is a problem RDMP will indicate the line number and nature of the problem.
For more information see When something goes wrong…
Steps
Step 1 - Open the Catalogues Collection
Click on Catalogues button in RDMP top menu
Select New->From File…
Step 2 - Data Source
Pick the file which contains the catalogue. At this stage the format of the file selected doesn't matter as long as it is a file which is data compatible. (Excel or flat file).
Step 3 - RDMP database Destination
Fill in the database details
UserID / Password (optional): When connecting the database to test if a catalogue can be created in it, the system will use these credentials. If not informed, RDMP will use windows credential by default.
Server: by default, the SQL server which has been setup when RDMP has been installed. It can be changed at this stage if another server exists.
Database: List of all database on the server. This should be where you store your clinical data, not the RDMP database itself.
If the server has been changed, click on the refresh button to update list of databases
Step 4 - Pipeline
Select an appropriate pipeline for reading your file type.
Click Execute to begin the upload process.
Step 5 - Configure Extractability view
Once the catalogue has been imported with success, a new window pops up to configure the level of confidentiality for each item of the new catalogue.
Catalogue Name (mandatory): By default, it's the name of the file and Table create in the database to store data related to the catalogue. It can be changed without impacting the table name
Table Name (mandatory): When created the table has been named by the file name. It can be modified at this stage, but it doesn't change the Catalogue name. Unless necessary, there is no specific reason to change it.
Acronym: Used for the metadata catalogue title. It can also be used to name the extracted file / folder, as a way of shortening the name down
Description: Used for the metadata catalogue description
Project link: If a project has been created previously and the catalogue is specific to this project only, click pick and select the project.
Individual configuration: each item is also configurable one by one in the main window.
Mass configuration: under Set All, at the bottom right of the window, select the extractability level you want all items to be at.
Patient Identifier Column: this field establish the patient identifier for the catalogue. This is mandatory and will be anonymised automatically when the catalogue is extracted
Filter: When the catalogue contains a lot of Catalogue Items, the Filter as the bottom of the window allows a quick search in the items
Final Step
Once all parameters for Catalogue extractability is in place, either:
press Ok, job done
or Add to existing Catalogue to JOIN with another table and create only one Catalogue (the linkage will be created automatically based on the Patient Identifier)
The number of columns added to the existing catalogue will depend on the extractability configuration just setup. Only Core items will be added.
or Cancel (Do not create a Catalogue)
At this stage the table with data has already be imported to the database suggested at the first screen. The system then asks if the table needs to be kept or not. If kept, the catalogue won't be created but it can be re-created later using the option Create Catalogue from table.