Troubleshotting Reloading Study in a new server with a new version of Labkey Server: /home/Support/LabKey Support Forum

Troubleshotting Reloading Study in a new server with a new version of Labkey Server

LabKey Support Forum

View Message

Troubleshotting Reloading Study in a new server with a new version of Labkey Server

hilariagrieve

2021-01-26 08:03

Status: Closed

Hi!
I have problems reloading the Study in a new version of Labkey Server in another Server.
I have instaled in one server Labkey v17 with a Study Proyect data.
Now we have e new server, and I installed the new version of labkey v20.7. I tried to creat the same Study Proyect exporting the Study Proyect from the old server and importing this export in the new server.
Everything was fine untill the pipeline got an error.

This is the log file:
26 ene. 2021 11:55:33,943 INFO : Starting to run task 'org.labkey.study.importer.StudyImportInitialTask' at location 'webserver'
26 ene. 2021 11:55:36,082 INFO : Reloading study from LIMS SGFER_2021-01-26_10-44-53.folder.zip
26 ene. 2021 11:55:36,222 INFO : Loading top-level study properties (label, start/end date, description, etc.)
26 ene. 2021 11:55:40,028 INFO : Done importing top-level study properties (label, start/end date, description, etc.)
26 ene. 2021 11:55:49,286 INFO : Loading QC states
26 ene. 2021 11:55:50,341 INFO : Done importing QC states
26 ene. 2021 11:55:50,349 INFO : Loading treatment data tables
26 ene. 2021 11:55:51,090 INFO : Done importing treatment data tables
26 ene. 2021 11:55:51,138 INFO : Loading assay schedule tables
26 ene. 2021 11:55:51,620 INFO : Done importing assay schedule tables
26 ene. 2021 11:55:51,780 INFO : Loading datasets manifest from datasets_manifest.xml
26 ene. 2021 11:55:51,900 INFO : Loading dataset schema from datasets_metadata.xml
26 ene. 2021 11:55:56,663 INFO : Successfully completed task 'org.labkey.study.importer.StudyImportInitialTask'
26 ene. 2021 11:55:56,910 INFO : Starting to run task 'org.labkey.study.pipeline.StudyImportDatasetTask' at location 'webserver'
26 ene. 2021 11:55:57,055 INFO : Start batch Lims SGFER.dataset
26 ene. 2021 11:55:57,079 INFO : Causas: Starting delete
26 ene. 2021 11:55:57,098 INFO : Causas: Deleted 5075 rows
26 ene. 2021 11:55:57,130 INFO : Causas: Starting import from dataset5001,tsv
26 ene. 2021 11:56:14,049 DEBUG: commit complete
26 ene. 2021 11:56:14,084 INFO : Causas: Successfully imported 5091 rows from dataset5001,tsv
26 ene. 2021 11:56:14,126 INFO : Secuenciacion ABI: Starting delete
26 ene. 2021 11:56:14,128 INFO : Secuenciacion ABI: Deleted 0 rows
26 ene. 2021 11:56:14,147 INFO : Secuenciacion ABI: Starting import from dataset5003,tsv
26 ene. 2021 11:56:15,202 ERROR: dataset5003,tsv -- Two columns mapped to target column: CausasId
26 ene. 2021 11:56:15,339 INFO : Extracción de ADN: Starting delete
26 ene. 2021 11:56:15,341 INFO : Extracción de ADN: Deleted 0 rows
26 ene. 2021 11:56:15,344 INFO : Extracción de ADN: Starting import from dataset5004,tsv
26 ene. 2021 11:56:15,716 ERROR: dataset5004,tsv -- Two columns mapped to target column: CausasId
26 ene. 2021 11:56:15,737 INFO : Cuantificación Fluorimétrica: Starting delete
26 ene. 2021 11:56:15,739 INFO : Cuantificación Fluorimétrica: Deleted 0 rows
26 ene. 2021 11:56:15,741 INFO : Cuantificación Fluorimétrica: Starting import from dataset5005,tsv
26 ene. 2021 11:56:15,899 ERROR: dataset5005,tsv -- Two columns mapped to target column: CausasId
26 ene. 2021 11:56:15,921 INFO : Cuantificación Real Time: Starting delete
26 ene. 2021 11:56:15,923 INFO : Cuantificación Real Time: Deleted 0 rows
26 ene. 2021 11:56:15,926 INFO : Cuantificación Real Time: Starting import from dataset5006,tsv
26 ene. 2021 11:56:16,017 ERROR: dataset5006,tsv -- Two columns mapped to target column: CausasId
26 ene. 2021 11:56:16,039 INFO : Amplificación por PCR: Starting delete
26 ene. 2021 11:56:16,041 INFO : Amplificación por PCR: Deleted 0 rows
26 ene. 2021 11:56:16,044 INFO : Amplificación por PCR: Starting import from dataset5007,tsv
26 ene. 2021 11:56:16,538 ERROR: dataset5007,tsv -- Two columns mapped to target column: CausasId
26 ene. 2021 11:56:16,545 INFO : Finish batch Lims SGFER.dataset
26 ene. 2021 11:56:16,549 INFO : Updating participant visits
26 ene. 2021 11:56:16,566 INFO : Updating participants
26 ene. 2021 11:56:16,749 INFO : Updating participant visit table
26 ene. 2021 11:56:16,750 INFO : Updating visit table
26 ene. 2021 11:56:16,752 INFO : Updating cohorts
26 ene. 2021 11:56:16,754 INFO : Clearing participant visit caches
26 ene. 2021 11:56:16,757 INFO : Finished updating participants
26 ene. 2021 11:56:16,807 INFO : Successfully completed task 'org.labkey.study.pipeline.StudyImportDatasetTask'

I attached a dataset file example that got that error, and this have 2 columns names that mapped to target column CausasId, "CaudasId" and "ParticipantId".

Please, can you help me to resolve this problem?
Everything is reloaded in the new study but the datasets data.
Thank you very much!
Hilaria

hilariagrieve responded:	2021-01-27 04:42
When I manually modify the .tsv file in the dataset 5003 and 5004, the systems returns the following log: 26 ene. 2021 12:58:23,219 INFO : Starting to run task 'org.labkey.study.pipeline.StandaloneDatasetTask' at location 'webserver' 26 ene. 2021 12:58:23,259 INFO : Start batch Lims SGFER.dataset 26 ene. 2021 12:58:23,319 INFO : Causas: Starting delete 26 ene. 2021 12:58:23,521 INFO : Causas: Deleted 5091 rows 26 ene. 2021 12:58:23,570 INFO : Causas: Starting import from dataset5001,tsv 26 ene. 2021 12:58:25,324 DEBUG: commit complete 26 ene. 2021 12:58:25,491 INFO : Causas: Successfully imported 5091 rows from dataset5001,tsv 26 ene. 2021 12:58:25,502 INFO : Secuenciacion ABI: Starting delete 26 ene. 2021 12:58:25,508 INFO : Secuenciacion ABI: Deleted 0 rows 26 ene. 2021 12:58:25,511 INFO : Secuenciacion ABI: Starting import from dataset5003,tsv 26 ene. 2021 12:58:27,614 ERROR: dataset5003,tsv -- Row 1 Missing value for required property: Run/STR_Kit1/STR_Kit_Id 26 ene. 2021 12:58:27,621 ERROR: dataset5003,tsv -- Row 1 Missing value for required property: Run/STR_Kit2/STR_Kit_Id 26 ene. 2021 12:58:27,625 ERROR: dataset5003,tsv -- Row 1 Missing value for required property: Run/STR_Kit3/STR_Kit_Id 26 ene. 2021 12:58:27,634 INFO : Extracción de ADN: Starting delete 26 ene. 2021 12:58:27,637 INFO : Extracción de ADN: Deleted 0 rows 26 ene. 2021 12:58:27,668 INFO : Extracción de ADN: Starting import from dataset5004,tsv 26 ene. 2021 12:58:28,004 ERROR: dataset5004,tsv -- Row 1 Missing value for required property: Run/IdInsumo/InsumoId 26 ene. 2021 12:58:28,020 INFO : Cuantificación Fluorimétrica: Starting delete 26 ene. 2021 12:58:28,023 INFO : Cuantificación Fluorimétrica: Deleted 0 rows 26 ene. 2021 12:58:28,032 INFO : Cuantificación Fluorimétrica: Starting import from dataset5005,tsv 26 ene. 2021 12:58:28,108 ERROR: dataset5005,tsv -- Two columns mapped to target column: CausasId 26 ene. 2021 12:58:28,298 INFO : Cuantificación Real Time: Starting delete 26 ene. 2021 12:58:28,301 INFO : Cuantificación Real Time: Deleted 0 rows 26 ene. 2021 12:58:28,316 INFO : Cuantificación Real Time: Starting import from dataset5006,tsv 26 ene. 2021 12:58:28,390 ERROR: dataset5006,tsv -- Two columns mapped to target column: CausasId 26 ene. 2021 12:58:28,434 INFO : Amplificación por PCR: Starting delete 26 ene. 2021 12:58:28,438 INFO : Amplificación por PCR: Deleted 0 rows 26 ene. 2021 12:58:28,450 INFO : Amplificación por PCR: Starting import from dataset5007,tsv 26 ene. 2021 12:58:28,566 ERROR: dataset5007,tsv -- Two columns mapped to target column: CausasId 26 ene. 2021 12:58:28,624 INFO : Finish batch Lims SGFER.dataset 26 ene. 2021 12:58:28,630 INFO : Updating participant visits 26 ene. 2021 12:58:28,631 INFO : Updating participants 26 ene. 2021 12:58:28,665 INFO : Updating participant visit table 26 ene. 2021 12:58:28,666 INFO : Updating visit table 26 ene. 2021 12:58:28,668 INFO : Updating cohorts 26 ene. 2021 12:58:28,670 INFO : Clearing participant visit caches 26 ene. 2021 12:58:28,685 INFO : Finished updating participants 26 ene. 2021 12:58:28,686 INFO : Successfully completed task 'org.labkey.study.pipeline.StandaloneDatasetTask'

chetc (LabKey Support) responded:	2021-02-22 10:16
Hello Hilaria, Thank you for reaching out to us! As you also pointed out, I see that you ran into "Two columns mapped to target column: CausasId" multiple times while importing in those datasets. It does look like one made it through without error the first time, dataset5001. After you made your changes the error message did change for the 2 tsv's you edited, however there is still an issue with those 2 and the other datasets. For dataset5005, dataset5006, and dataset5007 you should try and resolve the "Two columns mapped to target column: CausasId" issue as you did previously. For dataset5003 and dataset5004 you should try and determine the location of the missing value(s) it errors out on. Then populate the data. dataset5003 - Row 1 Missing value for required property: Run/STR_Kit1/STR_Kit_Id dataset5004 - Row 1 Missing value for required property: Run/IdInsumo/InsumoId What changes did you make to those dataset files? Thanks, Chet