Instrument data becomes even more useful when integrated with other data about the same participants in a
LabKey Study. For example, assay data can track how a blood marker changes over time. After integration, you can compare trends in that marker for patients receiving different treatments and draw more useful conclusions.
Assay data records are mapped to the study's participants and visits. Within the study, a new dataset is created which includes the assay data using a
lookup. This allows you to:
- Easily integrate assay data with clinical data.
- Create a broad range of visualizations, such as time charts.
- Utilize qc and workflow tools.
Manual Link of Assay Data to Study
Assay data that has already been uploaded can be linked to a study manually. You can link individual result rows, or select all the results in a run by selecting by run. Note that when linking large assays, selecting by run will give you more options for managing
larger bulk linking operations.
- Navigate to the view of runs or results containing the data you want to link.
- Select the desired rows in the checkbox column.
- Click Next.
- In the Sample/Specimen Match column, each row will show a (check) or icon indicating whether it can be resolved to a sample (or specimen) in that study.
- If your assay data includes a field of type "Sample", linking the assay results to Samples, then this column will be named Sample Match.
- Seeing the icon does not prevent linking assay data.
- For details on what these icons indicate, see below: What Do Checks and Xs Mean?.
- Click Link to Study to complete the linkage.
- You will see the new assay dataset within the target study. It is named for the assay design and now connected to the other participants in the study.
- Click a participant ID to see other information about that participant. (Then go back in your browser to return to this linked dataset.)
- Notice the View Source Assay link above the grid. Click it to return to your original assay folder.
- Click View Link to Study History to see how the linkage you just made was recorded.
- You can also see which rows were linked in the grid view of the assay data. Click "linked" to navigate to the dataset in the target study.
Date Fields During Linking
When linking assay data to a date-based study, the system looks for a field named
Date. If none is found, the system will look for another column of type "DateTime", i.e. a
DrawDate column might be present. Provided only one such column is found, it will be used for linking. If there are two or more "DateTime" columns, you may need to manually reenter the desired Date values for successful linking to study.
If your
Date column data includes a timestamp, you will only see and link the date portion by default (the time portion will become 00:00 in the linked dataset). To include the timestamp portion, click
Display DateTime link above the listing. This will add the time to the Date field so it will be included in the linked version. Note that whether or not the timestamp is displayed is controlled by
date display format settings.
Visit Fields During Linking
When linking assay data to a visit-based study, the system looks for a field named
VisitId. If your assay uses another field name, you may have to provide visit identifiers manually.
Automatic Link-to-Study upon Import
When you always plan to link data to the same study, you can automate the above manual process by building it directly into the assay design. Each run uploaded will then be automatically linked to the target study. You can set this field during assay creation, or edit an existing design as described here.
- In your assay folder, click the name of your assay design.
- Select Manage Assay Design > Edit Assay Design. If you want to preserve the original version of the design that does not do the link-to-study, select "Copy Assay Design" instead and copy it to the current folder, giving it a new name.
- From the dropdown for Auto-Link Data to Study, select the target study.
- See below for information about the "(Data import folder)" option, if present.
- If desired, set a Linked Dataset Category for the linked dataset. Learn more in this topic.
- Scroll down and click Save.
- The next data imported using this assay design will have all rows linked to the target study.
- You can find "linked-to-study" datasets in a study on the Clinical and Assay Data tab. Unless otherwise assigned a category, datasets added from assays will be "Uncategorized."
Shared Assays: Auto Link with Flexible Study Target
If your assay design is defined at the project level or in the
Shared project, it could potentially be used in many folders on your site.
- You can set a single auto-link study destination as described above if all users of this assay design will consolidate linked data in a single study folder.
- You may wish to configure it to always link data to a study, but not necessarily always to the same destination study. For this situation, choose the folder option "(Data import folder)". This will configure this assay design to auto link imported assay data to the study in the same folder where the data is imported.
You can then use this assay design throughout the scope where it is defined.
- If it is run in a folder that has both assay and study tools available, imported assay data will automatically be linked to that study.
- If it is used in a folder that is not also a study, the assay functionality will still succeed, but an error message about the inability to link to the study that doesn't exist will be logged.
Skip Verification Page (For Large Assay Files)
When your assay data has a large number of rows (thousands or millions of rows), the verification page can be impractical: it will a take a long time to render and to manually review each row. In these cases, we recommend skipping the verification page as follows:
- Navigate to the Runs view of the assay data. (The Results or Batches view of the data will not work.)
- Select the runs to be linked.
- Click Link to Study.
- Select the checkbox Auto link data from ## runs (## rows).
- This checkbox is available only when you initiate link-to-study from the Runs view of the assay data. If you don't see this checkbox, cancel the wizard, and return to the Runs view of the assay data.
- You will see both the number of runs you selected and the total number of result rows included.
- Click Next.
- The data will be linked to the study skipping the manual verification page. The linkage will run as a pipeline job in the background. Any assay data that is missing valid participant ids or timepoints will be ignored.
Recall Linked Rows
If you have edit permission on the linked dataset in the target study, you have the option to "recall" the linked row(s). Select one or more rows and click
Recall. When rows are recalled, they are deleted from the target study dataset.
The recall action is recorded on the "View Link to Study History" page in the assay folder.
Hover over the leftmost column in the grid for a
(details) icon. Click it for more details about that row.
What Do Checks and Xs Mean?
The
(check) or
icons when linking are relevant to
Sample or Specimen matches only. An X does not necessarily mean the link-to-study process should not proceed or does not make sense. For example, Xs will always be shown when linking into a study that does not contain specimens or refer to samples.
When assay data is linked to a study, it must be resolved to participants and dates or visits in the target study. There are several ways to provide this information, and participants and visits that do not already exist will be created. See
Participant/Visit Resolver Field. Matching incoming assay data to
Samples or Specimens is done when possible, but integrated assay data is still of use without such a match.
If the server can resolve a given ParticipantId/Visit pair in the assay data with a matching ParticipantId/Visit pair in the study, then it will incorporate the assay data into the existing visit structure of the study. If the server cannot resolve a given Participant/Visit pair in the assay data, then it will create a new participant and/or a new visit in the study to accommodate the new, incoming data from the assay. The assay data is linked into the study in either case, the difference is that in one case (when the assay data can be resolved) no new Visits or Participants are created in the study, in the other case (when the assay data cannot be resolved) new Visits and Participants will be created in the study.
Troubleshooting
Large Assay Link Operations
If you encounter errors during large link operations from the user interface, you may be attempting to link too many rows at once in one POST request. The upper limit is in the range of 9950 rows.
To workaround this limitation you can try the following:
- Split a manual link operation into several smaller chunks.
- If you are linking a large number of assay result rows, try starting from the Runs list instead to link the runs containing those results. You will then have the option to skip the verification page.
- If your data includes participant resolver information, use automatically link to study from within the data import wizard to bypass the manual import.
- If you routinely have many rows to link to the same study, do not need to interact with the data between import and link, and your data always includes the necessary participant resolver information, you can also enable the automatically link to study option in your assay design directly.
After completing a link-to-study by any method, you'll be able to review the set of result rows that were successfully linked on the
Link-to-Study History page.
Name Conflicts
In a study, you cannot have a dataset with the same name as an existing dataset or other table. This includes built in tables like "Cohort", "Location", and a table created for the "Subject Noun Singular" which is "Participant" by default. It is very unlikely that you would have an assay with the same name as one of these tables, but if you do, linking to study will fail with an error similar to:
Cannot invoke "org.labkey.api.data.TableInfo.getDefaultVisibleColumns()" because "tableInfo" is null
The exception to this name-conflict rule about is that if you have already linked rows from another assay with the same name, the new link action will successfully create a new dataset appending a 1 to that name.
Related Topics
