Version 8.3 represents a important step forward in the ongoing evolution of LabKey Server. Enhancements support LabKey's mission of providing the premier scientific data management system (SDMS) for experimental and observational study information.

New capabilities introduced in this release are summarized below. For a full query listing all improvements made in 8.3, see: Items Completed in 8.3.

Study

Quality Control for Study Data

LabKey’s quality control enhancements facilitate formal review procedures for new data acquired by your team. Team members can clearly mark and track the progress of data through a series quality control stages that are custom-tailored to your team's workflows.

• Quality Control via Automated Validation Checks
• Validation allows your team to check data for reasonableness and catch a broad range of field-level data-entry errors during the upload process.
• When an assay administrator adds or edits a schema field property, she can define range checks and/or regular expression checks on the property that are applied during data upload and row insertion.
• Uploaded assay data must satisfy any range or regular expression validations before it will be accepted into the database.
• Validation is available for all areas except Flow and MS1/MS2.
• Quality Control States to Facilitate Human Approval of Study Data
• The quality control process allows study administrators to define a series of approval and review states for data. These states can be associated with "public" or "nonpublic" settings that define the default visibility of the data.
• Different data approval states (and thus approval pathways) can be defined for data added to a study through different pathways (e.g., assay data copied to a study, CRF data imported via the Pipeline or TSV data inserted directly into a study dataset).
• Reviewers can the filter overview data grid by Quality Control State and thus find all data requiring review from a single screen.
• All quality control actions are audited.

Specimen Improvements

• Specimen report Participant ID (PTID) formatting. Specimen reports containing PTID lists are now output with one PTID per cell, rather than a delimited string in a single cell.
• Specimen notification configuration. Specimen notifications can now be configured to originate from either a fixed email address (previously the only option) or from the user who generates the notification.
• Repository selection. Requests by specimen (rather than by vial) now auto-select the best repository, or prompt the user to select if more than one is available.
• Specimen annotations. Specimen coordinators can now add comments at the vial or specimen level. These comments are visible at in both specimen and vial views by default. Comments should be maintained over specimen imports.
• Changing vial volumes. Specimen import now allows volumes for a single vial to change over time.
• Email notifications. If a specimen vial view with name "SpecimenEmail" is created, this view will be used for the vial list in all specimen request notification emails. This provides simpler and more readable email messages for the labs and repositories.

Fine-Grained Security for Editable Datasets

• Studies and the datasets they contain can be set to be editable by user groups.
• Users with edit permissions can edit existing rows of data, insert entire new rows or import a table that contains many new rows of data.
• Editing a dataset requires "write" permissions to both the folder and the dataset.

Performance Enhancements

• R View Caching.
• Scripts that are slow to render (due to large datasets or the complexity of the script) are now retained. This allows you to see the R View produced by this script without waiting for it to re-run.
• Currently available only for LabKey Study.
• Performed automatically.
• Dataset Snapshots
• Helps you quickly load datasets and views by minimizing data-reprocessing, which can slow data rendering.
• Allows you to create a snapshot of a dataset in time from a custom query, then build swift-loading views on top of this snapshot.
• Snapshots can be configured such that they will update regularly if the underlying data has been updated during a given time interval. This allows you to coalesce changes and re-process data only when necessary, in a way that does not slow your work.

Study Extensibility

• You can now define extra properties (metadata) that you wish to associate with your study or cohorts. This allows you to associate arbitrary buckets of data with your study.

Improved Study Search

• A search inside a study folder now finds matches for participant IDs, cohort names, dataset names, column names, dataset data itself, etc.
• Add the “Search” web part to search.

Visit Information Display on Portal Pages

• Admins can now add a “Visit” web part that introduces a Visit section to any study portal page.

Pipeline Data Processing

Configurable Workflows

• Add new analysis tools to preconfigured workflows
• Create your own workflows from scratch
• Add, remove, or edit command-line arguments to analysis tools
• Assign specific tasks to different computing resources
• XML file-based configuration, using the Spring Framework

Scalable Deployments

• Start with a single machine that runs the web server and all analysis tools
• Add remote work machines that run specific analysis tools
• Interface with clusters through the Globus Toolkit
• Monitor job status from a single location

Experimental Metadata

• Capture the inputs and outputs from each task as it runs
• Generate detailed experimental descriptions in XAR format to show exactly what analysis was performed
• Automatically load all recognized file types into the database when job is complete

Flow

ICS Metadata -- Beta Version

• ICS metadata supply the information necessary to distinguish background (control) wells from experimental wells and calculate average background values.

FlowJo Integration

• Improved “Import FlowJo Workspace” wizard.
• Ability to examine FCS files in directory separate from the workspace.
• Ability to render graphs from FlowJo-calculated statistics

Usability Improvements

• Improved comment editing
• Addition of discussion lists

Performance Enhancements

• Achieved through database optimization.

Site and Project Administration

User-Extensible Style Sheets

• Admins can upload and edit customized stylesheets that define themes on a project- and/or site-wide basis.

Project-Specific User Interface

• All of the "Look and Feel" UI elements that are currently configurable at the site level can now be superseded at the project level. This allows each project to have a custom web UI and custom string replacements in emails generated from the site.
• If a setting exists at the project level, it overrides the corresponding setting at the site level. Settings include all the UI elements currently set through the site administration pages, including logos, site name, support links, etc.

Project-Specific Administration

• With the new, expanded abilities for project-level administrators, site admins can delegate project-specific admin work without granting site-wide privileges to project admins.
• Project admins can view project member details and logs, plus impersonate any project member within the project.

Site Groups

• Site admins can now define and edit Site Groups. These provide a handy way for admins to manage individuals who have the same role across many projects.
• Project admins can assign permissions within projects to Site Groups.

Deactivated User Accounts

• User accounts can be deactivated, preventing undesired access while still retaining user information (e.g., authorship of issues, message board posts, specimen requests, etc.) for display in the audit log and author fields throughout the product.
• When deactivated user attempts to login, the event is logged.

User Impersonation Enhancements

• All admin session attributes (e.g., terms-of-use containers, lastFilter, previous page and container expansion state) are restored after impersonation is done.
• All actions of impersonators are now logged, so the audit trail for the actions of impersonators is more complete.

Wiki Link Checking

• You can now use generic link checking tools to check your site for missing links. Missing wiki pages return a 404 error.

Improved Display of Individual Lists on Portals

• All areas of LabKey Server now allow you to fully display a single list on a portal page via the "Single List" web part. You can select which view of the list to display.

Expanded LabKey API

The 8.3 release of LabKey Server expands the set of Server APIs available while adding two new Client API libraries along side the existing Javascript library, one library for R and one for Java. Client APIs make calling the Server API easier. The LabKey API enables developers for any LabKey installation to:

• Write scripts or programs in several languages to perform routine, automated tasks.
• Provide customized data visualizations or user interface for specific tasks that appears alongside the existing LabKey web server user interface.
• Develop entirely new user interfaces (web-based or otherwise) that run apart from the LabKey web server, but interact with its data and services.

The API is a secure and auditable way of programmatically accessing LabKey data and services. All APIs are executed within a user context with normal security and auditing applied.

Expanded Server API

• The Server API provides a set of URLs (or "links") exposed from the LabKey Server that return raw data instead of nicely-formatted HTML (or "web") pages. These may be called from any program capable of making an HTTP request and decoding the JSON format used for the response (Perl, JavaScript, Java, R, C++, C#, etc.). The new Server APIs are exposed by the new Client APIs covered below.

New Java Client API Library

• Allows selecting, inserting, updating and deleting data, plus executing arbitrary LabKey SQL.
• Provides javadoc documentation.

New R Client API Package

• Makes it easy to query LabKey Server and retrieve results in the native data structure for the language.
• Available for download via CRAN (the central repository of R packages) as the “Rlabkey” package.
• New "Export to R Script" option on the web site generates an R script stub that uses the Rlabkey package to retrieve the data you are currently viewing.

Enhanced Javascript Client API Library

• Exposes user information via security reporting APIs.
• Allows updating metadata on extensible objects (e.g., study properties, cohorts and schemas).
• Supplies lists of users in a project, folder or group, optionally filtered by name.
• Allows a developer to retrieve the container hierarchy visible to the current user.
• New EditorGridPanel and Store widgets, which are data-bound extensions of the Ext grid and store user interface widgets. The grid now exposes all of the properties, methods, and events from the Ext grid, and can participate in complex Ext layouts.

New Client APIs Available from Java, R and Javascript

• Allow modification of data (insert, update and delete) in study datasets, not just lists or custom schemas.
• Improve filtering, including support for “Equals One Of.”

LabKey SQL Additions

• Expand the choice of SQL functions that can be used within LabKey Server, including:
• SELECT DISTINCT
• FULL JOIN
• COALESCE