Full-Text Search Configuration

The Full-Text Search Configuration page allows you to configure the primary index, integrate external indexes, and review statistics about your index.

Primary Index Configuration

• Set Path. You can change the directory that stores the index (default: <tomcat>/temp/labkey_full_text_index) by entering a new path and clicking the Set Path button. Note that changing the location of the index requires re-indexing all data, which may affect performance. Resetting the path of the index is especially useful if you are running multiple LabKey deployments on the same Tomcat instance, because it allows each LabKey deployment to use a unique index.
• Start/Pause Crawler. The crawler, or document indexer, continually inventories your site when running. You might pause it to diagnose issues with memory or performance.
• Delete Index. You can delete the entire index for your server. Please do this with caution because rebuilding the index can be slow.
• Directory Type. This setting lets you can change the search indexing directory type. The setting marked "Default" allows the underlying search library to choose the directory implementation (based on operating system and 32-bit vs. 64-bit). The other options override the default heuristic and hard-code a specific directory implementation. These are provided in case the "Default" setting causes problems on a specific deployment. Use the default type unless you see a problem with search. Contact LabKey for assistance if full-text indexing or searching seems to have difficulty with the default setting.

You can integrate searching of other web sites (e.g., your organization's intranet) with LabKey Server's search functionality by pointing to an external search index. For example, you could generate a Lucene index using Nutch (an open-source web crawler), copy the index to a location accessible to your LabKey Server, and integrate that index into LabKey's search results.

To add an external index:

1. In the field External index description, enter a friendly name for the index.
2. In the field Path to external index directory, enter the absolute path to the index folder.
3. From the dropdown field Analyzer, select a method for extracting index terms. This analyzer must match the analyzer used at indexing time. Options include:

• SimpleAnalyzer: Splits text at whitespace and special characters such as '!', '@', and '#'. Uppercase words are not included in the index.
• KeywordAnalyzer: Indexes each term as a single word. Useful for zip codes, id numbers, url fragments, etc.
• EnglishAnalyzer: Attempts to extract the "stem" word for each indexable item, so that searches for "study", "studies", "studying", or "studied" will yield identical results.
• IdentifierAnalyzer: A simple, non-stemming analyzer for identifiers. Tokenizes only on whitespace; all punctuation is left intact.
• LabKeyAnalyzer: A hybrid analyzer that uses a non-stemming analyzer for categories and identifier fields, and a stemming English analyzer for all other fields. This is the standard analyzer optimized for LabKey Server usage.

This section provides information on the information that has been indexed by the system, plus identifies the limits that have been set for the indexer by the LabKey team. These limits enhance performance. For example, you will see the "Maximum Size" of files that will be scanned by the indexer; the maximum size allows the system to avoid indexing exceptionally large files.

Search Statistics

Lists average time in milliseconds for each phase of searching the primary index, from creating the query to processing hits.

Audit Log

To see the search audit log, click the Admin Console tab and click Audit Log in the "Management" section. Choose the Search option in the dropdown menu at the top of the auditing page. This displays the log of audited search events for your system. For example, you can see the terms entered by users in the search box. If someone has deleted your search index, this event will be displayed in the list, along with information on the user who ordered the delete.

Setup for Folder-Specific Search Boxes

By default, a site-wide search box is included on the top right side of every page of your LabKey Server site. You can add additional, scoped search boxes to individual projects or folders.

Add Search Web Part. To supply a search box that searches only a particular container, add the Search web part to the Portal page of a project or folder. See Add Web Parts for further details on how to add web parts. To see an example of a search box applied to a particular container, use the search box next to the labkey.org documentation, to the right of this page.

Set SubFolder Searching. Administrators can specify whether a search box searches just the current container or the current container and its sub-containers by default. Click on the "..." box on the title bar of the Search web part you've added. Now you can select or unselect "Search Subfolders" and set the default depth of search.

Searching List and External Schema Metadata

By default, the search index includes metadata for lists and external schemas (including table names, table descriptions, column names, column labels, and column descriptions).

You can turn off indexing of List metadata by unchecking the checkbox Index list meta data when creating or editing a list definition.

You can turn off indexing of external schema metadata by unchecking the checkbox Index Schema Meta Data when creating or editing an external schema definition. For details see External Schemas and Data Sources.

Include/Exclude a Folder from Search

You may want to exclude the contents of certain folders from searches. For example, you may not want archived folders or work in progress to appear in search results.

Searches that originate from that folder will still include its contents in the results, but searches from any other folder will not.

Exclude a File System Directory from Search

LabKey Server automatically indexes the file system directories associated with projects and folders. It will ignore the contents of directories named ".svn" and ".Trash". To tell the server to not index the content of a directory, you can add a file named ".nocrawl". The content of the file does not matter, an empty text file is sufficient.