XQuery/Advanced Search

Motivation
You have multiple fields that you would like to search on. You want to allow users to optionally search on specific fields and perform a boolean "AND" when multiple fields are used.

For example you may have a database of people. Each person has a first name, last name, e-mail and phone. You want to allow users to search on any single field or multiple fields together. If two fields are entered only records that match both fields will be returned.

Method
We will use a standard HTML form with multiple input and selection fields. We will check each incoming search request for each parameter and if the parameter is not null we will concatenate a single query with many predicates and then evaluate it using the util:eval function.

Example XML Data Set
In the following example we will use an XML file that contains list of people. The format will be the following:

Background on Predicates
If you have a single "where clause" (called a predicate) you can always place this predicate to the end of an XPath expression. For example the following FLWOR expression will return all person records in the system:

for $person in collection('/db/apps/directory')//person return $person

You can now restrict this to only include faculty by adding a predicate:

for $person in collection('/db/apps/directory')//person[type='faculty'] return $person

You can now search for all faculty with a first name of "mark" buy just adding an additional predicate:

for $person in collection('/db/apps/directory')//person[type='faculty'][firstName='mark'] return $person

Sample HTML code for advanced search form
The following is an HTML form section for this form.

When the user adds a name of "John" to the first name field and selects a type of "staff" and then the submit query button is pressed the following is an example of the URL created by this form:

advanced-search.xq?firstname=John&lastname=&email=&phone=&type=staff&Submit=Submit+Query

Note that most of the fields are null. Only firstname and type have a value to the right of the equal sign.

Sample Search Service
The search service will have the following code sections.

Getting the URL parameters
The following code fragment will get the URL parameters from the incoming URL request and assign them to XQuery variables.

Note that each of the incoming parameters is first converted to lowercase before any comparisons are done.

Building the Predicate Strings
We are now ready to start building our predicates. Since many of the fields will be empty we will only construct a predicate if the variable exists.

For the firstname, lastname, and email we are comparing the incoming parameter with the lowercase string in the XML file. With the phone number we are using the contains function to return all records that have a string somewhere in the phone number. The type is using an exact match since both the case of the data and keyword are known precisely.

The most challenging aspect of this program is learning how to get the order of the quotes correct. In general I use single quotes for enclosing static strings unless that string itself must contain a single quote. Then we use double quotes. The most difficult part is to assemble a string such as [type = 'staff'] and to remember to put the single quotes around the word staff. If you can figure this out the rest will be easy.

If you are having trouble you can also break the concat into multiple lines:

concat(     '[type',      " = '",      $type,      "']"   )

Where each line clearly must start and end with the same type of quote.

Concatenating the Query
To create an eval string we just need to create a single long string starting with the collection and add each of the predicates. If there was no argument the predicate strings will be null.

The query with just a lastname and type would then look like this:

collection('/db/apps/directory/data')//person[lower-case(firstname) = 'John'][lower-case(type) = 'faculty']

Note that some advanced system will modify the order of the predicates based on the most likely to narrow the search. Since there are fewer records with the first name John than there are faculty it is always more efficient to put the first name before the type. This means that fewer nodes need to be moved from hard disk into RAM and the query will execute much faster.

Executing the Query
The execution of the query is done by passing the eval string to the util:eval function. let $persons := util:eval($eval-string)

Displaying the Results
We are now ready to display all of the results. We do this by creating a FLWOR statement for each person and returning a  element for each hit. The  elements each have single link with the lastname, firstname and type as the link content. When the user clicks on each link an item viewer is used and the ID of the person is passed to the item viewer.

NGram Searching
In your conf.xml module make sure the following line is uncommented:

Here is the page on NGram elements in your collection.xconf file:

NGram Configuration File

After you edit then reindex.

You can now use any of the following functions:

NGram Functions

Acknowledgments
This example has been provided by Eric Palmer and his staff at the University of Richmond, USA.