Column filters

Column filters is a mechanism that allows to limit displayed data by only including the rows that meet criteria that you specify and hides rows that you do not want displayed. Criteria can be related to values in one or more columns, hence the column filter term. After you filter data, you can browse the resulting rows like before. Adding, editing or removing filters does not affect the actual data, it only limits amount of data displayed.

Table of Contents

Column filters

Creating basic column filters

Basic filter can be created "by example" using context menu or using the Column Filters Editor window for given tabular view.

Figure 1. Column filter consisted of two rules created for the Peaks table of a Byomap document

This filter contains two rules:

"Show rows having Apex Time values larger than 42.80 and having Normed area % values smaller than 8.0 %.

Rules of basic filters are always combined using the conjunction (AND). For given row all the rules must be evaluated successfully or else the row will be filtered out. To overcome this limitation so OR, negation and more complex logical rules can be supported, filters can be defined using scripting. See the Script column filters section for more information.

Supported basic filter operators

=, ≠, <, ≤, >, ≥ operators

These operators require one argument of type that matches column type. Text matching is case-sensitive. For floating point numbers comparisons are limited to number of significant digits currently specified for given column.

These are well-known math operators, as follows:

Operator	Description
`=`	Equal to the compared value
`≠`	Not equal to the compared value
`<`	Less than the compared value
`≤`	Less or equal than the compared value
`>`	Greater than the compared value
`≥`	Greater or equal than the compared value

"Is empty", "Is not empty" operators

These operators require no argument and can be used with any type.

Operator	Description
`Is empty`	Accepts values that are empty (they are empty strings or the value is missing)
`Is not empty`	Accepts values that are not empty (they are not empty strings or the value is of other type exists)

"Starts with", "Does not start with", "Ends with", "Does not end with", "Contains", "Does not contain" operators

These operators require one argument of text type. Before matching, values are first converted to text if the type is not text. Text matching is case-insensitive.

Operator	Description
`Starts with`	Accepts values having textual representation starting with given text
`Does not start with`	Accepts values having textual representation not starting with given text
`Ends with`	Accepts values having textual representation ending with given text
`Does not end with`	Accepts values having textual representation not ending with given text
`Contains`	Accepts values having textual representation containing given text
`Does not contain`	Accepts values having textual representation not containing given text

"Like", "Not like" operators

These operators require one textual argument that is a wildcard pattern. Text matching is case-insensitive.

Operator	Description
`Like`	Accepts values having textual representation matching specified wildcard pattern
`Not like`	Accepts values having textual representation not matching specified wildcard pattern

Format of the pattern is similar to command shells such as GNU bash or cmd.exe. Following rules apply:

Any character (other than special characters mentioned below) represents itself. For example c matches the character c.
? character matches any single character. For example the pattern a?c matches texts abc and axc but not ac.
\* character matches zero or more of any characters. For example the pattern a*c matches texts abc and axc and ac.
[...] represents set of characters. Backslash character has no special meaning here. For example the pattern a[bx]c matches texts abc and axc but not azc.

The backslash character \ escapes special characters ?, *, [ and ] and \ itself. For example \? adds the ? character to the pattern.

"Matches", "Does not match" operators

These operators require one textual argument that is a regular expression pattern. Regular expressions are more complex but also more powerful than the wildcard patterns of the Like operator. Text matching is case-insensitive.

Format of the regular expression pattern is based on so called Perl-compatible regular expressions (pcre). Detailed information can be found in the Perl's regular expression documentation and the Perl's regular expression tutorial. Following are most frequently used rules:

Pattern consisted of a string of ordinary characters matches texts is equal to this string. For example the pattern bc matches text bc and BC but not bcd.
Special characters (or meta-characters) are: { } [ ] ( ) ^ $ . | * + ? - \. They can't be used as-is in a match but can be used by putting a backslash before them. For example the pattern abc\? matches text abc?.
Non-printable characters are represented by escape sequences. Common examples: \t for a tab, \n for a newline. A sequence of arbitrary bytes can be specified using the octal escape sequence, e.g., \033 or hexadecimal escape sequence, e.g., \x1B.
Anchor meta-characters ^ and $ that force matching at the beginning or end of the text cannot be used because they are added automatically to the pattern.
Character class is a set of allowed characters to match at a particular point of the pattern. Character classes are denoted by using brackets [...]. For example the pattern [bcr]at matches texts bat, CAT and Rat but not brat.
The special character - acts as a range operator within character classes, so that a contiguous set of characters can be written as a range so the pattern can be shortened. For example the pattern [012345] is the same as [0-5] and the pattern [abcde] is the same as [a-e].
The special character ^ in the first position of a character class denotes a negated character class, which matches any character but those in the brackets. For example the pattern [^ab]at matches Rat and CAT but not match text aat or Bat.
The ^ character can be used with range operators. For example the pattern [^0-9] matches a single non-numeric character.
To save keystrokes and make patterns more readable there are several abbreviations for common character classes (can be used both inside and outside of [...]):
\d matches a single digit (so it is equivalent of [0-9]). For example the pattern \\d\\d:\\d\\d:\\d\\d matches texts in hh:mm:ss time format.
\s matches a whitespace character, the set [ \t\r\n\f] and others. For example the pattern [\d\s] matches any digit or whitespace character.
\w matches a word character (alphanumeric or _), not just [0-9a-zA-Z_] but also digits and non-roman characters.
\D is a negated \d; it represents any character other than a digit, or [^\d].
\S is a negated \s; it represents any non-whitespace character [^\s].
\W is a negated \w; it represents any non-word character [^\w].
The period . matches any character but \n. For example the pattern ..rt matches any two characters followed by rt. To match real period character it needs to be escaped with slash (.`).
\b is a word anchor; it matches a boundary between a word character and a non-word character, \w\W or \W\w. Start and end of text are also considered to be the boundary. For example the pattern \bcat\b matches text Housecat catenates house and cat (the matching fragment is the last word).
The alternation metacharacter | can be used to match different possible words or character strings. For example the pattern dog|cat matches dog and cat texts. The a|b|c and [abc] patterns are equivalent.
Grouping metacharacters () allows parts of a pattern to be treated as a single unit. For example the pattern hot(cat|dog) matches hotcat and hotdog texts. Alternatives can include empty words, for example the pattern hot(dog|) matches hotdog and hot texts.
If groups are combined, all possible combinations are checked until there is a match. For example the pattern (ab|cd)(12|34) matches ab12, ab34, cd12, and cd34 texts.
The special quantifier metacharacter * can be used to match the preceding element (character or group) 0 or more times. For example the pattern ab*c matches texts abc and abbc but also ac.
The special quantifier metacharacter + can be used to match the preceding element (character or group) 1 or more times. For example the pattern ab+c matches texts abc and abbc but not ac.
The special quantifier metacharacter ? can be used to match the preceding element (character or group) 0 or one time. For example the pattern ab?c matches texts ac and abc but not abbc.
The special quantifier metacharacter {n} can be used to match the preceding element (character or group) exactly n times. For example the pattern (ha){3} matches exactly one text hahaha.
The special quantifier metacharacter {n,} can be used to match the preceding element (character or group) at least n times. For example the pattern (ha){3,} matches text hahaha but also hahahaha and so on.
The special quantifier metacharacter {n,m} can be used to match the preceding element (character or group) at least n times but not more than m times. For example the pattern (ha){3,4} only matches texts hahaha and hahahaha.

Script column filters

Script column filters is an extension of the basic filters. Instead of creating rules using the graphical interface, user programs the filters using the JavaScript language, programming language known from the Web. Basic knowledge of the language is useful but ability of writing mathematical expressions is often sufficient to build relatively advanced script filters.

Following is a script code that is equivalent of the two rules from the Creating basic column filters section.

return Column.value("Apex Time") > 42.80 && Column.value("Normed area %") < 8.0

&& means AND, Column.value provides value for a column (column is pointed by name). Type of both columns is numerical so their values are compared to numeric constants using comparison operators < and >.

Requirements for the script column filters

Script column filter is a program in JavaScript language. All parts of the script have to construct a valid program, both in terms of JavaScript syntax and used functions and constants. One or more syntax or name errors results in an invalid script which does not work as a column filter. The script can also use JavaScript-compatible comments.

Example: This script gives a name error because abc is undeclared identifier:

return abc

Example: This script gives a syntax error because return keyword is misused:

if return

Example: This script gives a runtime error assuming that the "Foobar" column is not present:

return Column.value("Foobar") > 0

Script column filter is a program that returns boolean value. The program returns a boolean value by returning a true or false constant or more complex expression that evaluates to a boolean constant. This means that only expressions that are evaluated to a true or false value are valid expressions. Empty expression is invalid. Returning is performed using the standard JavaScript return keyword. If this keyword is missing, entire script column filter is invalid.

Example: valid script:

return false;

Example: invalid, missing return keyword:

true

Meaning of invalid column filter script programs. Applying invalid script program to a table of data results with filtering-out entire data. Invalid programs are thus equivalents of a program that always returns false value. In addition, applications can show approximate location of error within text of the invalid script and reason of error.

Blocks of commands. The requirement of returning a boolean value does not mean that scripts have to only contain a single boolean expression. Column filter script can be based on any valid JavaScript program, that is, block of commands, assuming that correct return command is present on the end.

Example script consisted of a block of commands:

var width = 150;
var height = 20;
var area = 15  *20;
return Column.value("Area") > area; // assuming the "Area" column exists, it's a valid program

Debugging functions for script column filters

Column filters presets

Creating column filters and adjusting their parameters can be time-consuming. To preserve the work applications offer presets for the column filters defined by the user.

Single preset stores one or more column filters of any kind. There are two locations where presets can be stored:

current document
file

Presets that are saved in the current document have names (descriptions) that can be any non-empty text. Presets that are exported to files have names that are just filenames. Extension for a preset file is cft.

To access functions of Presets click the bottom-left Presets button in the Column Filters Editor window. Supported commands are:

Load from document, a sub-menu that lists presets recently saved for given tabular view in the current document. Clicking one of the items loads the preset. Existing filters in the Column Filters Editor will be replaced with the filters loaded from the preset.
Manage presets, a command in the Load from document sub-menu, that shows dialog window providing a list of presets saved so far for given tabular view of the current document. It shows name, creation date and author who originally saved the preset. In this window names of presets can be changed, selected presets can be deleted from document, and one selected preset can be loaded.
Import from file, a sub-menu that lists presets recently exported to files for given tabular view of the current document. Clicking one of the listed items imports the preset to the Column Filters Editor. Existing filters in the dialog will be replaced with the filters imported from the preset file. In addition there is a Select file... command in the same sub-menu that allows to pick a filename to import that is not displayed on the recent list.
Save to document, a sub-menu that, like Load from document, lists presets recently saved for given tabular view in the current document. Clicking one of the items saves current set of filters from the Column Filters Editor to the current document. The same sub-menu offers a Save with new name... command that allows to enter new name for a preset before saving it to the document. Finally, in the same sub-menu there is a Manage presets command. It is there for convenience and is a duplication of the same command from the Load from document sub-menu.
Export from file, a sub-menu that lists presets recently exported to files for given tabular view of the current document. Clicking one of the listed items exports filters from the Column Filters Editor to file with given name. Existing file will be replaces by a new content. Application will not ask about overwriting. In addition there is a Export... command in the same sub-menu that allows to pick a filename to export that is not displayed on the recent list. Application will ask for overwriting the file in this case.