ClassAds

The ClassAd language consists of two parts: structured data (called “ClassAds”), and expressions.

HTCondor uses ClassAds to describe various things, primarily machines and jobs; it uses expressions as constraints for querying ClassAds, and for defining what it means for two ClassAds to match each other.

Data Syntax

A ClassAd is a list of attribute-value pairs, separated by newlines. Values may be booleans, integers, reals, strings, dictionaries, lists, or the special values UNDEFINED and ERROR. Dictionaries are marked by square brackets and lists by braces; dictionaries separate elements with semicolons, and lists separate elements with commas.

attribute_name  = "attribute-value"
pi              = 3.141
count           = 3
list            = { "red", "green", "blue" }
dictionary      = [ type = "complex"; real = 7.75; imaginary = -3 ]
structured_attr = [ hostnames = { "submit-1", "submit", "submit1" };
                    ip = "127.0.0.1"; port = "9618" ]

For the list of ClassAd attributes generated by HTCondor, see https://htcondor.readthedocs.io/en/latest/classad-attributes/index.html.

Expression Syntax

An expression consists of literals (from the data syntax) and attribute references composed with operators and functions. The value of a ClassAd attribute may be an expression.

MY.count < 10 && regexp( ".*example.*", attribute_name )

Attribute References

An attribute reference always includes an attribute name. In HTCondor, when determining if two ClassAds match, an expression may specify which ad’s value is used by prefixing it with MY. or TARGET.. Attribute references are case-insensitive.

MY.count
TARGET.machine

An element of a dictionary is referenced by using the subscript operator ([]) with an expression that evaluates to a string or with a dot (.), as follows:

MY.structured_attr.hostnames
MY.structured_attr["hostnames"]

Note that the following references the attribute named by the attribute hostnames, not the attribute named hostnames:

MY.structured_attr[hostnames]

List elements are referenced by an expression that evaluates to an integer, where the first element in the list is numbered 0. For example, if colors = { [ x = "1" ], [ x = "2", y = "3" ] }, then MY.structure_attr.colors[0] results in [ x = "1" ].

UNDEFINED and ERROR

The ClassAd language includes two special values, UNDEFINED and ERROR. An attribute may be set to either explicitly, but these values typically result from referring to an attribute that doesn’t exist, or asking for something impossible:

undefined_reference = MY.undefined_attribute
explicitly_undefined = UNDEFINED
one_error_value = "three" * 7
another_error_value = 1.3 / 0

Most expressions that refer to values that are UNDEFINED will evaluate to UNDEFINED. The same applies to ERROR.

Operators

The operators *, /, + and - operate arithmetically, on integers and reals.

The comparison operators ==, !=, <=, <, >= and > operate on booleans, integers, reals and strings. String comparison is case-insensitive. Comparing a string and a non-string value results in ERROR.

The special comparison operator =?= is like == except in the following two ways: it is case-sensitive when comparing strings; and, when comparing values to UNDEFINED, results in FALSE instead of UNDEFINED. (If comparing UNDEFINED to itself, the operator =?= results in TRUE).

The special comparison operator =!= is the negation of =?=.

The logical operators && and || operate on integers and reals; non-zero is true, and zero is false.

The ternary operator x ? y : z operates on expressions.

The default operator x ?: z returns x if x is defined and z otherwise.

The IS and ISNT operators are synonyms for =?= and =!=.

Functions

Function name are case-insensitive. Unless otherwise noted, if any of a function’s arguments are UNDEFINED or ERROR, so is the result. If an argument’s type is noted, the function will return ERROR unless the argument has that type.

integer INT( expr ): If expr is numeric, return the closest integer. If expr evaluates to a string, attempt to convert the string to an integer. Return ERROR if the string is not an integer, or if expr is neither numeric nor a string.

boolean MEMBER( expr, list l ): Returns TRUE if expr is equal, as defined by the operator ==, to any member of the list l, or FALSE if it isn’t.

boolean REGEXP( string pattern, string target[, string options] ): Return TRUE if the PCRE regular expression pattern matches target, or FALSE if it doesn’t. Return ERROR if pattern is a not a valid regular expression. If specified, options is a PCRE option string (one or more of f, i, s, m, and g). See the Specification section for details.

list SPLIT( string s[, string tokens ] ): Separate s by whitespace or comma, or instead by any of the characters in tokens, if specified, and return the result as a list of strings.

boolean STRINGLISTIMEMBER( string s, string list[, string tokens] ): Equivalent to MEMBER( *s*, SPLIT( *list*, *tokens* )).

string SUBSTR( string s, integer offset[, integer length] )

Returns the substring of s from offset to the end of the string, or instead for length characters, if specified. The first character in s is at position 0. If offset is negative, the substring begins offset characters before the end of the string. If length is negative, the substring ends that many characters before the end of the string. If the substring contains no characters, return the empty string. Thus, the following two calls both return the string “78”:

substr( "0123456789", 7, 2 )
substr( "0123456789", -3, -1 )

All ClassAd functions are defined in the references below.

Reserved Words

The words UNDEFINED, ERROR, IS, ISNT, TRUE, FALSE, MY, TARGET, and PARENT may not be used as attribute names.

Testing ClassAd Expressions

Use classad_eval to test ClassAd expressions. For instance, if you want to test to see if a regular expression matches some fixed string, you could check in the following way (on Linux or Mac; the quoting rules are different on Windows):

$ classad_eval 'regexp( ".*tr.*", "string" )'
[ ]
true

This prints out the ClassAd used as context in the evaluation (in this case, there wasn’t one, so it’s the empty ad) and the result.

Examples

These examples assume a Linux shell environment and a working HTCondor pool.

Selecting a Slot based on Job ID

If job 288.7 is running:

$ condor_status -const 'JobId == "288.7"'

Selecting Jobs based on Execute Machine

If jobs are running on the machine example-execute-node:

$ condor_q -all -const 'regexp( "@example-execute-node$", RemoteHost )'

String Manipulation

In this example, an administrator has just added twelve new hosts to the pool – compute-296 to compute-307 – and wants to see if they’ve started running jobs yet.

$ condor_status -const '296 <= int(substr( Machine, 8 )) && int(substr( Machine, 8 )) <= 307'

You could also write this as follows:

$ condor_status -const '296 <= int(split(Machine, "-")[1]) && int(split(Machine, "-")[1]) <= 307'

Selecting Machines with a Particular File-Transfer Plugin

If you’re considering using the gdrive file-transfer plugin, and you’d like to see which machines have it, select from the slot ads based on the corresponding attribute, but only print out the machine name, and then throw away the duplicates:

$ condor_status -af Machine \
    -const 'StringListIMember( "gdrive", HasFileTransferPluginMethods )' \
    | uniq

You could instead use a constraint to ignore dynamic slots for a report on the resources available to run jobs which require the gdrive plugin. Note that you can also use expressions when formatting the output. In this case, it’s just to make the output prettier.

$ condor_status -af Machine CPUs Memory Disk \
    '(GPUs =!= undefined && GPUs >= 1) ? CUDACapability : "[no GPUs]"' \
    -const 'SlotType =!= "Dynamic" && StringListIMember( "gdrive", HasFileTransferPluginMethods )'

Specification

For use in HTCondor, including a complete list of functions, see https://htcondor.readthedocs.io/en/latest/classads/classad-mechanism.html.

For the language specification, see https://research.cs.wisc.edu/htcondor/classad/refman/.

Author

Center for High Throughput Computing, University of Wisconsin-Madison