ClassAds Introduction

Launch this tutorial in a Jupyter Notebook on Binder:

In this tutorial, we will learn the basics of the ClassAd language, the policy and data exchange language that underpins all of HTCondor. ClassAds are fundamental in the HTCondor ecosystem, so understanding them will be good preparation for future tutorials.

The Python implementation of the ClassAd language is in the classad module:

import classad

Expressions

The ClassAd language is built around values and expressions. If you know Python, both concepts are familiar. Examples of familiar values include: - Integers (1, 2, 3), - Floating point numbers (3.145, -1e-6) - Booleans (true and false).

Examples of expressions are: - Attribute references: foo - Boolean expressions: a && b - Arithmetic expressions: 123 + c - Function calls: ifThenElse(foo == 123, 3.14, 5.2)

Expressions can be evaluated to values. Unlike many programming languages, expressions are lazily-evaluated: they are kept in memory as expressions until a value is explicitly requested. ClassAds holding expressions to be evaluated later are how many internal parts of HTCondor, like job requirements, are expressed.

Expressions are represented in Python with ExprTree objects. The desired ClassAd expression is passed as a string to the constructor:

arith_expr = classad.ExprTree("1 + 4")
print(f"ClassAd arithemetic expression: {arith_expr} (of type {type(arith_expr)})")

ClassAd arithemetic expression: 1 + 4 (of type <class 'classad.classad.ExprTree'>)

Expressions can be evaluated on-demand:

print(arith_expr.eval())

5

Here’s an expression that includes a ClassAd function:

function_expr = classad.ExprTree("ifThenElse(4 > 6, 123, 456)")
print(f"Function expression: {function_expr}")

value = function_expr.eval()
print(f"Corresponding value: {value} (of type {type(value)})")

Function expression: ifThenElse(4 > 6,123,456)
Corresponding value: 456 (of type <class 'int'>)

Notice that, when possible, we convert ClassAd values to Python values. Hence, the result of evaluating the expression above is the Python int 456.

There are two important values in the ClassAd language that have no direct equivalent in Python: Undefined and Error.

Undefined occurs when a reference occurs to an attribute that is not defined; it is analogous to a NameError exception in Python (but there is no concept of an exception in ClassAds). For example, evaluating an unset attribute produces Undefined:

print(classad.ExprTree("foo").eval())

Undefined

Error occurs primarily when an expression combines two different types or when a function call occurs with the incorrect arguments. Note that even in this case, no Python exception is raised!

print(classad.ExprTree('5 + "bar"').eval())
print(classad.ExprTree('ifThenElse(1, 2, 3, 4, 5)').eval())

Error
Error

ClassAds

The concept that makes the ClassAd language special is, of course, the ClassAd!

The ClassAd is analogous to a Python or JSON dictionary. Unlike a dictionary, which is a set of unique key-value pairs, the ClassAd object is a set of key-expression pairs. The expressions in the ad can contain attribute references to other keys in the ad, which will be followed when evaluated.

There are two common ways to represent ClassAds in text. The “new ClassAd” format:

[
  a = 1;
  b = "foo";
  c = b
]

And the “old ClassAd” format:

a = 1
b = "foo"
c = b

Despite the “new” and “old” monikers, “new” is over a decade old. HTCondor command line tools utilize the “old” representation. The Python bindings default to “new”.

A ClassAd object may be initialized via a string in either of the above representation. As a ClassAd is so similar to a Python dictionary, they may also be constructed from a dictionary.

Let’s construct some ClassAds!

ad1 = classad.ClassAd("""
[
  a = 1;
  b = "foo";
  c = b;
  d = a + 4;
]""")
print(ad1)

    [
        a = 1;
        b = "foo";
        c = b;
        d = a + 4
    ]

We can construct the same ClassAd from a dictionary:

ad_from_dict = classad.ClassAd(
{
    "a": 1,
    "b": "foo",
    "c": classad.ExprTree("b"),
    "d": classad.ExprTree("a + 4"),
})
print(ad_from_dict)

    [
        d = a + 4;
        c = b;
        b = "foo";
        a = 1
    ]

ClassAds are quite similar to dictionaries; in Python, the ClassAd object behaves similarly to a dictionary and has similar convenience methods:

print(ad1["a"])
print(ad1["not_here"])

1

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_282/3690994919.py in <module>
      1 print(ad1["a"])
----> 2 print(ad1["not_here"])

KeyError: 'not_here'

print(ad1.get("not_here", 5))

5

ad1.update({"e": 8, "f": True})
print(ad1)

    [
        f = true;
        e = 8;
        a = 1;
        b = "foo";
        c = b;
        d = a + 4
    ]

Remember our example of an Undefined attribute above? We now can evaluate references within the context of the ad:

print(ad1.eval("d"))

5

Note that an expression is still not evaluated until requested, even if it is invalid:

ad1["g"] = classad.ExprTree("b + 5")
print(ad1["g"])
print(type(ad1["g"]))
print(ad1.eval("g"))

b + 5
<class 'classad.classad.ExprTree'>
Error

Onto HTCondor!

ClassAds and expressions are core concepts in interacting with HTCondor. Internally, machines and jobs are represented as ClassAds; expressions are used to filter objects and to define policy.

There’s much more to learn in ClassAds! For now, you have enough background to continue to the next tutorial - HTCondor Introduction.