Migrating From Version 1

When switching from version 1 of the HTCondor Python bindings to version 2, it should mostly just work to do the following:

import classad2 as classad
import htcondor2 as htcondor

The goal of this guide is to help when it doesn’t. In general, code that you haven’t (but need to) migrate will simply raise an exception of some kind.

Warning

There is an incompability between classad and classad2 that won’t manifest itself as an exception. You should check your code for calls to eval() and simplify() that don’t specify a positional parameter (the scope): classad2.ExprTree no longer records which classad2.ClassAd, if any, it was extracted from. Calls to eval() or simplify() without an argument may therefore return different values (likely None).

This change was required for proper memory management.

You should otherwise be able to search this document for the method or class mentioned in the exception.

The classad2 Module

Building New ExprTrees

The big difference between the version 1 classad API and the classad2 API is that the version 2 module does not allow construction or modification of ExprTree objects. For example, the following is no longer supported:

>>> import classad
>>> seven = classad.ExprTree("7")
>>> eight = classad.ExprTree("8")
>>> fifteen = seven + eight
>>> print(fifteen)
7 + 8
>>> print(fifteen.eval())
15

You could instead write the following:

>>> import classad2
>>> seven = classad2.ExprTree("7")
>>> eight = classad2.ExprTree("8")
>>> fifteen = classad2.ExprTree(f"{seven} + {eight}")
>>> print(fifteen)
7 + 8
>>> print(fifteen.eval())
15

Classes or methods only in Version 1:

classad.register()

This method was dropped; as far as we know, no one was using it.

Classes or methods only in Version 1:

The htcondor Module

Job Submission

Note

Oops, we changed it again.

The widest-ranging change to the htcondor2 API is the elimination of all of the previously-deprecated submission-related functionality, and the correction and expansion of the remaining method, htcondor2.Schedd.submit(). The new submit method only takes htcondor.Submit objects; you may no longer submit via raw ClassAds. However, the Submit object now supports all submit files, and the submit() method respects any queue statements in the Submit object unless you specify otherwise.

Submitting jobs now looks something like this:

import htcondor2

submit = htcondor2.Submit("""
    executable = my_prog
    arguments = data_file $(SCENARIO) --seed $(SEED)
    transfer_input_files = data_file $(SCENARIO)
    transfer_output_files = $(SCENARIO).result

    request_cpus = 1
    request_memory = 4096

    out = $(SCENARIO).out
    err = $(SCENARIO).err
    log = my_prog.log

    queue 1 SCENARIO matching file scenarios/*
""")
submit["SEED"] = "0xDEADBEEF"

schedd = htcondor2.Schedd()
result = schedd.submit(submit)

Note that the submit-language variable SEED was specified as a a string. The HTCondor submit language (which is not the ClassAd language) only understands strings, so in version 2, the Submit object no longer attempts to translate them for you.

Classes or methods only in Version 1:

Unimplemented Methods

These methods are not presently implemented in the version 2 bindings, but we will consider implementing them upon request. We generally hope that they are no longer necessary.

Classes or methods only in Version 1:

Unimplemented Classes

htcondor.SecMan is not presently implemented. Because htcondor.Token was only used by SecMan, it has also been removed.

In many cases, you can replace SecMan by setting the appropriate HTCondor configuration values. For example, the following does a remote submit using the IDTOKEN stored in the /home/myuser/tokens directory.

import htcondor2

# The default value is ~/.condor/tokens.d, so you can skip this
# if the token you want to use is on disk there.  Note that on-
# disk tokens must be readable only by the owner.
htcondor2.param["SEC_TOKEN_DIRECTORY"] = "/home/myuser/tokens"

coll = htcondor2.Collector("cm.pool.tld")
hostname_job = htcondor2.Submit({
    "executable": "/bin/hostname",  # the program to run on the execute node
    "output": "hostname.out",       # anything the job prints to standard output will end up in this file
    "error": "hostname.err",        # anything the job prints to standard error will end up in this file
    "log": "hostname.log",          # this file will contain a record of what happened to the job
    "request_cpus": "1",            # how many CPU cores we want
    "request_memory": "128MB",      # how much memory we want
    "request_disk": "128MB",        # how much disk space we want
    "transfer_input_files": "hello.txt",
    })

sad = coll.locate(htcondor2.DaemonType.Schedd, "remote-schedd.pool.tld")
schedd = htcondor2.Schedd(sad)
res = schedd.submit(hostname_job, spool=True)
schedd.spool(res)

htcondor.TokenRequest is not presently implemented.

Classes or methods only in Version 1:

Deprecated Classes or Methods Now Removed

We removed the deprecated method htcondor.Schedd.xquery(); use htcondor2.Schedd.query() instead. This removed the need for its dedicated return type, htcondor.QueryIterator, and for the htcondor.poll() method and its return type, htcondor.BulkQueryIterator, and for the latter’s dedicated enumeration, htcondor.BlockingMode.

Classes or methods only in Version 1:

Other Missing Classes or Methods

htcondor.HistoryIterator has been replaced as the return type of htcondor2.Schedd.history() with List[ClassAd].

htcondor.QueueItemsIterator has been replaced as the return type of htcondor2.Submit.itemdata() with Iterator[List[str]].

htcondor.VacateTypes has been removed, as it was never used in a documented interface.

htcondor.CredStatus was never fully documented and has been removed, as it was was never used in a documented interface.

Classes or methods only in Version 1: