Version 9.0 LTS Releases

These are Long Term Support (LTS) releases of HTCondor. As usual, only bug fixes (and potentially, ports to new platforms) will be provided in future 9.0.y releases. New features will be added in the 9.x.y feature releases.

The details of each version are described below.

Version 9.0.17

Release Notes:

  • HTCondor version 9.0.17 released on September 29, 2022.

New Features:

  • Increased the length of the password generated for Windows default slot user accounts from 14 characters to 32 characters, and added some code to ensure that complexity measures that look at character set and not length will still be satisfied. (HTCONDOR-1232)

  • Added -debug option to condor_drain tool. (HTCONDOR-1236)

  • Removed support from the condor_startd for querying keyboard and mouse idle time, on legacy x86 Linux machines that used an 8042 keyboard controller. This caused significant performance degradation in the condor_startd on machines with many CPUs. (HTCONDOR-1297)

Bugs Fixed:

  • Fixed a bug that would cause the condor_schedd to leak file descriptors, eventually run out, and crash, when unable to launch the scheduler universe job for any reason. (HTCONDOR-1261)

  • When a failure occurs with a grid universe job of type batch, the local job is now always put on hold, instead of the remote job being canceled and automatically resubmitted. (HTCONDOR-1226)

  • Job attribute GridJobId is no longer altered for batch grid universe jobs when the job enters Removed status. (HTCONDOR-1224)

  • Fixed a bug where forwarding a refreshed X.509 proxy for a batch grid universe job would fail. (HTCONDOR-1222)

  • Fixed a bug where DAGMan would fail when the keyword DONE was added to the JOB line in a DAG input file. (HTCONDOR-1267)

  • Fixed a bug where the FS and MUNGE authentication methods would treat local user accounts with very large UID values (greater than 2^31) as the condor user. (HTCONDOR-1229)

  • Fixed a bug with the condor_credmon_oauth where scope and audience claims were dropped from OAuth refresh tokens on their first renewal. (HTCONDOR-1270)

  • Added the appropriate Python cryptography package as a dependency to the condor-credmon-oauth RPM package. (HTCONDOR-1279)

  • Fixed bugs with creation of a job manifest; see the manifest job submit command in the condor_submit man page. (HTCONDOR-1350)

  • If “Singularity” is really the “Apptainer” runtime, HTCondor now sets environment variables to be passed to the job appropriately, which prevents Apptainer from displaying ugly warnings about how this won’t work in the future. (HTCONDOR-1137)

Version 9.0.16

Release Notes:

  • HTCondor version 9.0.16 released on August 16, 2022.

New Features:

  • Singularity jobs now mount /tmp and /var/tmp under the scratch directory, not in tmpfs. (HTCONDOR-1193)

Bugs Fixed:

  • Fixed a bug where if the submit file set checkpoint_exit_code, and the administrator enabled Singularity support on the execute node, the job would go on hold at checkpoint time. (HTCONDOR-837)

  • Fixed a bug where the condor_gridmanager would delete the job’s X.509 proxy file when it meant to delete a temporary copy of the proxy file. (HTCONDOR-1223)

  • Fixed a file descriptor leak when using SciTokens for authentication. (HTCONDOR-1188)

  • Fixed a bug on Windows that caused a misleading error message about the SharedPortEndpoint when a daemon exits. (HTCONDOR-1178)

  • Fixed a bug where the condor_check_config tool raised an UnboundLocalError due to an undefined variable. (HTCONDOR-1186)

  • Fixed a bug in condor_gpu_discovery which would cause the tool to crash when OpenCL devices were detected and GPU_DEVICE_ORDINAL was set in the environment. (HTCONDOR-1191)

  • Fix a bug that could cause daemons to crash if their log rotates during shutdown. (HTCONDOR-1200)

  • Fixed a bug where the condor_starter would wait forever for a reconnect from the condor_shadow if a network failure occurred during cleanup after the job completed. (HTCONDOR-1213)

  • The condor-credmon-oath package now properly pulls in python3-mod_wsgi on Enterprise Linux 8. (HTCONDOR-1217)

Version 9.0.15

Release Notes:

  • HTCondor version 9.0.15 released on July 21, 2022.

New Features:

  • For batch grid universe jobs, report resources provisioned by the batch scheduler when available. (HTCONDOR-1199)

Bugs Fixed:

  • None.

Version 9.0.14

Release Notes:

  • HTCondor version 9.0.14 released on July 12, 2022.

New Features:

  • Made SciTokens mapping failures more prominent in the daemon logs. (HTCONDOR-1072)

  • The manual page, usage and logging of the condor_set_shutdown tool has been improved to clarify what this tool does and how to use it. (HTCONDOR-1102)

Bugs Fixed:

  • Fixed a bug where if a job’s output and error were directed to the same file, no other output files would be transferred. (HTCONDOR-1101)

  • Ensure that the matching set of Python bindings is installed when HTCondor is upgraded on RPM based platforms. (HTCONDOR-1127)

  • Fixed a bug that caused $(OPSYSANDVER) to expand to nothing in a JOB_TRANSFORM. (HTCONDOR-1121)

  • Fixed a bug in the Python bindings that prevented context managed htcondor.SecMan sessions from working. (HTCONDOR-924)

  • Fixed a bug where RemoteUserCpu and RemoteSysCpu attributes are occasionally set to 0 for successfully completed jobs. (HTCONDOR-1162)

  • Make condor-externals package dependency less strict to ease transition between CHTC and OSG RPM repositories. (HTCONDOR-1177)

Version 9.0.13

Release Notes:

  • HTCondor version 9.0.13 released on May 26, 2022.

New Features:

  • If the configuration macro [SCHEDD|STARTD]_CRON_LOG_NON_ZERO_EXIT is set to true, the corresponding daemon will write the cron job’s non-zero exit code to the log, followed by the cron job’s output. (HTCONDOR-971)

  • condor_config_val will now print an @=end/@end pair rather than simply = when printing multi-line configuration values for -dump, -summary, and -verbose mode output. (HTCONDOR-1032)

  • Add a SEC_CREDENTIAL_STORECRED_OPTS variable to condor_vault_storer to enable sending additional options to every condor_store_cred command. (HTCONDOR-1091)

  • Recognize the new format of vault tokens, beginning with hvs. in addition to the old format beginning with s. . (HTCONDOR-1091)

Bugs Fixed:

  • The condor_run tool now reports job submit errors and warnings to the terminal rather than writing them into a log file. (HTCONDOR-1002)

  • Fixed a bug where Kerberos Authentication would fail for DAGMan. (HTCONDOR-1060)

  • Fix problem that can cause HTCondor to not start up when the network configuration is complex. Long hostnames, multiple CCB addresses, having both IPv4 and IPv6 addresses, and long private network names all contribute to complexity. (HTCONDOR-1070)

  • Updated the Windows build of HTCondor to use SSL 1.1.1m. (HTCONDOR-840)

Version 9.0.12

Release Notes:

  • HTCondor version 9.0.12 released on April 19, 2022.

New Features:

  • None.

Bugs Fixed:

  • Fixed a bug in the parallel universe that caused the condor_schedd to crash with partitionable slots. (HTCONDOR-986)

  • Fixed a bug that could cause a daemon to erase its security session to its family of daemon processes and subsequently crash when trying to connect to one of those daemons. (HTCONDOR-937)

  • Fixed a bug that prevented the High-Availability Daemon (HAD) from working when user-based security is enabled. (HTCONDOR-891)

  • In a HAD configuration, the negotiator is now more robust when trying to update to collectors that may have failed. It will no longer block and timeout for an extended period of time should this happen. (HTCONDOR-816)

  • The Job Router no longer sets an incorrect User job attribute when routing a job between two condor_schedd s with different values for configuration parameter UID_DOMAIN. (HTCONDOR-1005)

  • Fixed a bug in the condor_startd drain command in the Python bindings that prevented it from working with zero arguments. (HTCONDOR-936)

  • Fixed a bug that prevented administrators from setting certain rare custom Linux parameters in the linux_kernel_tuning_script. (HTCONDOR-990)

  • DAGMan now publishes its status (total number of nodes, nodes done, nodes failed, etc.) to the job ad immediately at startup. (HTCONDOR-968)

  • Fixed a bug where a credential file with an underscore in its filename could not be used by the curl plugin when doing HTTPS transfers with a bearer token. It can now be accessed by replacing “_” with “.” in the URL scheme. (HTCONDOR-1011)

  • Fixed several unlikely bugs when parsing the time strings in ClassAds. (HTCONDOR-998)

  • condor_version now reports the build ID on Debian and Ubuntu platforms. (HTCONDOR-749)

Version 9.0.11

Release Notes:

  • HTCondor version 9.0.11 released on March 15, 2022.

New Features:

  • The condor_job_router can now create an IDTOKEN and send it them along with a routed job for use by the job. This is controlled by a new configuration variable JOB_ROUTER_CREATE_IDTOKEN_NAMES and a new route option SendIDTokens. (HTCONDOR-735)

Bugs Fixed:

  • HTCondor will now properly transfer checkpoints if stream_output or stream_error is set and output or error, respectively, is not an absolute path. (HTCONDOR-736)

  • A problem where HTCondor would not create a directory on the execute node before trying to transfer a file into it should no longer occur. (This would cause the job which triggered this problem to go on hold.) One way to trigger this problem was by setting preserve_relative_paths and specifying the same directory in both transfer_input_files and transfer_checkpoint_files. (HTCONDOR-857)

  • The condor_annex tool no longer duplicates the first tag if given multiple -tag options on the command line. You can now set longer user data on the command-line. (HTCONDOR-910)

  • Fixed a bug in the condor_job_router that could result in routes and transforms substituting a default configuration value rather than the value from the configuration files when a route or transform was applied (HTCONDOR-902)

  • For batch grid universe jobs, a small default memory value is no longer generated when request_memory is not specified in the submit file. This restores the behavior in versions 9.0.1 and prior. (HTCONDOR-904)

  • Fixed a bug in the FileTransfer mechanism where URL transfers caused subsequent failures to report incorrect error messages. (HTCONDOR-915)

  • Fixed a bug in the condor_dagman parser which caused SUBMIT-DESCRIPTION statements to return an error even after parsing correctly. (HTCONDOR-928)

  • Fix problem where condor_ssh_to_job may fail to connect to a job running under an HTCondor tarball installation (glidein) built from an RPM based platform. (HTCONDOR-942)

  • The Python bindings no longer segfault when the htcondor.Submit constructor is passed a dictionary with an entry whose value is None. (HTCONDOR-950)

Version 9.0.10

Release Notes:

  • HTCondor version 9.0.10 released on March 15, 2022.

New Features:

  • None.

Bugs Fixed:

Version 9.0.9

Release Notes:

  • HTCondor version 9.0.9 released on January 13, 2022.

  • Since CentOS 8 has been retired, we now build for Enterprise Linux 8 on Rocky Linux 8. (HTCONDOR-911)

  • Debian 11 (bullseye) has been added as a supported platform. (HTCONDOR-94)

New Features:

  • The OAUTH credmon is packaged for the Enterprise Linux 8 platform. (HTCONDOR-825)

Bugs Fixed:

  • When a grid universe job of type condor fails on the remote system, the local job is now put on hold, instead of automatically resubmitted. (HTCONDOR-871)

  • Fixed a bug where a running parallel universe job would go to idle status when the job policy indicated it should be held. (HTCONDOR-869)

  • Fixed a bug running jobs in a Singularity container where the environment variables added by HTCondor could include incorrect pathnames to the location of the job’s scratch directory. This occurred when setting the SINGULARITY_TARGET_DIR configuration option. (HTCONDOR-885)

  • Fixed a bug where the condor_job_router could crash while trying to report an invalid router configuration when C-style comments were used before an old syntax route ClassAd. As a result of this fix the job router now treats C-style comments as a indication that the route is old syntax. (HTCONDOR-864)

  • Fixed a bug where binary bytes were trying to be written via an ASCII file handler in condor_credmon_oauth when using Python 3. (HTCONDOR-633)

  • Fixed a bug in condor_top where two daemon ClassAds were assumed to be the same if some specific attributes were missing from the latest ClassAd. Also condor_top now exits early if no stats are provided by the queried daemon. (HTCONDOR-880)

  • Fixed a bug where the user job log could be written in the wrong directory when a spooled job’s output was retrieved with condor_transfer_data. (HTCONDOR-886)

  • Fixed a bug in condor_adstash where setting a list of condor_startds to query in the configuration lead to no condor_startds being queried. (HTCONDOR-888)

Version 9.0.8

Release Notes:

  • HTCondor version 9.0.8 released on December 2, 2021.

New Features:

  • None.

Bugs Fixed:

  • Fixed a bug where very large values of ImageSize and other job attributes that have _RAW equivalents would get rounded incorrectly, and end up negative. (HTCONDOR-780)

  • Fixed a bug with the handling of MAX_JOBS_PER_OWNER in the condor_schedd where it was treated as a per-factory limit rather than as a per-owner limit for jobs submitted with the max_idle or max_materialize submit keyword. (HTCONDOR-755)

  • Fixed a bug in how the condor_schedd selects a new job to run on a dynamic slot after the previous job completes. The condor_schedd could choose a job that requested more disk space than the slot provided, resulting in the condor_startd refusing to start the job. (HTCONDOR-798)

  • Fixed daemon log message that could allow unintended processes to use the condor_shared_port service. (HTCONDOR-725)

  • Fixed a bug in the ClassAds function substr() that could cause a crash if the offset argument was out of range. (HTCONDOR-823)

  • Fixed bugs in the Kerberos authentication code that cause a crash on macOS and can leak memory. (HTCONDOR-200)

  • Fixed a bug where if condor_schedd fails to claim a condor_startd, the job matched to that condor_startd won’t be rematched for up to 20 minutes. (HTCONDOR-769)

Version 9.0.7

Release Notes:

  • HTCondor version 9.0.7 released on November 2, 2021.

New Features:

  • The configuration parameter SEC_TOKEN_BLACKLIST_EXPR has been renamed to SEC_TOKEN_REVOCATION_EXPR. The old name is still recognized if the new one isn’t set. (HTCONDOR-744)

Bugs Fixed:

  • condor_watch_q no longer has a limit on the number of job event log files it can watch. (HTCONDOR-658)

  • Fix a bug in condor_watch_q which would cause it to fail when run on older kernels. (HTCONDOR-745)

  • Fixed a bug where condor_gpu_discovery could segfault on some older versions of the nVidia libraries. This would result in GPUs not being detected. The bug was introduced in HTCondor 9.0.6 and is known to occur with CUDA run time 10.1. (HTCONDOR-760)

  • Fixed a bug that could crash the condor_startd when claiming a slot with p-slot preemption. (HTCONDOR-737)

  • Fixed a bug where the NumJobStarts and JobCurrentStartExecutingDate job ad attributes weren’t updated if the job began executing while the condor_shadow and condor_starter were disconnected. (HTCONDOR-752)

  • Ensure the HTCondor uses version 0.6.2 or later SciTokens library so that WLCG tokens can be read. (HTCONDOR-801)

Version 9.0.6

Release Notes:

  • HTCondor version 9.0.6 released on September 23, 2021.

New Features:

  • Added a new option -log-steps to condor_job_router_info. When used with the -route-jobs option, this option will log each step of the route transforms as they are applied. (HTCONDOR-578)

  • The stdin passed to condor_job_router hooks of type _TRANSLATE_JOB will now be passed information on the route in a format that is the same as what was passed in 8.8 LTS releases. It will always be a ClassAd, and include the route Name as an attribute. (HTCONDOR-646)

  • Added configuration parameter AUTH_SSL_REQUIRE_CLIENT_CERTIFICATE, a boolean value which defaults to False. If set to True, then clients that authenticate to a daemon using SSL must present a valid SSL credential. (HTCONDOR-236)

  • The location of database files for the condor_schedd and the condor_negotiator can now be configured directly by using the configuration variables JOB_QUEUE_LOG and ACCOUNTANT_DATABASE_FILE respectively. Formerly you could control the directory of the negotiator database by configuring SPOOL but not otherwise, and the configuration variable JOB_QUEUE_LOG existed but was not visible. (HTCONDOR-601)

  • The condor_watch_q command now refuses to watch the queue if doing so would require using more kernel resources (“inotify watches”) than allowed. This limit can be increased by your system administrator, and we expect to remove this limitation in a future version of the tool. (HTCONDOR-676)

Bugs Fixed:

  • The CUDA_VISIBLE_DEVICES environment variable may now contain CUDA<n> and GPU-<uuid> formatted values, in addition to integer values. (HTCONDOR-669)

  • Updated condor_gpu_discovery to be compatible with version 470 of nVidia’s drivers. (HTCONDOR-620)

  • If run with only the CUDA runtime library available, condor_gpu_discovery and condor_gpu_utilization no longer crash. (HTCONDOR-668)

  • Fixed a bug in condor_gpu_discovery that could result in no output or a segmentation fault when the -opencl argument was used. (HTCONDOR-729)

  • Fixed a bug that prevented Singularity jobs from running when the Singularity binary emitted many warning messages to stderr. (HTCONDOR-698)

  • The Windows MSI installer has been updated so that it no longer reports that a script failed during installation on the latest version of Windows 10. This update also changes the permissions of the configuration files created by the installer so the installing user has edit access and all users have read access. (HTCONDOR-684)

  • Fixed a bug that prevented condor_ssh_to_job from working to a personal or non-rootly condor. (HTCONDOR-485)

  • The bosco_cluster tool now clears out old installation files when the –add option is used to update an existing installation. (HTCONDOR-577)

  • Fixed a bug that could cause the condor_had daemon to fail at startup when the local machine has multiple IP addresses. This bug is particularly likely to happen if PREFER_IPV4 is set to False. (HTCONDOR-625)

  • For the machine ad attributes OpSys* and configuration parameters OPSYS*, treat macOS 11.X as if it were macOS 10.16.X. This represents the major version numbers in a consistent, if somewhat inaccurate manner. (HTCONDOR-626)

  • Fixed a bug that ignored the setting of per-Accounting Group GROUP_AUTOREGROUP from working. Global autoregroup worked correctly. (HTCONDOR-632)

  • A self-checkpointing job’s output and error logs will no longer be interrupted by eviction if the job specifies transfer_checkpoint_files; HTCondor now automatically considers them part of the checkpoint the way it automatically considers them part of the output. (HTCONDOR-656)

  • HTCondor now transfers the standard output and error logs when when_to_transfer_output is ON_SUCCESS and transfer_output_files is empty. (HTCONDOR-673)

  • Fixed a bug that could cause the starter to crash after transferring files under certain rare circumstances. This also corrected a problem which may have been causing the number of bytes transferred to be undercounted. (HTCONDOR-722)

Version 9.0.5

Release Notes:

  • HTCondor version 9.0.5 released on August 18, 2021.

New Features:

  • If the SCITOKENS authentication method succeeds (that is, the client presented a valid SciToken) but the user-mapping fails, HTCondor will try the next authentication method in the list instead of failing. (HTCONDOR-589)

  • The bosco_cluster command now creates backup files when the --override option is used. (HTCONDOR-591)

  • Improved the detection of Red Hat Enterprise Linux based distributions. Previously, only CentOS was recognized. Now, other distributions such as Scientific Linux and Rocky should be recognized. (HTCONDOR-609)

  • The condor-boinc package is no longer required to be installed with HTCondor, thus making condor-boinc optional. (HTCONDOR-644)

Bugs Fixed:

  • Fixed a bug on the Windows platform where condor_submit would crash rarely after successfully submitting a job. This caused problems for programs that look at the return status of condor_submit, including condor_dagman (HTCONDOR-579)

  • The job attribute ExitCode is no longer missing from the job ad after OxExitHold triggers. (HTCONDOR-599)

  • Fixed a bug where running condor_who as a non-root user on a Unix system would print a confusing warning to stderr about running as non-root. (HTCONDOR-590)

  • Fixed a bug where condor_gpu_discovery would not report any GPUs if any MIG-enabled GPU on the system were configured in certain ways. Fixed a bug which could cause condor_gpu_discovery’s output to become unparseable after certain errors. (HTCONDOR-476)

  • HTCondor no longer ignores files in a job’s spool directory if they happen to share a name with an entry in transfer_input_files. This allows jobs to specify the same file in transfer_input_files and in transfer_checkpoint_files, and still resume properly after a checkpoint. (HTCONDOR-583)

  • Fixed a bug where jobs running on Linux machines with cgroups enabled would not count files created in /dev/shm in the MemoryUsage attribute. (HTCONDOR-586)

  • Fixed a bug in the condor_now tool, where the condor_schedd would not use an existing security session to run the selected job on the claimed resources. This could often lead to the job being unable to start. (HTCONDOR-603)

Version 9.0.4

Release Notes:

  • HTCondor version 9.0.4 released on July 29, 2021.

New Features:

  • None.

Bugs Fixed:

Version 9.0.3

Release Notes:

  • HTCondor version 9.0.3 released on July 27, 2021 and pulled two days later when an issue was found with a patch.

New Features:

  • None.

Bugs Fixed:

  • None.

Version 9.0.2

Release Notes:

  • HTCondor version 9.0.2 released on July 8, 2021.

  • Removed support for GRAM grid jobs. (HTCONDOR-561)

New Features:

  • HTCondor can now be configured to only use FIPS 140-2 approved security functions by using the new configuration template: use security:FIPS. (HTCONDOR-319)

  • Added new command-line flag to condor_gpu_discovery, -divide, which functions like -repeat, except that it divides the GPU attribute GlobalMemoryMb by the number of repeats (and adds the GPU attribute DeviceMemoryMb, which is the undivided total). To enable this new behavior, modify GPU_DISCOVERY_EXTRA appropriately. (HTCONDOR-454)

  • The maximum line length for STARTD_CRON and SCHEDD_CRON job output has been extended from 8k bytes to 64k bytes. (HTCONDOR-498)

  • Added two new commands to condor_submit - use_scitokens and scitokens_file. (HTCONDOR-508)

  • Reduced condor_shadow memory usage by 40% or more on machines with many (more than 64) cores. This allows a correspondingly greater number of shadows and thus jobs to run on these submit machines. (HTCONDOR-540)

  • Added support for using an authenticated SMTP relay on port 587 to condor_mail.exe on Windows. (HTCONDOR-303)

  • The condor_job_router_info tool will now show info for a rootly JobRouter even when the tool is not running as root. This change affects the way jobs are matched when using the -match or -route options. (HTCONDOR-525)

  • condor_gpu_discovery now recognizes Capability 8.6 devices and reports the correct number of cores per Compute Unit. (HTCONDOR-544)

  • Added command line option --copy-ssh-key to bosco_cluster. When set to no, this option prevents bosco_cluster from installing an ssh key on the remote system, and assume passwordless ssh is already possible. (HTCONDOR-270)

  • Update to be able to link in scitokens-cpp library directly, rather than always using dlopen(). This allows SciTokens to be used with the conda-forge build of HTCondor. (HTCONDOR-541)

Bugs Fixed:

  • When a Singularity container is started, and the test is run before the job, and the test fails, the job is now put back to idle instead of held. (HTCONDOR-539)

  • Fixed Munge authentication, which was broken starting with HTCondor 8.9.9. (HTCONDOR-378)

  • Fixed a bug in the Windows MSI installer where installation would only succeed at the default location of C:\Condor. (HTCONDOR-543)

  • Fixed a bug that prevented docker universe jobs from running on machines whose hostnames were longer than about 60 characters. (HTCONDOR-473)

  • Fixed a bug that prevented bosco_cluster from detecting the remote host’s platform when it is running Scientific Linux 7. (HTCONDOR-503)

  • Fixed a bug that caused the query-krb and delete-krb options of condor_store_cred to fail. This bug also affected the Python bindings query_user_cred and delete_user_cred methods (HTCONDOR-533)

  • Attribute GridJobId is no longer removed from the job ad of grid-type batch jobs when the job enters Completed or Removed status. (HTCONDOR-534)

  • Fixed a bug that could prevent HTCondor from noticing new events in job event logs, if those logs were being written from one machine and read from another via AFS. (HTCONDOR-463)

  • Using expressions for values in the ads of grid universe jobs of type batch now works correctly. (HTCONDOR-507)

  • Fixed a bug that prevented a personal condor from running in a private user namespace. (HTCONDOR-550)

  • Fixed a bug in the condor_who program that caused it to hang on Linux systems, especially those running AFS or other shared filesystems. (HTCONDOR-530) (HTCONDOR-573)

  • Fixed a bug that cause the condor_master to hang for up to two minutes when shutting down, if it was configured to be a personal condor. (HTCONDOR-548)

  • When a grid universe job of type nordugrid fails on the remote system, the local job is now put on hold, instead of automatically resubmitted. (HTCONDOR-535)

  • Fixed a bug that caused SSL authentication to crash on rare occasions. (HTCONDOR-428)

  • Added the missing Ceiling attribute to negotiator user priorities in the Python bindings. (HTCONDOR-560)

  • Fixed a bug in DAGMan where SUBMIT-DESCRIPTION statements were incorrectly logging duplicate description warnings. (HTCONDOR-511)

  • Add the libltdl library to the HTCondor tarball. This library was inadvertently omitted when streamlining the build process in version 8.9.12. (HTCONDOR-576)

Version 9.0.1

Release Notes:

  • HTCondor version 9.0.1 released on May 17, 2021.

  • The installer for Windows will now replace the condor_config file even on an update. You must use condor_config.local or a configuration directory to customize the configuration if you wish to preserve configuration changes across updates.

Known Issues:

  • There is a known issue with the installer for Windows where it does not honor the Administrator Access list set in the MSI permissions dialog on a fresh install. Instead it will always set the Administrator access to the default value.

  • MUNGE security is temporarily broken.

New Features:

  • The Windows MSI installer now sets up user-based authentication and creates an IDTOKEN for local administration. (HTCONDOR-407)

  • When the AssignAccountingGroup configuration template is in effect and a user submits a job with a requested accounting group that they are not permitted to use, the submit will be rejected with an error message. This configuration template has a new optional second argument that can be used to quietly ignore the requested accounting group instead. (HTCONDOR-426)

  • Added the OpenBLAS environment variable OPENBLAS_NUM_THREADS to the list of environment variables exported by the condor_starter per these recommendations. (HTCONDOR-444)

  • HTCondor now parses /usr/share/condor/config.d/ for configuration before /etc/condor/config.d, so that packagers have a convenient place to adjust the HTCondor configuration. (HTCONDOR-45)

  • Added a boolean option LOCAL_CREDMON_TOKEN_USE_JSON for the local issuer condor_credmon_oauth that is used to decide whether or not the bare token string in a generated access token file is wrapped in JSON. Default is LOCAL_CREDMON_TOKEN_USE_JSON = true (wrap token in JSON). (HTCONDOR-367)

Bugs Fixed:

  • Fixed a bug where sending an updated proxy to an execute node could cause the condor_starter to segfault when AES encryption was enabled (which is the default). (HTCONDOR-456) (HTCONDOR-490)

  • Fixed a bug with jobs that require running on a different machine after a failure by referring to MachineAttrX attributes in their requirements expression. (HTCONDOR-434)

  • Fixed a bug in the way AutoClusterAttrs was calculated that could cause matchmaking to ignore attributes changed by job_machine_attrs. (HTCONDOR-414)

  • Fixed a bug in the implementation of the submit commands max_retries and success_exit_code which would cause jobs which exited on a signal to go on hold (instead of exiting or being retried). (HTCONDOR-430)

  • Fixed a memory leak in the job router, usually triggered when job policy expressions cause removal of the job. (HTCONDOR-408)

  • Fixed some bugs that caused bosco_cluster --add to fail. Allow remote_gahp to work with older Bosco installations via the --rgahp-script option. Fixed security authorization failure between condor_gridmanager and condor_ft-gahp. (HTCONDOR-433) (HTCONDOR-438) (HTCONDOR-451) (HTCONDOR-452) (HTCONDOR-487)

  • Fixed a bug in condor_submit when a SEC_CREDENTIAL_PRODUCER was configured that could result in condor_submit reporting that the Queue statement of a submit file was missing or invalid. (HTCONDOR-427)

  • Fixed a bug in the local issuer condor_credmon_oauth where SciTokens version 2.0 tokens were being generated without an “aud” claim. The “aud” claim is now set to LOCAL_ISSUER_TOKEN_AUDIENCE. The “ver” claim can be changed from the default of “scitokens:2.0” by setting LOCAL_ISSUER_TOKEN_VERSION. (HTCONDOR-445)

  • Fixed several bugs that could result in the condor_token_ tools aborting with a c++ runtime error on newer versions of Linux. (HTCONDOR-449)

Version 9.0.0

Release Notes:

  • HTCondor version 9.0.0 released on April 14, 2021.

  • The installer for Windows platforms was not ready for 9.0.0. Windows support will appear in 9.0.1.

  • Removed support for CREAM and Unicore grid jobs, glexec privilege separation, DRMAA, and condor_cod.

Known Issues:

  • MUNGE security is temporarily broken.

  • The bosco_cluster command is temporarily broken.

New Features:

  • A new tool condor_check_config can be used after an upgrade when you had a working condor configuration before the upgrade. It will report configuration values that should be changed. In this version the tool for a few things related to the change to a more secure configuration by default. (HTCONDOR-384)

  • The condor_gpu_discovery tool now defaults to using -short-uuid form for GPU ids on machines where the CUDA driver library has support for them. A new option -by-index has been added to select index-based GPU ids. (HTCONDOR-145)

Bugs Fixed:

  • Fixed a bug introduced in 8.9.12 where the condor_job_router inside a CE would crash when evaluating periodic expressions (HTCONDOR-402)