Version 24.0 LTS Releases

These are Long Term Support (LTS) versions of HTCondor. As usual, only bug fixes (and potentially, ports to new platforms) will be provided in future 24.0.y versions. New features will be added in the 24.x.y feature versions.

The details of each version are described below.

Version 24.0.19

Release Notes:

  • HTCondor version 24.0.19 released on April 16, 2026.

New Features:

  • HTCondor tarballs now contain Pelican 7.24.0

  • The condor package now requires pelican-7.24.0.

Bugs Fixed:

  • Fixed a bug where an NVIDIA GPU that has been subdivided in MIG sub-devices would not properly pass the device names in the various NVIDIA related environment variables. (HTCONDOR-3567)

  • Fixed a performance problem with the classad2 python bindings that caused the parseAds methods to run in n-squared time. (HTCONDOR-3632)

  • Fixed a bug where allowed_execute_duration would possibly not be respected for jobs that did not do input file transfer. (HTCONDOR-3633)

  • Fixed a bug for batch grid universe jobs where submitting to a remote system would fail if the username on the remote system contained a dot. (HTCONDOR-3581)

  • Fixed a bug where some of the values reported for job execution time were enormous if the Access Point daemons restarted while the job was running. (HTCONDOR-3592)

  • Fixed a bug where an administrator couldn’t query a daemon’s configuration using condor_config_val when authentication is required for READ level authorization. (HTCONDOR-3572)

  • The htcondor2.send_command() method no longer incorrectly requires a target parameter. (HTCONDOR-3597)

Version 24.0.18

Release Notes:

  • HTCondor version 24.0.18 released on March 12, 2026.

New Features:

  • Enable the in-memory cache option in newer versions of the scitokens-cpp library. This allows the library to continue operating when the on-disk cache is not useable. (HTCONDOR-3560)

  • HTCondor tarballs now contain Pelican 7.23.0

  • The condor package now requires pelican-7.23.0.

Bugs Fixed:

  • Fixed a bug in condor_submit that could expand the executable path incorrectly when using late materialization and a different executable for each job. (HTCONDOR-3546)

  • Fixed a bug where new STARTD_CRON values didn’t trigger an immediate update of slot ads to the collector when configured to do so. (HTCONDOR-3526)

  • Update Annex’s knowledge about the Delta HPC system. (HTCONDOR-3559)

Version 24.0.17

Release Notes:

  • HTCondor version 24.0.17 released on February 12, 2026.

New Features:

  • None.

Bugs Fixed:

Version 24.0.16

Release Notes:

  • HTCondor version 24.0.16 released on January 29, 2026.

New Features:

  • HTCondor tarballs now contain Pelican 7.22.0

  • The condor package now requires pelican-7.22.0.

  • HTCondor tarballs now contain Apptainer 1.4.5

  • The condor RPM package now requires at least apptainer version 1.4.5.

Bugs Fixed:

Version 24.0.15

Release Notes:

  • HTCondor version 24.0.15 released on December 15, 2025.

New Features:

  • HTCondor tarballs now contain Pelican 7.21.1

  • The condor package now requires pelican-7.21.1.

  • HTCondor tarballs now contain Apptainer 1.4.4

  • The condor RPM package now requires at least apptainer version 1.4.4.

Bugs Fixed:

  • Fixed a bug that could cause the AP to fail to read job credential files. (HTCONDOR-3377)

  • Fixed bugs that could cause a crash in the authentication code. (HTCONDOR-3394)

  • Fixed a bug where the condor_negotiator would fail to contact the condor_schedd to perform matchmaking if its ALLOW_CLIENT configuration parameter didn’t authorize the identity submit-side@matchsession. (HTCONDOR-3378)

  • Fixed a bug where tools like condor_status would print an incorrect value for the -af:r option when the value to be printed was a single attribute reference. (HTCONDOR-3347)

Version 24.0.14

Release Notes:

  • HTCondor version 24.0.14 released on November 3, 2025.

New Features:

  • None.

Bugs Fixed:

  • Fix problem running PyTorch jobs on multiple GPUs with newer versions of the CUDA library by providing long GPU IDs in the CUDA_VISIBLE_DEVICES environment variable (HTCONDOR-3350)

  • Calling htcondor2.Schedd.history() on an object whose corresponding daemon can’t be contacted will no longer cause a segmentation fault. (HTCONDOR-3314)

  • Executions points using disk enforcement failed when provided backing LVM_VOLUME_GROUP_NAME and LVM_THINPOOL_NAME contained hyphens. (HTCONDOR-3334)

  • Add a timeout for all HTTP operations in the grid universe (affects arc, ec2, and gce grid types). (HTCONDOR-3300)

  • Fixed a bug where administrative tools (e.g. condor_drain) can fail if authentication is not required for queries to the condor_collector. (HTCONDOR-3301)

  • The condor_startd now correctly removes docker containers that have escaped from docker universe jobs launched by condor_starter's that have exited uncleanly. (HTCONDOR-3338)

  • Annexes no longer download a default configuration tarball, making them more robust. This required upgrading the default version of HTCondor run by annexes to 25.1.0, which can figure the details out on its own. On Delta, annexes now give Delta additional time to start file-transfer plug-ins, so they should be more-reliably available there. (HTCONDOR-3220)

  • condor_adstash no longer skips entire history files when it encounters a line that cannot be decoded to UTF-8. (HTCONDOR-3335)

  • Fixed a bug in blahpd when /tmp is mounted with the noexec option. (HTCONDOR-3343)

Version 24.0.13

Release Notes:

  • HTCondor version 24.0.13 released on October 9, 2025.

New Features:

  • HTCondor tarballs now contain Pelican 7.20.2

  • The condor package now requires pelican-7.20.2.

Bugs Fixed:

Version 24.0.12

Release Notes:

  • HTCondor version 24.0.12 released on September 23, 2025.

New Features:

  • Updated condor_upgrade_check to test for well known gotchas between v24 and v25 of HTCondor installations. (HTCONDOR-3209)

  • HTCondor tarballs now contain Pelican 7.19.3

  • The condor package now requires pelican-7.19.3.

  • The condor RPM package now requires at least apptainer version 1.4.2.

Bugs Fixed:

  • Fixed a bug where the blahpd would fail when using Python 3.12 or later. (HTCONDOR-3225)

  • Fixed bug where jobs held due to exceeding disk usage had the same HoldReasonCode and HoldReasonSubCode as jobs that exceeded memory usage. (HTCONDOR-3248)

  • Fixed a bug where Flock Collectors could be forgotten after having a connection interruption. (HTCONDOR-3200)

  • Fixed a bug where a condor_schedd would not attempt to flock to collectors in the FLOCK_TO list when a communication failure occurred to a collector earlier in the list. (HTCONDOR-3200)

  • Fixed a packaging bug in tarballs for RPM-based systems which resulted in the Pelican file-transfer plug-in not being enabled by default. (HTCONDOR-3239)

Version 24.0.11

Release Notes:

  • HTCondor version 24.0.11 released on August 21, 2025.

New Features:

Bugs Fixed:

  • Fixed a bug with docker universe jobs that have a PostCmd. This PostCmd script was not passed the environment variables _CONDOR_MAINJOB_EXIT_CODE or _CONDOR_MAINJOB_EXIT_SIGNAL (HTCONDOR-3185)

  • Fixed a bug where the ImageSize attribute in a slot on a Windows EP was a large random value while the slot was running a job. The bug was due to a change in the Win32 API where the total virtual memory of a process is no longer reported. From now on ImageSize on a Windows EP will have the same value as ResidentSetSize. (HTCONDOR-3179)

  • Fixed a bug where an old client (version 9.0 or earlier) with lax security settings (authentication, encryption, and integrity all disabled or optional) would fail to communicate with a daemon with stronger security settings. (HTCONDOR-3189)

  • condor_token_request no longer fails with an error if the token is automatically approved by the daemon. (HTCONDOR-239)

Version 24.0.10

Release Notes:

  • HTCondor version 24.0.10 released on July 28, 2025.

New Features:

  • HTCondor tarballs now contain Pelican 7.17.2

  • The condor package now requires pelican-7.17.2.

Bugs Fixed:

Version 24.0.9

Release Notes:

  • HTCondor version 24.0.9 released on June 26, 2025.

New Features:

  • Initial Support for Enterprise Linux 10, including the x86_64_v2 platform. (HTCONDOR-3090)

  • HTCondor tarballs now contain Pelican 7.17.0

  • The condor package now requires pelican-7.17.0.

Bugs Fixed:

  • The results of key in htcondor2.param and key in htcondor2.param.keys() now match for keys which are defined to have no value. (Previously, such keys would be returned by keys().) (HTCONDOR-3085)

  • Fixed a memory leak in the condor_schedd when late materialization is used. (HTCONDOR-3096)

  • Fixed a bug where the condor_master would not start up on systems where ulimit -n was close to 2 ^ 31 file descriptors. (HTCONDOR-3079)

  • Fixed a bug in condor_adstash that would prevent ads from being ingested to Elasticsearch or OpenSearch when a lookup of indexes did not start with a writable index. (HTCONDOR-3109)

  • Fixed a bug when DAGMan’s log file was on a full filesystem, DAGMan would not exit with the correct log file full exit code. (HTCONDOR-3066)

  • The submit commands kill_sig, remove_kill_sig, and hold_kill_sig are now ignored for Windows jobs. These control Unix process signals, which are not relevant on Windows. (HTCONDOR-3078)

  • Fixed a bug introduced in HTCondor 23.0.21, 23.10.21, and 24.0.5 that caused the condor_gridmanager to fail at startup if GRIDMANAGER_LOG_APPEND_SELECTION_EXPR was set to True. (HTCONDOR-3099)

  • The plug-in used to handle http and https URLs now sets the user-agent header by default (condor_curl_plugin/0.2). Web servers which return a 403/Forbidden error when the user-agent is empty will now function as expected. (HTCONDOR-3121)

Version 24.0.8

Release Notes:

  • HTCondor version 24.0.8 released on June 12, 2025.

New Features:

  • HTCondor tarballs now contain Pelican 7.16.5

  • The condor package now requires pelican-7.16.5.

  • Pelican 7.16.5 now includes end-to-end integrity checks for clients

  • HTCondor tarballs now contain Apptainer 1.4.1

Bugs Fixed:

  • Fixed a bug introduced in 24.0.7 and 24.7.3 when running on Linux cgroup v1 systems, jobs that were killed by the out-of-memory killer were considered completed instead of being put on hold. (HTCONDOR-3094)

  • The htcondor2.Credd initializer now properly raised a TypeError if the location argument isn’t a classad2.ClassAd, rather than failing to raise a TypError. (HTCONDOR-2993)

  • Fixed bug where DAGMAN_MAX_JOBS_IDLE value was not being respected by DAGMan even if no limit was specified via condor_submit_dags -maxidle option. (HTCONDOR-3011)

  • Fixed some bugs with parallel universe jobs that can cause the condor_schedd to crash. (HTCONDOR-3049)

  • Fixed a bug that caused the condor_starter to crash if a job was vacated during input file transfer. (HTCONDOR-3016)

  • condor_watch_q will now correctly display the full range of job ids for each batch grouping. (HTCONDOR-2992)

  • Fixed bug in one of condor_upgrade_check's tests that would cause the test to check for incorrect hostname when running inside of a container. (HTCONDOR-3014)

Version 24.0.7

Release Notes:

  • HTCondor version 24.0.7 released on April 22, 2025.

New Features:

  • The condor_stard now distributes the LoadAvg assigned to a partitionable slot to the idle resources of the partitionable slot, and then to the dynamic slots. Machines that have only a single partitionable slot will now have the same behavior under a DESKTOP policy as they did in version 23.10.18 and 24.0.1. (HTCONDOR-2901)

  • HTCondor tarballs now contain Pelican 7.15.1

  • The condor package now requires pelican-7.15.1. The weak dependency is no longer used, because dnf would not update to the requested pelican version.

  • HTCondor tarballs now contain Apptainer 1.4.0

  • The condor RPM package now requires at least apptainer version 1.3.6.

Bugs Fixed:

  • When using delegated cgroup v2, HTCondor no longer reports that that main job (often a pilot) has an out of memory condition when only the sub-job has hit an oom. (HTCONDOR-2944)

  • Fixed a bug that could cause the condor_starter to crash when running docker universe jobs with custom volume mounts. (HTCONDOR-2890)

  • Fixed a bug preventing spooled or remote jobs using preserve_relative_paths from working. (HTCONDOR-2877)

  • The condor_kbdd now also looks in the XDG_RUNTIME_DIRECTORY when trying to find a XAuthority file to use to connect to a local X server. (HTCONDOR-2921)

  • Fixed a bug that prevented daemons from updating their ads in the condor_collector when authentication is disabled but encryption or integrity is enabled. (HTCONDOR-2888)

  • Fixed a bug in condor_adstash that caused it to fail to discover condor_startd daemons using ENABLE_STARTD_DAEMON_AD (enabled by default since HTCondor 23.9). (HTCONDOR-2908)

  • Fixed a bug with transfer_output_remaps when given an erroneous trailing semicolon. (HTCONDOR-2910)

  • Fixed some bugs with parallel universe jobs that can cause the condor_schedd to crash. (HTCONDOR-3049)

  • Fixed inflated memory usage reporting for docker universe jobs on hosts using cgroups V2. The reported memory no longer includes the cached memory. (HTCONDOR-2961)

  • Fixed a bug where specifying transfer_output_remaps from a path which didn’t exist to a file:// URL would cause HTCondor to report a useless (albeit correct) error. (HTCONDOR-2790)

  • Fixed a bug that could cause the condor_shadow daemon to crash when the job transfer input file list was very long (thousands of characters). (HTCONDOR-2859)

  • Fixed a bug where two different condor_gridmanager processes could attempt to manage the same jobs when GRIDMANAGER_SELECTION_EXPR evaluated to Undefined or an empty string for any job. (HTCONDOR-2895)

  • Fixed a rare bug in the condor_schedd, when PER_JOB_HISTORY_DIR is set that could cause a repeated restart loop. (HTCONDOR-2902)

  • X.509 proxy delegation no longer fails when using OpenSSL 3.4.0 or later. (HTCONDOR-2904)

  • Fixed a bug that could cause the condor_gridmanager to crash when there were ARC CE jobs with no X509userproxy. (HTCONDOR-2907)

  • Fixed a bug that usually prevented manifest from populating the in and out files. (HTCONDOR-2916)

  • Fixed a bug that could cause a job submission to fail if a previous job submission to the same condor_schedd failed. (HTCONDOR-2917)

  • Fixed a bug where daemons wouldn’t immediately apply new security policy to incoming commands after a reconfigure. (HTCONDOR-2929)

  • Fixed a bug where condor_history would crash when reading a history file larger than 2GB in the default mode (backwards). (HTCONDOR-2933)

  • Fixed a bug that caused the ce-audit plugin to fail. (HTCONDOR-2963)

  • Removed a scary-looking message in the log of the condor_collector about denying NEGOTIATOR-level authorization when the client wasn’t requesting that authorization level. (HTCONDOR-2964)

  • Fixed a bug that caused most updates of collector ads via UDP to be rejected. (HTCONDOR-2975)

  • Fixed a bug where the condor_shadow would wait for the job lease to expire (usually 40 minutes) before returning a job to idle status when the condor_starter failed to initialize. (HTCONDOR-2997)

  • The condor_startd now checks to see if the START expression of a static slot still

    evaluates to true before it allows a slot to be claimed. This helps to give an accurate reply to the condor_schedd when it tries to claim a slot with a START expression that changes frequently.

    (HTCONDOR-3013)

Version 24.0.6

Release Notes:

  • HTCondor version 24.0.6 released on March 27, 2025.

New Features:

  • None.

Bugs Fixed:

Version 24.0.5

Release Notes:

  • HTCondor version 24.0.5 released on March 4, 2025.

New Features:

Bugs Fixed:

  • Fixed a bug where chirp would not work in container universe jobs using the docker runtime. (HTCONDOR-2866)

  • Fixed a bug where referencing htcondor2.JobEvent.cluster could crash if processed log event was not associated with job(s) (i.e. had a negative value). (HTCONDOR-2881)

  • Fixed a bug that caused the condor_gridmanager to abort if a job that it was managing disappeared from the job queue (i.e. due to someone running condor_rm -force). (HTCONDOR-2845)

  • Fixed a bug that caused grid ads from different Access Points to overwrite each other in the collector. (HTCONDOR-2876)

  • Fixed a memory leak that can occur in any HTCondor daemon when an invalid ClassAd expression is encountered. (HTCONDOR-2847)

  • Fixed a bug that caused daemons to go into infinite recursion, eventually crashing when they ran out of stack memory. (HTCONDOR-2873)

Version 24.0.4

Release Notes:

  • HTCondor version 24.0.4 released on February 4, 2025.

New Features:

  • For arc grid universe jobs, the new submit command arc_data_staging can be used to supply additional elements to the DataStaging block of the ARC ADL that HTCondor constructs. (HTCONDOR-2774)

Bugs Fixed:

  • Fixed a bug in the negotiator that caused it to crash when matching offline ads. (HTCONDOR-2819)

  • Fixed a memory leak in the schedd that could be caused by SCHED_CRON scripts that generate standard error output. (HTCONDOR-2817)

  • Fixed a bug that cause the condor_schedd to crash with a segmentation fault if a condor_off -fast command was run while a schedd cron script was running. (HTCONDOR-2815)

  • Fixed issue where EPs using STARTD_ENFORCE_DISK_LIMITS would fill up the EPs filesystem due to excessive saving of metadata to /etc/lvm/archive. (HTCONDOR-2791)

  • Fixed bug where container_service_names did not work. (HTCONDOR-2829)

  • Fixed very rare bug that could cause the condor_startd to crash when the condor_collector times out queries and DNS is running very slowly. (HTCONDOR-2831)

  • Updated condor_upgrade_check to test for use for PASSWORD authentication and warn about the authenticated identity changing. (HTCONDOR-2823)

Version 24.0.3

Release Notes:

  • HTCondor version 24.0.3 released on January 6, 2025.

New Features:

Bugs Fixed:

Version 24.0.2

Release Notes:

  • HTCondor version 24.0.2 released on November 26, 2024.

New Features:

  • Added a new configuration parameter, STARTER_ALWAYS_HOLD_ON_OOM which defaults to true. When true, if a job is killed with an OOM signal, it is put on hold. When false, the system tries to determine if the job was out of memory, or the system was, and if the latter, evicts the job and sets it back to idle. (HTCONDOR-2686)

Bugs Fixed:

  • Fixed a bug that prevents condor_ssh_to_job from working with sftp and scp modes. (HTCONDOR-2687)

  • Fixed a bug where a daemon would repeatedly try to use its family security session when authenticating with another daemon that doesn’t know about the session. (HTCONDOR-2685)

  • Fixed a bug where a job would sometimes match but then fail to start on a machine with a START expression that referenced the KeyboardIdle attribute. (HTCONDOR-2689)

  • htcondor2.Submit.itemdata() now correctly accepts an optional qargs parameter (as in version 1). (HTCONDOR-2618)

  • Stop signaling the condor_credmon_oauth daemon on every job submission when there’s no work for it to do. This will hopefully reduce the frequency of some errors in the condor_credmon_oauth. (HTCONDOR-2653)

  • Fixed a bug that could cause the condor_schedd to crash if a job’s ClassAd contained a $$() macro that couldn’t be expanded. (HTCONDOR-2730)

  • Docker universe jobs now check the Architecture field in the image, and if it doesn’t match the architecture of the EP, the job is put on hold. The new parameter DOCKER_SKIP_IMAGE_ARCH_CHECK skips this. (HTCONDOR-2661)

Version 24.0.1

Release Notes:

New Features:

  • condor_gpu_discovery can now detect GPUs using AMD’s HIP 6 library. HIP detection will be used if the new -hip option is used or if no detection method is specified and no CUDA devices are detected. (HTCONDOR-2509)

Bugs Fixed:

  • On Windows the htcondor tool now uses the Python C API to try and launch the python interpreter. This will fail with a message box about installing python if python 3.9 is not in the path. (HTCONDOR-2650)

  • htcondor2.Submit.from_dag() now recognizes DoRecov as a synonym for DoRecovery. This improves compatibility with version 1. (HTCONDOR-2613)

  • htcondor2.Submit.itemdata() now (correctly) returns an iterator over dictionaries if the htcondor2.Submit object specified variable names in its queue statement. (HTCONDOR-2613)

  • When you specify item data using a dict, HTCondor will now correctly reject values containing newlines. (HTCONDOR-2616)

  • When docker universe jobs failed with a multi-line errors from docker run, the job used to fail with an “unable to inspect container” message. Now the proper hold message is set and the job goes on hold as expected. (HTCONDOR-2679)

  • htcondor annex now reports a proper error if you request an annex from a GPU-enabled queue but don’t specify how many GPUs per node you want (and the queue does not always allocate whole nodes). (HTCONDOR-2633)

  • Fixed a bug where HTCondor systems configured to use cgroups on Linux to measure memory would reuse the peak memory from the previous job in a slot, if any process in the former job was unkillable. This can happen if the job is stuck in NFS or running GPU code. Instead, HTCondor polls the current memory and keeps the peak itself internally. (HTCONDOR-2647)

  • Fixed a bug where the -divide flag to condor_gpu_discovery would be ignored on servers with only one type of GPU device. (HTCONDOR-2669)

  • Fixed a bug introduced in HTCSS v23.8.1 which prevented an EP from running multiple jobs on a single GPU device when -divide or -repeat was added to to configuration knob GPU_DISCOVERY_EXTRA. Also fixed problems with any non-fungible machine resource inventory that contained repeated identifiers. (HTCONDOR-2678)

  • Fixed a bug where condor_watch_q would display None for jobs with no JobBatchName instead of the expected ClusterId. (HTCONDOR-2625)

  • htcondor2.Schedd.submit() now correctly raises a TypeError when passed a description that is not a htcondor2.Submit object. (HTCONDOR-2631)

  • When submitting jobs to an SGE cluster via the grid universe, the blahp no longer saves the output of its wrapper script in the user’s home directory (where the files would accumulate and never be cleaned up). (HTCONDOR-2630)

  • Improved the error message when job submission as a disallowed user fails (i.e. submitting as the ‘condor’ or ‘root’ user). (HTCONDOR-2638)

  • Fixed bug in htcondor server status that caused incorrect output if DAEMON_LIST contained commas. (HTCONDOR-2667)

  • Fixed the new default security configuration to work with older binaries. (HTCONDOR-2701)

  • An unresponsive libvirtd daemon no longer causes the condor_startd to block indefinitely. (HTCONDOR-2644)