Print Formats
Many HTCondor tools that work with ClassAds use a formatting engine called the ClassAd pretty printer. Tools that have a -format or -autoformat argument use those arguments to configure the ClassAd pretty printer, and then use the pretty printer to produce output from ClassAds.
The condor_q, condor_history and condor_status tools, as well as others that have a -print-format or -pr argument can configure the ClassAd pretty using a file. The syntax of this file is described below.
Not all tools support all of the print format options.
Syntax
A print format file consists of a heading line and
zero or more formatting lines
followed by optional constraint, sort and summary lines.
These sections of the format file begin with the keywords
SELECT
, WHERE
, GROUP
, or SUMMARY
which must be in that order if they appear.
These keywords must be all uppercase and must be the first word on the line.
A line beginning with # is treated as a comment
A custom print format file must begin with the SELECT
keyword.
The SELECT
keyword can be followed by options to qualify the type of
query, the global formatting options and whether or not there will be column
headings. The prototype for the SELECT
line is:
SELECT [FROM AUTOCLUSTER | UNIQUE] [BARE | NOTITLE | NOHEADER | NOSUMMARY] [LABEL [SEPARATOR <string>]] [<separators>]
The first two optional keywords indicate the query type. These options work only in condor_q.
FROM AUTOCLUSTER
Used with condor_q to query the schedd’s default autocluster set.
UNIQUE
Used with condor_q to ask the condor_schedd to count unique values. This option tells the schedd to building a new
FROM AUTOCLUSTER
set using the given attributes
The next set of optional keywords enable or disable various things that are normally printed before or after the classad output.
NOTITLE
Disables the title on tools that have a title, like the Schedd name from condor_q.
NOHEADER
Disables column headers.
NOSUMMARY
Disables the summary output such as the totals by job stats at the bottom of normal condor_q output.
BARE
Shorthand for
NOTITLE
NOHEADER
NOSUMMARY
In the descriptions below <string>
is text. If the text starts with a single quote, then it continues to
the next single quote. If it starts with a double quote, it continues to the next double quote. If it
starts with neither, then it continues until the next space or tab. a n, r or t inside the string will
be converted into a newline, carriage return or tab character respectively.
LABEL [SEPARATOR <string>]
Use item labels rather than column headers. The separator between the label and the value will be = unless the
SEPARATOR
is used to define a different one.RECORDPREFIX <string>
The value of
<string>
will be printed before each ClassAd. The default is to print nothing.RECORDSUFFIX <string>
The value of
<string>
will be printed after each ClassAd. The default is to print the newline character.FIELDPREFIX <string>
The value of
<string>
will be printed before each attribute or expression. The default is to print nothing.FIELDSUFFIX <string>
The value of
<string>
will be printed after each attribute or expression. The default is to print a single space.
After the SELECT
line, there should be zero or more formatting lines one line for each field in the output.
Each formatting line is a ClassAd attribute or expression followed by zero or more keywords that control formatting,
the first valid keyword ends the expression. Keywords are all uppercase and space delimited.
The prototype for each formatting line is:
<expr> [AS <label>] [PRINTF <format-string> | PRINTAS <function-name> [ALWAYS] | WIDTH [AUTO | [-]<INT>] ] [FIT | TRUNCATE] [LEFT | RIGHT] [NOPREFIX] [NOSUFFIX]
AS <string>
defines the label or column heading. if the formatting line has no AS keyword, then
<expr>
will be used as the label or column headingPRINTF <string>
<string>
should be a c++ printf format string, the same as used by the -format command line arguments for toolsPRINTAS <function>
Format using the built-in function. The Valid function names for
PRINTAS
are defined by the code and differ between the various tools, refer to the table at the end of this page.WIDTH [-]<int>
Align the data to the given width, negative values left align.
WIDTH AUTO
Use a width sized to fit the largest item.
FIT
Adjust column width to fit the data, normally used with WIDTH AUTO
TRUNCATE
If the data is larger than the given width, truncate it
LEFT
Left align the data to the given width
RIGHT
Right align the data to the given width
NOPREFIX
Do not include the
FIELDPREFIX
string for this fieldNOSUFFIX
Do not include the
FIELDSUFFIX
string for this fieldOR <char>[<char>]
if the field data is undefined, print
<char>
, if<char>
is doubled, fill the column with<char>
. Allowed values for<char>
are space or one of the following?*.-_#0
After the field formatting lines, there may be sections in the file that define a query constraint, sorting and grouping and the summary line. These sections can be multiple lines, but must begin with a keyword.
WHERE <constraint-expr>
Display only ClassAds where the expression
<constraint-expr>
evaluates to true.GROUP BY <sort-expr> [ASCENDING | DESCENDING]
Sort the ClassAds by evaluating
<sort-expr>
. If multiple sort keys are desired, theGROUP BY
line can be followed by lines containing additional expressions, for exampleGROUP BY Owner ClusterId DESCENDING
SUMMARY [STANDARD | NONE]
Enable or disable the summary totals. The summary can also be disabled using
NOSUMMARY
orBARE
keywords on theSELECT
line.
Examples
This print format file produces the default -nobatch
output of condor_q
# queue.cpf
# produce the standard output of condor_q
SELECT
ClusterId AS " ID" NOSUFFIX WIDTH AUTO
ProcId AS " " NOPREFIX PRINTF ".%-3d"
Owner AS "OWNER" WIDTH -14 PRINTAS OWNER
QDate AS " SUBMITTED" WIDTH 11 PRINTAS QDATE
RemoteUserCpu AS " RUN_TIME" WIDTH 12 PRINTAS CPU_TIME
JobStatus AS ST PRINTAS JOB_STATUS
JobPrio AS PRI
ImageSize AS SIZE WIDTH 6 PRINTAS MEMORY_USAGE
Cmd AS CMD PRINTAS JOB_DESCRIPTION
SUMMARY STANDARD
This print format file produces only totals
# q_totals.cpf
# show only totals with condor_q
SELECT NOHEADER NOTITLE
SUMMARY STANDARD
This print format file shows typical fields of the Schedd autoclusters.
# negotiator_autocluster.cpf
SELECT FROM AUTOCLUSTER
Owner AS OWNER WIDTH -14 PRINTAS OWNER
JobCount AS COUNT PRINTF %5d
AutoClusterId AS " ID" WIDTH 3
JobUniverse AS UNI PRINTF %3d
RequestMemory AS REQ_MEMORY WIDTH 10 PRINTAS READABLE_MB
RequestDisk AS REQUEST_DISK WIDTH 12 PRINTAS READABLE_KB
JobIDs AS JOBIDS
GROUP BY Owner
This print format file shows the use of SELECT UNIQUE
# count_jobs_by_owner.cpf
# aggregate by the given attributes, return unique values plus count and jobids.
# This query builds an autocluster set in the schedd on the fly using all of the displayed attributes
# And all of the GROUP BY attributes (except JobCount and JobIds)
SELECT UNIQUE NOSUMMARY
Owner AS OWNER WIDTH -20
JobUniverse AS "UNIVERSE " PRINTAS JOB_UNIVERSE
JobStatus AS STATUS PRINTAS JOB_STATUS_RAW
RequestCpus AS CPUS
RequestMemory AS MEMORY
JobCount AS COUNT PRINTF %5d
JobIDs
GROUP BY
Owner
PRINTAS functions for condor_q
Some of the tools that interpret a print format file have specialized formatting functions for certain
ClassAd attributes. These are selected by using the PRINTAS
keyword followed
by the function name. Available function names depend on the tool. Some functions implicitly use the
value of certain attributes, often multiple attributes. The list for condor_q is.
BATCH_NAME
Used for the
BATCH_NAME
column of the default output of condor_q. This function constructs a batch name string using value of the JobBatchName attribute if it exists, otherwise it constructs a batch name from JobUniverse, ClusterId, DAGManJobId, andDAGNodeName
.BUFFER_IO_MISC
Used for the
MISC
column of the-io
output of condor_q. This function constructs an IO string that varies by JobUniverse. It refers to TransferringInput, TransferringOutput and TransferQueued.CPU_TIME
Used for the
RUN_TIME
orCPU_TIME
column of the default condor_q output. The result of the function depends on whether the-currentrun
argument is used with condor_q. If RemoteUserCpu is undefined, this function returns undefined. It returns the value of RemoteUserCpu if it is non-zero. Otherwise it reports the amount of time that the condor_shadow has been alive. If the-currentrun
argument is used with condor_q, this will be the shadow lifetime for the current run only. If it is not, then the result is the sum of RemoteWallClockTime and the current shadow lifetime. The result is then formatted using the%T
format.CPU_UTIL
Used for the
CPU_UTIL
column of the default condor_q output. This function returns RemoteUserCpu divided by CommittedTime if CommittedTime is non-zero. It returns undefined if CommittedTime is undefined, zero or negative. The result is then formatted using the%.1f
format.DAG_OWNER
Used for the Owner column of default condor_q output. This function returns the value of the Owner attribute when the
-dag
option is not passed to condor_q. When the-dag
option is passed, it returns the value ofDAGNodeName
for jobs that have a DAGManJobId defined, and Owner for all other jobs.GRID_JOB_ID
Used for the
GRID_JOB_ID
column of the-grid
output of condor_q. This function extracts and returns the job id from theGridJobId
attribute.GRID_RESOURCE
Used for the
GRID->MANAGER HOST
column of the-grid
output of condor_q. This function extracts and returns the manager and host from the GridResource attribute. For ec2 jobs the host will be the value of EC2RemoteVirtualMachineName attribute.GRID_STATUS
Used for the
STATUS
column of the-grid
output of condor_q. This function renders the status of grid jobs from the GridJobStatus attribute. If the attribute has a string value it is reported unmodified. Otherwise, if GridJobStatus is an integer, it is presumed to be a condor job status and converted to a string.JOB_DESCRIPTION
Used for the Cmd column of the default output of condor_q. This function renders a job description from the
MATCH_EXP_JobDescription
, JobDescription or Cmd and Args or Arguments job attributes.JOB_FACTORY_MODE
Used for the
MODE
column of the-factory
output of condor_q. This function renders an integer value into a string value using the conversion forJobMaterializePaused
modes.JOB_ID
Used for the
ID
column of the default output of condor_q. This function renders a string job id from the ClusterId and ProcId attributes of the job.JOB_STATUS
Used for the
ST
column of the default output of condor_q. This function renders a one or two character job status from the JobStatus, TransferringInput, TransferringOutput, TransferQueued and LastSuspensionTime attributes of the job.JOB_STATUS_RAW
This function converts an integer to a string using the conversion for JobStatus values.
JOB_UNIVERSE
Used for the
UNIVERSE
column of the-idle
and-autocluster
output of condor_q. This function converts an integer to a string using the conversion for JobUniverse values. Values that are outside the range of valid universes are rendered asUnknown
.MEMORY_USAGE
Used for the
SIZE
column of the default output of condor_q. This function renders a memory usage value in megabytes the MemoryUsage or ImageSize attributes of the job.- Owner
Used for the Owner column of the default output of condor_q. This function renders an Owner string from the Owner attribute of the job. Prior to 8.9.9, this function would modify the result based on the NiceUser attribute of the job, but it no longer does so.
- QDate
Used for the
SUBMITTED
column of the default output of condor_q. This function converts a Unix timestamp to a string date and time with 2 digit month, day, hour and minute values.READABLE_BYTES
Used for the
INPUT
andOUTPUT
columns of the-io
output of condor_q This function renders a numeric byte value into a string with an appropriate B, KB, MB, GB, or TB suffix.READABLE_KB
This function renders a numeric Kibibyte value into a string with an appropriate B, KB, MB, GB, or TB suffix. Use this for Job attributes that are valued in Kb, such as DiskUsage.
READABLE_MB
This function renders a numeric Mibibyte value into a string with an appropriate B, KB, MB, GB, or TB suffix. Use this for Job attributes that are valued in Mb, such as MemoryUsage.
REMOTE_HOST
Used for the
HOST(S)
column of the-run
output of condor_q. This function extracts the host name from a job attribute appropriate to the JobUniverse value of the job. For Local and Scheduler universe jobs, the Schedd that was queried is used using a variable internal to condor_q. For grid universe jobs, the EC2RemoteVirtualMachineName or GridResource attributes are used. for all other universes theRemoteHost
job attribute is used.STDU_GOODPUT
Used for the
GOODPUT
column of the-goodput
output of condor_q. This function renders a floating point goodput time in seconds from the JobStatus, CommittedTime,ShadowBDay
,LastCkptTime
, and RemoteWallClockTime attributes.STDU_MPBS
Used for the
Mb/s
column of the-goodput
output of condor_q. This function renders a Megabytes per second goodput value from theBytesSent
,BytesRecvd
job attributes and total job execution time as calculated by theSTDU_GOODPUT
output.
PRINTAS functions for condor_status
ACTIVITY_CODE
Render a two character machine state and activity code from the State and Activity attributes of the machine ad. The letter codes for State are:
~
None
O
Owner
U
Unclaimed
M
Matched
C
Claimed
P
Preempting
S
Shutdown
X
Delete
F
Backfill
D
Drained
#
<undefined>
?
<error>
The letter codes for Activity are:
0
None
i
Idle
b
Busy
r
Retiring
v
Vacating
s
Suspended
b
Benchmarking
k
Killing
#
<undefined>
?
<error>
For example if State is Claimed and Activity is Idle, then this function returns Ci.
ACTIVITY_TIME
Used for the
ActvtyTime
column of the default output of condor_status. The function renders the given Unix timestamp as an elapsed time since theMyCurrentTime
orLastHeardFrom
attribute.CONDOR_PLATFORM
Used for the optional
Platform
column of the-master
output of condor_status. This function extracts the Arch and Opsys information from the given string.CONDOR_VERSION
Used for the
Version
column of the-master
output of condor_status. This function extract the version number and build id from the given string.DATE
This function converts a Unix timestamp to a string date and time with 2 digit month, day, hour and minute values.
DUE_DATE
This function converts an elapsed time to a Unix timestamp by adding the
LastHeardFrom
attribute to it, and then converts it to a string date and time with 2 digit month, day, hour and minute values.ELAPSED_TIME
Used in multiple places, for instance the
Uptime
column of the-master
output of condor_status. This function converts a Unix timestamp to an elapsed time by subtracting it from theLastHeardFrom
attribute, then formats it as a human readable elapsed time.LOAD_AVG
Used for the
LoadAv
column of the default output of condor_status Render the given floating point value using%.3f
format.PLATFORM
Used for the
Platform
column of the-compact
output of condor_status. Render a compact platform name from the value of the OpSys, OpSysAndVer, OpSysShortName and Arch attributes.READABLE_KB
This function renders a numeric Kibibyte value into a string with an appropriate B, KB, MB, GB, or TB suffix. Use this for Job attributes that are valued in Kb, such as DiskUsage.
READABLE_MB
This function renders a numeric Mibibyte value into a string with an appropriate B, KB, MB, GB, or TB suffix. Use this for Job attributes that are valued in Mb, such as MemoryUsage.
STRINGS_FROM_LIST
Used for the
Offline Universes
column of the-offline
output of condor_status. This function converts a ClassAd list into a string containing a comma separated list of items.TIME
Used for the
KbdIdle
column of the default output of condor_status. This function converts a numeric time in seconds into a string time including number of days, hours, minutes and seconds.UNIQUE
Used for the
Users
column of the compact-claimed
output of condor_status This function converts a classad list into a string containing a comma separate list of unique items.