The HTCondor User and Job Log Reader API¶
HTCondor has the ability to log an HTCondor job’s significant events during its lifetime. This is enabled in the job’s submit description file with the Log command.
This section describes the API defined by the C++ ReadUserLog
class,
which provides a programming interface for applications to read and
parse events, polling for events, and saving and restoring reader state.
Constants and Enumerated Types¶
The following define enumerated types useful to the API.
ULogEventOutcome
(defined incondor_event.h
):ULOG_OK
: Event is validULOG_NO_EVENT
: No event occurred (like EOF)ULOG_RD_ERROR
: Error reading log fileULOG_MISSED_EVENT
: Missed eventULOG_UNK_ERROR
: Unknown Error
ReadUserLog::FileStatus
LOG_STATUS_ERROR
: An error was encounteredLOG_STATUS_NOCHANGE
: No change in file sizeLOG_STATUS_GROWN
: File has grownLOG_STATUS_SHRUNK
: File has shrunk
Constructors and Destructors¶
All ReadUserLog
constructors invoke one of the initialize()
methods. Since C++ constructors cannot return errors, an application
using any but the default constructor should call isIinitialized()
to verify that the object initialized correctly, and for example, had
permissions to open required files.
Note that because the constructors cannot return status information,
most of these constructors will be eliminated in the future. All
constructors, except for the default constructor with no parameters,
will be removed. The application will need to call the appropriate
initialize()
method.
ReadUserLog::ReadUserLog(bool isEventLog) Synopsis: Constructor default Returns: None Constructor parameters:
bool
isEventLog
(Optional with default =false
) Iftrue
, theReadUserLog
object is initialized to read the schedd-wide event log. NOTE: IfisEventLog
istrue
, the initialization may silently fail, so the value of ReadUserLog::isInitialized should be checked to verify that the initialization was successful. NOTE: TheisEventLog
parameter will be removed in the future.
- ReadUserLog::ReadUserLog(FILE *fp, bool is_xml, bool enable_closeSynopsis: Constructor of a limited functionality reader: no rotation handling, no lockingReturns: NoneConstructor parameters:
FILE *
fp
File pointer to the previously opened log file to read.bool
is_xml
Iftrue
, the file is treated as XML; otherwise, it will be read as an old style file.bool
enable_close
(Optional with default =false
) Iftrue
, the reader will open the file read-only.
NOTE: The ReadUserLog::isInitialized method should be invoked to verify that this constructor was initialized successfully.NOTE: This constructor will be removed in the future. - ReadUserLog::ReadUserLog(const char *filename, bool read_only)Synopsis: Constructor to read a specific log fileReturns: NoneConstructor parameters:
const char *
filename
Path to the log file to readbool
read_only
(Optional with default =false
) Iftrue
, the reader will open the file read-only and disable locking.
NOTE: This constructor will be removed in the future.
- ReadUserLog::ReadUserLog(const FileState &state, bool read_only)Synopsis: Constructor to continue from a persisted reader stateReturns: NoneConstructor parameters:
const FileState &
state
Reference to the persisted state to restore frombool
read_only
(Optional with default =false
) Iftrue
, the reader will open the file read-only and disable locking.
NOTE: The ReadUserLog::isInitialized method should be invoked to verify that this constructor was initialized successfully.NOTE: This constructor will be removed in the future. ReadUserLog::˜ReadUserLog(void) Synopsis: Destructor Returns: None Destructor parameters:
None.
Initializers¶
These methods are used to perform the initialization of the
ReadUserLog
objects. These initializers are used by all constructors
that do real work. Applications should never use those constructors,
should use the default constructor, and should instead use one of these
initializer methods.
All of these functions will return false
if there are problems such
as being unable to open the log file, or true
if successful.
bool
ReadUserLog::initialize(void) Synopsis: Initialize to read the EventLog file. NOTE: This method will likely be eliminated in the future, and this functionality will be moved to a newReadEventLog
class. Returns:bool
;true
: success,false
: failed Method parameters:None.
bool
ReadUserLog::initialize(const char *filename, bool handle_rotation, bool check_for_rotated, bool read_only) Synopsis: Initialize to read a specific log file. Returns:bool
;true
: success,false
: failed Method parameters:const char *
filename
Path to the log file to readbool
handle_rotation
(Optional with default =false
) Iftrue
, enable the reader to handle rotating log files, which is only useful for global user logsbool
check_for_rotated
(Optional with default =false
) Iftrue
, try to open the rotated files (with file names appended with.old
or.1
,.2
, …) first.bool
read_only
(Optional with default =false
) Iftrue
, the reader will open the file read-only and disable locking.
bool
ReadUserLog::initialize(const char *filename, int max_rotation, bool check_for_rotated, bool read_only) Synopsis: Initialize to read a specific log file. Returns:bool
;true
: success,false
: failed Method parameters:const char *
filename
Path to the log file to readint
max_rotation
Limits what previously rotated files will be considered by the number given in the file name suffix. A value of 0 disables looking for rotated files. A value of 1 limits the rotated file to be that with the file name suffix of.old
. As only event logs are rotated, this parameter is only useful for event logs.bool
check_for_rotated
(Optional with default =false
) Iftrue
, try to open the rotated files (with file names appended with.old
or.1
,.2
, …) first.bool
read_only
(Optional with default =false
) Iftrue
, the reader will open the file read-only and disable locking.
bool
ReadUserLog::initialize(const FileState &state, bool read_only) Synopsis: Initialize to continue from a persisted reader state. Returns:bool
;true
: success,false
: failed Method parameters:const FileState &
state
Reference to the persisted state to restore frombool
read_only
(Optional with default =false
) Iftrue
, the reader will open the file read-only and disable locking.
bool
ReadUserLog::initialize(const FileState &state, int max_rotation, bool read_only) Synopsis: Initialize to continue from a persisted reader state and set the rotation parameters. Returns:bool
;true
: success,false
: failed Method parameters:const FileState &
state
Reference to the persisted state to restore fromint
max_rotation
Limits what previously rotated files will be considered by the number given in the file name suffix. A value of 0 disables looking for rotated files. A value of 1 limits the rotated file to be that with the file name suffix of.old
. As only event logs are rotated, this parameter is only useful for event logs.bool
read_only
(Optional with default =false
) Iftrue
, the reader will open the file read-only and disable locking.
Primary Methods¶
ULogEventOutcome
ReadUserLog::readEvent(ULogEvent *& event) Synopsis: Read the next event from the log file. Returns:ULogEventOutcome
; Outcome of the log read attempt.ULogEventOutcome
is an enumerated type. Method parameters:ULogEvent
*&event
Pointer to anULogEvent
that is allocated by this call to ReadUserLog::readEvent. If no event is allocated, this pointer is set toNULL
. Otherwise the event needs to be delete()ed by the application.
bool
ReadUserLog::synchronize(void) Synopsis: Synchronize the log file if the last event read was an error. This safe guard function should be called if there is some error reading an event, but there are events after it in the file. It will skip over the bad event, meaning it will read up to and including the event separator, so that the rest of the events can be read. Returns:bool
;true
: success,false
: failed Method parameters:None.
Accessors¶
ReadUserLog::FileStatus
ReadUserLog::CheckFileStatus(void) Synopsis: Check the status of the file, and whether it has grown, shrunk, etc. Returns:ReadUserLog::FileStatus
; the status of the log file, an enumerated type. Method parameters:None.
ReadUserLog::FileStatus
ReadUserLog::CheckFileStatus(bool &is_empty) Synopsis: Check the status of the file, and whether it has grown, shrunk, etc. Returns:ReadUserLog::FileStatus
; the status of the log file, an enumerated type. Method parameters:bool &
is_empty
Set totrue
if the file is empty,false
otherwise.
Methods for saving and restoring persistent reader state¶
The ReadUserLog::FileState
structure is used to save and restore the
state of the ReadUserLog
state for persistence. The application
should always use InitFileState() to initialize this structure.
All of these methods take a reference to a state buffer as their only parameter.
All of these methods return true
upon success.
Save state to persistent storage¶
To save the state, do something like this:
ReadUserLog reader;
ReadUserLog::FileState statebuf;
status = ReadUserLog::InitFileState( statebuf );
status = reader.GetFileState( statebuf );
write( fd, statebuf.buf, statebuf.size );
...
status = reader.GetFileState( statebuf );
write( fd, statebuf.buf, statebuf.size );
...
status = UninitFileState( statebuf );
Restore state from persistent storage¶
To restore the state, do something like this:
ReadUserLog::FileState statebuf;
status = ReadUserLog::InitFileState( statebuf );
read( fd, statebuf.buf, statebuf.size );
ReadUserLog reader;
status = reader.initialize( statebuf );
status = UninitFileState( statebuf );
....
API Reference¶
static
bool
ReadUserLog::InitFileState(ReadUserLog::FileState &state) Synopsis: Initialize a file state buffer Returns:bool
;true
if successful,false
otherwise Method parameters:ReadUserLog::FileState &
state
The file state buffer to initialize.
static
bool
ReadUserLog::UninitFileState(ReadUserLog::FileState &state) Synopsis: Clean up a file state buffer and free allocated memory Returns:bool
;true
if successful,false
otherwise Method parameters:ReadUserLog::FileState &
state
The file state buffer to un-initialize.
bool
ReadUserLog::GetFileState(ReadUserLog::FileState &state)const
Synopsis: Get the current state to persist it or save it off to disk Returns:bool
;true
if successful,false
otherwise Method parameters:ReadUserLog::FileState &
state
The file state buffer to read the state into.
bool
ReadUserLog::SetFileState(const ReadUserLog::FileState &state) Synopsis: Use this method to set the current state, after restoring it. NOTE: The state buffer is NOT automatically updated; a call MUST be made to the GetFileState() method each time before persisting the buffer to disk, or however else is chosen to persist its contents. Returns:bool
;true
if successful,false
otherwise Method parameters:const ReadUserLog::FileState &
state
The file state buffer to restore from.
Access to the persistent state data¶
If the application needs access to the data elements in a persistent
state, it should instantiate a ReadUserLogStateAccess
object.
Constructors / Destructors
ReadUserLogStateAccess::ReadUserLogStateAccess(const ReadUserLog::FileState &state) Synopsis: Constructor default Returns: None Constructor parameters:
const ReadUserLog::FileState &
state
Reference to the persistent state data to initialize from.
ReadUserLogStateAccess::˜ReadUserLogStateAccess(void) Synopsis: Destructor Returns: None Destructor parameters:
None.
Accessor Methods
bool
ReadUserLogStateAccess::isInitialized(void)const
Synopsis: Checks if the buffer initialized Returns:bool
;true
if successfully initialized,false
otherwise Method parameters:None.
bool
ReadUserLogStateAccess::isValid(void)const
Synopsis: Checks if the buffer is valid for use by ReadUserLog::initialize() Returns:bool
;true
if successful,false
otherwise Method parameters:None.
bool
ReadUserLogStateAccess::getFileOffset(unsigned long &pos)const
Synopsis: Get position within individual file. NOTE: Can return an error if the result is too large to be stored in along
. Returns:bool
;true
if successful,false
otherwise Method parameters:unsigned long &
pos
Byte position within the current log file
bool
ReadUserLogStateAccess::getFileEventNum(unsigned long &num)const
Synopsis: Get event number in individual file. NOTE: Can return an error if the result is too large to be stored in along
. Returns:bool
;true
if successful,false
otherwise Method parameters:unsigned long &
num
Event number of the current event in the current log file
bool
ReadUserLogStateAccess::getLogPosition(unsigned long &pos)const
Synopsis: Position of the start of the current file in overall log. NOTE: Can return an error if the result is too large to be stored in along
. Returns:bool
;true
if successful,false
otherwise Method parameters:unsigned long &
pos
Byte offset of the start of the current file in the overall logical log stream.
bool ReadUserLogStateAccess::getEventNumber(unsigned long &num)
const
Synopsis: Get the event number of the first event in the current file NOTE: Can return an error if the result is too large to be stored in along
. Returns: bool;true
if successful,false
otherwise Method parameters:unsigned long &
num
This is the absolute event number of the first event in the current file in the overall logical log stream.
bool ReadUserLogStateAccess::getUniqId(char *buf, int size)
const
Synopsis: Get the unique ID of the associated state file. Returns: bool;true
if successful,false
otherwise Method parameters:char *
buf
Buffer to fill with the unique ID of the current file.int
size
Size in bytes ofbuf
. This is to prevent ReadUserLogStateAccess::getUniqId from writing past the end ofbuf
.
bool
ReadUserLogStateAccess::getSequenceNumber(int &seqno)const
Synopsis: Get the sequence number of the associated state file. Returns:bool
;true
if successful,false
otherwise Method parameters:int &
seqno
Sequence number of the current file
Comparison Methods
bool
ReadUserLogStateAccess::getFileOffsetDiff(const ReadUserLogStateAccess &other, unsigned long &pos)const
Synopsis: Get the position difference of two states given bythis
andother
. NOTE: Can return an error if the result is too large to be stored in along
. Returns:bool
;true
if successful,false
otherwise Method parameters:const ReadUserLogStateAccess &
other
Reference to the state to compare to.long &
diff
Difference in the positions
bool ReadUserLogStateAccess::getFileEventNumDiff(const ReadUserLogStateAccess &other, long &diff)
const
Synopsis: Get event number in individual file. NOTE: Can return an error if the result is too large to be stored in along
. Returns: bool;true
if successful,false
otherwise Method parameters:const ReadUserLogStateAccess &
other
Reference to the state to compare to.long &
diff
Event number of the current event in the current log file
bool ReadUserLogStateAccess::getLogPosition(const ReadUserLogStateAccess &other, long &diff)
const
Synopsis: Get the position difference of two states given bythis
andother
. NOTE: Can return an error if the result is too large to be stored in along
. Returns: bool;true
if successful,false
otherwise Method parameters:const ReadUserLogStateAccess &
other
Reference to the state to compare to.long &
diff
Difference between the byte offset of the start of the current file in the overall logical log stream and that ofother
.
bool ReadUserLogStateAccess::getEventNumber(const ReadUserLogStateAccess &other, long &diff)
const
Synopsis: Get the difference between the event number of the first event in two state buffers (this - other). NOTE: Can return an error if the result is too large to be stored in along
. Returns: bool;true
if successful,false
otherwise Method parameters:const ReadUserLogStateAccess &
other
Reference to the state to compare to.long &
diff
Difference between the absolute event number of the first event in the current file in the overall logical log stream and that ofother
.
Future persistence API¶
The ReadUserLog::FileState
will likely be replaced with a new C++
ReadUserLog::NewFileState
, or a similarly named class that will self
initialize.
Additionally, the functionality of ReadUserLogStateAccess
will be
integrated into this class.