DAGMan Workflows

• DAGMan Introduction
  • Describing Workflows with DAGMan
  • Example: Diamond DAG
  • JOB
  • PARENT/CHILD Relationships
  • Node Job Submit File Contents
• Scripts
  • PRE and POST scripts
  • HOLD scripts
  • DEFER retries
  • Scripts as part of a DAG workflow
  • Examples that use PRE or POST scripts
  • Special script argument macros
• Node Success/Failure
  • PRE_SKIP
  • Retrying Failed Nodes
  • Stopping the DAG on Node Failure
• File Paths in DAGs
• Running and Managing DAGMan
  • DAG Submission
  • DAG Monitoring
  • Editing a Running DAG
  • Removing a DAG
  • Suspending a Running DAG
• DAG Save Point Files
• Resubmitting a Failed DAG
  • The Rescue DAG
  • DAG Recovery
• Node Priorities
  • Setting Priorities for Nodes
  • Effective node priorities
• Single Submission of Multiple, Independent DAGs
• Composing workflows from multiple DAG files
  • A DAG Within a DAG Is a SUBDAG
  • DAG Splicing
• DAGMan Throttling
  • Throttling at DAG Submission
  • Throttling Nodes by Category
• Optimization of Submission Time
• Managing Large Numbers of Jobs with DAGMan
• Custom Variables for Nodes
  • Variable Values Associated with Nodes
  • Prepend or Append Variables to Node
  • Multiple macroname definitions
  • Special characters within VARS string definitions
  • Using special macros within a definition
  • Using VARS to define ClassAd attributes
• DAG Manager Job Specifications
  • Classad Attributes in the DAG Manager Job
  • Environment Variables in the DAG Manager Job
• Configuration Specific to a DAG
• INCLUDE
  • Example: Using INCLUDE to simplify multiple similar workflows
• ALL_NODES Option
• DAGMan and Accounting Groups
• Special Node Types
  • FINAL Node
  • PROVISIONER Node
  • SERVICE Node
• Visualizing DAGs
• Capturing the Status of Nodes in a File
• Machine-Readable Event History
• Workflow Metrics