{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "pycharm": {}
      },
      "source": [
        "# Advanced Schedd Interaction\n",
        "\n",
        "Launch this tutorial in a Jupyter Notebook on Binder: \n",
        "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/htcondor/htcondor-python-bindings-tutorials/master?urlpath=lab/tree/Advanced-Schedd-Interactions.ipynb)\n",
        "\n",
        "The introductory tutorial only scratches the surface of what the Python bindings\n",
        "can do with the ``condor_schedd``; this module focuses on covering a wider range\n",
        "of functionality:\n",
        "\n",
        "*  Job and history querying.\n",
        "*  Advanced job submission.\n",
        "*  Python-based negotiation with the Schedd.\n",
        "\n",
        "As usual, we start by importing the relevant modules:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "pycharm": {}
      },
      "outputs": [],
      "source": [
        "import htcondor\n",
        "import classad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "pycharm": {}
      },
      "source": [
        "## Job and History Querying\n",
        "\n",
        "In [HTCondor Introduction](HTCondor-Introduction.ipynb), we covered the `Schedd.query` method\n",
        "and its two most important keywords:\n",
        "\n",
        "*  ``requirements``: Filters the jobs the schedd should return.\n",
        "*  ``projection``: Filters the attributes returned for each job.\n",
        "\n",
        "For those familiar with SQL queries, ``requirements`` performs the equivalent\n",
        "as the ``WHERE`` clause while ``projection`` performs the equivalent of the column\n",
        "listing in ``SELECT``.\n",
        "\n",
        "There are two other keywords worth mentioning:\n",
        "\n",
        "*  ``limit``: Limits the number of returned ads; equivalent to SQL's ``LIMIT``.\n",
        "*  ``opts``: Additional flags to send to the schedd to alter query behavior.\n",
        "   The only flag currently defined is `QueryOpts.AutoCluster`; this\n",
        "   groups the returned results by the current set of \"auto-cluster\" attributes\n",
        "   used by the pool.  It's analogous to ``GROUP BY`` in SQL, except the columns\n",
        "   used for grouping are controlled by the schedd.\n",
        "\n",
        "To illustrate these additional keywords, let's first submit a few jobs:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "pycharm": {}
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "19\n"
          ]
        }
      ],
      "source": [
        "schedd = htcondor.Schedd()\n",
        "sub = htcondor.Submit({\n",
        "    \"executable\": \"/bin/sleep\",\n",
        "    \"arguments\": \"5m\",\n",
        "    \"hold\": \"True\",\n",
        "})\n",
        "submit_result = schedd.submit(sub, count=10)\n",
        "print(submit_result.cluster())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "pycharm": {}
      },
      "source": [
        "**Note:** In this example, we used the ``hold`` submit command to indicate that\n",
        "the jobs should start out in the ``condor_schedd`` in the *Hold* state; this\n",
        "is used simply to prevent the jobs from running to completion while you are\n",
        "running the tutorial.\n",
        "\n",
        "We now have 10 jobs running under ``cluster_id``; they should all be identical:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "pycharm": {}
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "10\n"
          ]
        }
      ],
      "source": [
        "print(len(schedd.query(projection=[\"ProcID\"], constraint=f\"ClusterId=={submit_result.cluster()}\")))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "pycharm": {}
      },
      "source": [
        "## History Queries\n",
        "\n",
        "After a job has finished in the Schedd, it moves from the queue to the history file.  The\n",
        "history can be queried (locally or remotely) with the `Schedd.history` method:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "pycharm": {}
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "    [\n",
            "        JobStatus = 3; \n",
            "        ProcId = 99; \n",
            "        ClusterId = 18\n",
            "    ]\n",
            "\n",
            "    [\n",
            "        JobStatus = 3; \n",
            "        ProcId = 98; \n",
            "        ClusterId = 18\n",
            "    ]\n"
          ]
        }
      ],
      "source": [
        "schedd = htcondor.Schedd()\n",
        "for ad in schedd.history(\n",
        "    constraint='true',\n",
        "    projection=['ProcId', 'ClusterId', 'JobStatus'],\n",
        "    match=2,  # limit to 2 returned results\n",
        "):\n",
        "    print(ad)"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}