{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Personal Pools\n", "\n", "Launch this tutorial in a Jupyter Notebook on Binder: \n", "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/htcondor/htcondor-python-bindings-tutorials/master?urlpath=lab/tree/Personal-Pools.ipynb)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A Personal HTCondor Pool is an HTCondor Pool that has a single owner, who is:\n", "- The pool’s administrator.\n", "- The only submitter who is allowed to submit jobs to the pool.\n", "- The owner of all resources managed by the pool.\n", "\n", "The HTCondor Python bindings provide a submodule, `htcondor.personal`, which allows you to manage personal pools from Python.\n", "Personal pools are useful for:\n", "- Utilizing local computational resources (i.e., all of the cores on a lab server).\n", "- Created an isolated testing/development environment for HTCondor workflows.\n", "- Serving as an entrypoint to other computational resources, like annexes or flocked pools (not yet implemented)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can start a personal pool by instantiating a `PersonalPool`.\n", "This object represents the personal pool and lets us manage its \"lifecycle\": start up and shut down.\n", "We can also use the `PersonalPool` to interact with the HTCondor pool once it has been started up.\n", "\n", "Each Personal Pool must have a unique \"local directory\", corresponding to the HTCondor configuration parameter `LOCAL_DIR`. For this tutorial, we'll put it in the current working directory so that it's easy to find.\n", "\n", "> Advanced users can configure the personal pool using the `PersonalPool` constructor. See the documentation for details on the available options." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:15:54.917439Z", "iopub.status.busy": "2021-09-16T13:15:54.916430Z", "iopub.status.idle": "2021-09-16T13:15:54.981456Z", "shell.execute_reply": "2021-09-16T13:15:54.982135Z" } }, "outputs": [], "source": [ "import htcondor\n", "from htcondor.personal import PersonalPool\n", "from pathlib import Path" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:15:54.987077Z", "iopub.status.busy": "2021-09-16T13:15:54.986239Z", "iopub.status.idle": "2021-09-16T13:15:55.050831Z", "shell.execute_reply": "2021-09-16T13:15:55.052234Z" } }, "outputs": [ { "data": { "text/plain": [ "PersonalPool(local_dir=./personal-condor, state=INITIALIZED)" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pool = PersonalPool(local_dir = Path.cwd() / \"personal-condor\")\n", "pool" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To tell the personal pool to start running, call the `start()` method:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:15:55.058386Z", "iopub.status.busy": "2021-09-16T13:15:55.057476Z", "iopub.status.idle": "2021-09-16T13:16:00.937696Z", "shell.execute_reply": "2021-09-16T13:16:00.939859Z" } }, "outputs": [ { "data": { "text/plain": [ "PersonalPool(local_dir=./personal-condor, state=READY)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pool.start()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`start()` doesn't return until the personal pool is `READY`, which means that it can accept commands (e.g., job submission)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`Schedd` and `Collector` objects for the personal pool are available as properties on the `PersonalPool`:" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:00.969558Z", "iopub.status.busy": "2021-09-16T13:16:00.964939Z", "iopub.status.idle": "2021-09-16T13:16:01.040226Z", "shell.execute_reply": "2021-09-16T13:16:01.049981Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pool.schedd" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:01.078300Z", "iopub.status.busy": "2021-09-16T13:16:01.074272Z", "iopub.status.idle": "2021-09-16T13:16:01.151102Z", "shell.execute_reply": "2021-09-16T13:16:01.153306Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pool.collector" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "For example, we can submit jobs using `pool.schedd`:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:01.175090Z", "iopub.status.busy": "2021-09-16T13:16:01.165822Z", "iopub.status.idle": "2021-09-16T13:16:01.266290Z", "shell.execute_reply": "2021-09-16T13:16:01.269344Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "ClusterID is 2\n" ] } ], "source": [ "sub = htcondor.Submit(\n", " executable = \"/bin/sleep\",\n", " arguments = \"$(ProcID)s\",\n", ")\n", "\n", "schedd = pool.schedd\n", "submit_result = schedd.submit(sub, count=10)\n", "\n", "print(f\"ClusterID is {submit_result.cluster()}\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "And we can query for the state of those jobs:" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:01.285631Z", "iopub.status.busy": "2021-09-16T13:16:01.281492Z", "iopub.status.idle": "2021-09-16T13:16:01.340355Z", "shell.execute_reply": "2021-09-16T13:16:01.336797Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[ ClusterID = 2; ProcID = 0; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 1; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 2; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 3; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 4; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 5; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 6; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 7; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 8; JobStatus = 1; ServerTime = 1631798161 ]\n", "[ ClusterID = 2; ProcID = 9; JobStatus = 1; ServerTime = 1631798161 ]\n" ] } ], "source": [ "for ad in pool.schedd.query(\n", " constraint = f\"ClusterID == {submit_result.cluster()}\", \n", " projection = [\"ClusterID\", \"ProcID\", \"JobStatus\"]\n", "):\n", " print(repr(ad))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can use the collector to query the state of pool:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:01.366786Z", "iopub.status.busy": "2021-09-16T13:16:01.358969Z", "iopub.status.idle": "2021-09-16T13:16:01.431402Z", "shell.execute_reply": "2021-09-16T13:16:01.432610Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", " [\n", " UpdateSequenceNumber = 1; \n", " TargetType = \"none\"; \n", " AuthenticationMethod = \"FAMILY\"; \n", " Name = \"jovyan@abae0fbbde81\"; \n", " AccountingGroup = \"\"; \n", " WeightedUnchargedTime = 0.0; \n", " DaemonStartTime = 1631798156; \n", " WeightedResourcesUsed = 2.000000000000000E+00; \n", " LastHeardFrom = 1631798160; \n", " Priority = 5.000000000000000E+02; \n", " LastUpdate = 1631798160; \n", " SubmitterLimit = 2.000000000000000E+00; \n", " MyType = \"Accounting\"; \n", " PriorityFactor = 1.000000000000000E+03; \n", " IsAccountingGroup = false; \n", " Ceiling = -1; \n", " ResourcesUsed = 1; \n", " DaemonLastReconfigTime = 1631798156; \n", " AuthenticatedIdentity = \"condor@family\"; \n", " NegotiatorName = \"jovyan@abae0fbbde81\"; \n", " UnchargedTime = 0; \n", " SubmitterShare = 1.000000000000000E+00\n", " ]\n", "\n", " [\n", " AuthenticatedIdentity = \"condor@family\"; \n", " EffectiveQuota = 0.0; \n", " GroupSortKey = 0.0; \n", " ResourcesUsed = 1; \n", " PriorityFactor = 1.000000000000000E+03; \n", " NegotiatorName = \"jovyan@abae0fbbde81\"; \n", " Name = \"\"; \n", " AccumulatedUsage = 0.0; \n", " ConfigQuota = 0.0; \n", " LastHeardFrom = 1631798160; \n", " SubtreeQuota = 0.0; \n", " DaemonStartTime = 1631798156; \n", " LastUsageTime = 0; \n", " SurplusPolicy = \"byquota\"; \n", " TargetType = \"none\"; \n", " AuthenticationMethod = \"FAMILY\"; \n", " LastUpdate = 1631798160; \n", " WeightedAccumulatedUsage = 0.0; \n", " Priority = 5.000000000000000E+02; \n", " MyType = \"Accounting\"; \n", " IsAccountingGroup = true; \n", " BeginUsageTime = 0; \n", " AccountingGroup = \"\"; \n", " UpdateSequenceNumber = 3; \n", " DaemonLastReconfigTime = 1631798156; \n", " WeightedResourcesUsed = 2.000000000000000E+00; \n", " Requested = 0.0\n", " ]\n", "\n", " [\n", " CCBReconnects = 0; \n", " MachineAdsPeak = 0; \n", " DetectedCpus = 2; \n", " UpdatesInitial_Accouting = 1; \n", " CurrentJobsRunningLinda = 0; \n", " StatsLifetime = 1; \n", " MaxJobsRunningAll = 0; \n", " CondorPlatform = \"$CondorPlatform: X86_64-Ubuntu_20.04 $\"; \n", " MaxJobsRunningJava = 0; \n", " MaxJobsRunningGrid = 0; \n", " MaxJobsRunningPVMD = 0; \n", " RecentUpdatesLostMax = 0; \n", " UpdatesLost = 0; \n", " RecentUpdatesLostRatio = 0.0; \n", " MonitorSelfRegisteredSocketCount = 2; \n", " UpdatesTotal_Collector = 1; \n", " MonitorSelfTime = 1631798156; \n", " RecentUpdatesTotal_Collector = 1; \n", " CondorAdmin = \"root@abae0fbbde81\"; \n", " MaxJobsRunningLinda = 0; \n", " CurrentJobsRunningPVM = 0; \n", " UpdatesLost_Collector = 0; \n", " CCBRequests = 0; \n", " CurrentJobsRunningPipe = 0; \n", " RecentUpdatesLost_Negotiator = 0; \n", " RecentUpdatesTotal = 3; \n", " RecentCCBRequestsFailed = 0; \n", " MaxJobsRunningVM = 0; \n", " CCBEndpointsConnected = 0; \n", " UpdatesLost_Accouting = 0; \n", " CurrentJobsRunningScheduler = 0; \n", " CurrentJobsRunningVanilla = 0; \n", " IdleJobs = 0; \n", " RecentUpdatesInitial_Accouting = 1; \n", " PendingQueriesPeak = 0; \n", " RecentUpdatesLost_Accouting = 0; \n", " ActiveQueryWorkersPeak = 2; \n", " MonitorSelfAge = 1; \n", " MonitorSelfCPUUsage = 1.800000000000000E+01; \n", " PendingQueries = 0; \n", " ActiveQueryWorkers = 0; \n", " DetectedMemory = 1988; \n", " CurrentJobsRunningMPI = 0; \n", " UpdateInterval = 21600; \n", " CurrentJobsRunningPVMD = 0; \n", " DroppedQueries = 0; \n", " RecentCCBRequestsSucceeded = 0; \n", " CCBEndpointsConnectedPeak = 0; \n", " StatsLastUpdateTime = 1631798157; \n", " CondorVersion = \"$CondorVersion: 8.9.11 Dec 29 2020 BuildID: Debian-8.9.11-1.2 PackageID: 8.9.11-1.2 Debian-8.9.11-1.2 $\"; \n", " MaxJobsRunningPipe = 0; \n", " CurrentJobsRunningParallel = 0; \n", " CCBEndpointsRegisteredPeak = 0; \n", " UpdatesInitial_Collector = 1; \n", " RecentDaemonCoreDutyCycle = 3.488135394901704E-02; \n", " SubmitterAdsPeak = 0; \n", " RecentUpdatesTotal_Accouting = 1; \n", " DaemonCoreDutyCycle = 3.488135394901704E-02; \n", " UpdatesTotal_Accouting = 1; \n", " MaxJobsRunningParallel = 0; \n", " UpdatesTotal = 3; \n", " RecentStatsLifetime = 1; \n", " MonitorSelfSecuritySessions = 2; \n", " CCBEndpointsRegistered = 0; \n", " LastHeardFrom = 1631798157; \n", " ForkQueriesFromCOLLECTOR = 2; \n", " HostsTotal = 0; \n", " CurrentJobsRunningJava = 0; \n", " RecentUpdatesTotal_Negotiator = 1; \n", " RecentForkQueriesFromCOLLECTOR = 2; \n", " CurrentJobsRunningAll = 0; \n", " RecentCCBRequestsNotFound = 0; \n", " Name = \"My Pool - 127.0.0.1@abae0fbbde81\"; \n", " HostsOwner = 0; \n", " TargetType = \"\"; \n", " CCBRequestsNotFound = 0; \n", " CurrentJobsRunningStandard = 0; \n", " SubmitterAds = 0; \n", " UpdatesLost_Negotiator = 0; \n", " MonitorSelfResidentSetSize = 11084; \n", " CCBRequestsSucceeded = 0; \n", " RecentUpdatesLost_Collector = 0; \n", " RecentUpdatesInitial_Collector = 1; \n", " RecentUpdatesLost = 0; \n", " RecentCCBRequests = 0; \n", " UpdatesTotal_Negotiator = 1; \n", " UpdatesInitial_Negotiator = 1; \n", " RecentDroppedQueries = 0; \n", " CurrentJobsRunningUnknown = 0; \n", " RecentUpdatesInitial_Negotiator = 1; \n", " HostsUnclaimed = 0; \n", " MachineAds = 0; \n", " RecentCCBReconnects = 0; \n", " UpdatesLostMax = 0; \n", " CollectorIpAddr = \"<172.17.0.2:46143?addrs=172.17.0.2-46143&alias=abae0fbbde81&noUDP&sock=collector>\"; \n", " UpdatesInitial = 3; \n", " HostsClaimed = 0; \n", " MaxJobsRunningLocal = 0; \n", " AddressV1 = \"{[ p=\\\"primary\\\"; a=\\\"172.17.0.2\\\"; port=46143; n=\\\"Internet\\\"; alias=\\\"abae0fbbde81\\\"; spid=\\\"collector\\\"; noUDP=true; ], [ p=\\\"IPv4\\\"; a=\\\"172.17.0.2\\\"; port=46143; n=\\\"Internet\\\"; alias=\\\"abae0fbbde81\\\"; spid=\\\"collector\\\"; noUDP=true; ]}\"; \n", " MaxJobsRunningUnknown = 0; \n", " MyAddress = \"<172.17.0.2:46143?addrs=172.17.0.2-46143&alias=abae0fbbde81&noUDP&sock=collector>\"; \n", " Machine = \"abae0fbbde81\"; \n", " CurrentJobsRunningGrid = 0; \n", " RunningJobs = 0; \n", " MyType = \"Collector\"; \n", " MaxJobsRunningMPI = 0; \n", " MaxJobsRunningScheduler = 0; \n", " MyCurrentTime = 1631798156; \n", " RecentUpdatesInitial = 3; \n", " UpdatesLostRatio = 0.0; \n", " MaxJobsRunningVanilla = 0; \n", " CurrentJobsRunningLocal = 0; \n", " CCBRequestsFailed = 0; \n", " CurrentJobsRunningVM = 0; \n", " MaxJobsRunningStandard = 0; \n", " MonitorSelfImageSize = 16224; \n", " MaxJobsRunningPVM = 0\n", " ]\n" ] } ], "source": [ "# get 3 random ads from the daemons in the pool\n", "for ad in pool.collector.query()[:3]:\n", " print(ad)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When you're done using the personal pool, you can `stop()` it:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:01.439413Z", "iopub.status.busy": "2021-09-16T13:16:01.437242Z", "iopub.status.idle": "2021-09-16T13:16:03.527598Z", "shell.execute_reply": "2021-09-16T13:16:03.529149Z" } }, "outputs": [ { "data": { "text/plain": [ "PersonalPool(local_dir=./personal-condor, state=STOPPED)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pool.stop()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`stop()`, like `start()` will not return until the personal pool has actually stopped running.\n", "The personal pool will also automatically be stopped if the `PersonalPool` object is garbage-collected, or when the Python interpreter stops running.\n", "\n", "> To prevent the pool from being automatically stopped in these situations, call the `detach()` method. The corresponding `attach()` method can be used to \"re-connect\" to a detached personal pool." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When working with a personal pool in a script, you may want to use it as a context manager. This pool will automatically start and stop at the beginning and end of the context:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2021-09-16T13:16:03.539557Z", "iopub.status.busy": "2021-09-16T13:16:03.537690Z", "iopub.status.idle": "2021-09-16T13:16:11.681575Z", "shell.execute_reply": "2021-09-16T13:16:11.683204Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/jovyan/tutorials/another-personal-condor\n" ] } ], "source": [ "with PersonalPool(local_dir = Path.cwd() / \"another-personal-condor\") as pool: # note: no need to call start()\n", " print(pool.get_config_val(\"LOCAL_DIR\"))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.6" } }, "nbformat": 4, "nbformat_minor": 4 }