Cloud Computing

Although HTCondor has long supported accessing cloud resources as though they were part of the Grid, the differences between clouds and the Grid have made it difficult to convert access into utility; a job in the Grid universe starts a virtual machine, rather than the user’s executable.

We offer two solutions to this problem. The first, a tool called condor_annex, helps users or administrators extend an existing HTCondor pool with cloud resources. The second is an easy way to create an entire HTCondor pool from scratch on the cloud, using our Google Cloud Marketplace Entry.

The rest of this chapter is concerned with using the condor_annex tool to add nodes to an existing HTCondor pool; it includes instructions on how to create a single-node HTCondor installation as a normal user so that you can expand it with cloud resources. It also discusses how to manually construct a HTCondor in the Cloud using condor_annex.

• Introduction
  • Use Case: Deadlines
  • Use Case: Capabilities
  • Use Case: Capacities
  • Use Case: Experimental Convenience
• HTCondor Annex User’s Guide
  • Considerations and Limitations
  • Basic Usage
  • Start an Annex
  • Monitor your Annex
  • Run a Job
  • Stop an Annex
  • Using Different or Multiple AWS Regions
  • Advanced Usage
• Using condor_annex for the First Time
  • Install a Personal HTCondor
  • Prepare your AWS account
  • Configure condor_annex
• HTCondor Annex Customization Guide
  • Amazon Web Services
• HTCondor Annex Configuration
  • User Settings
  • Logging
  • Expert Settings
  • Developer Settings
• HTCondor in the Cloud
  • The HTCondor in the Cloud Seed
  • Security
  • Making a HTCondor in the Cloud
  • Creating a Seed
• Google Cloud Marketplace Entry
• Google Cloud HPC Toolkit