GCB Tutorial European Condor Week June 2006 INFN Milan, Italy

GCB Tutorial European Condor Week June 2006 INFN Milan, Italy Todd Tannenbaum Condor Team http://www.cs.wisc.edu/condor What is GCB? › GCB is the Generic Connection Broker Linux-only Included in Condor 6.7.13 (Nov 2005) and later › It solves the “firewall traversal problem” › So what is the firewall traversal problem? www.cs.wisc.edu/condor A Simple Condor Pool Communication is initiated in two directions Matchmaker Submitter Note: This is a subset of communication in Condor Executor www.cs.wisc.edu/condor What If There Is A Firewall? › Firewalls usually block incoming traffic on › most ports “Incoming” depends on your perspective: computers outside the organization Individual computers have firewalls to protect from other computers Organizations have firewalls to protect from www.cs.wisc.edu/condor A Condor Pool With Firewall X Matchmaker X Submitter Executor www.cs.wisc.edu/condor How Can You Traverse Firewalls? › Punch a hole Configure firewall to allow traffic on a certain range of ports to come through Tell Condor to restrict itself to use only this range Bummer: Condor can use many ports Bummer: Punching holes makes people nervous www.cs.wisc.edu/condor How Can You Traverse Firewalls? › Use Condor-C Put host on network edge Open a couple of ports for it Delegate jobs to this host Matchmaker Re-Submitter www.cs.wisc.edu/condor Submitter Executor How Can You Traverse Firewalls? › Change Condor to always use outgoing traffic What if there are two firewalls or private › GCB automates this solution networks? Which direction is “outgoing”? It knows which direction is outgoing It can proxy if there are two firewalls www.cs.wisc.edu/condor GCB: Contacting Executor (One Possible Scenario) 2 1. Executor registers with GCB (Permanent TCP connection) 2. Executor advertises to matchmaker (GCB IP address) 3. After match, submitter contacts executor, via GCB 4. GCB tell executor to open connection 5. Executor opens connection to submitter Matchmaker 3 GCB 4 1 Executor 5 Submitter www.cs.wisc.edu/condor (Acting as Proxy) 1. Assume 1 port open for matchmaker. (Can avoid…) GCB 1 3 2. Executor advertises with GCB (permanent connection) 3. Executor advertises to Matchmaker matchmaker (GCB IP address) 4. After match, submitter contacts executor, via GCB 5. Communication flows through GCB, using both connections GCB 4 5 2 Executor Submitter www.cs.wisc.edu/condor GCB Advantages › Good connectivity Works with multiple private networks Works with network address translation › Don’t need to punch holes in firewall › GCB does not need to be run as root › No changes to firewall configuration www.cs.wisc.edu/condor › GCB is a point of failure address (of GCB) GCB Disadvantages › Computers behind a firewall share an IP › Doesn’t work with Kerberos security › Can slow down network performance › Scalability issues › Complex to configure and debug www.cs.wisc.edu/condor  All communications through GCB, so if GCB fails…  Makes host-based security difficult  A single GCB server is limited by number of ports available on computer Now for the Nitty Gritty… www.cs.wisc.edu/condor Setting Up GCB 1. Install GCB 2. Configure GCB 3. Configure Condor to use GCB www.cs.wisc.edu/condor › GCB comes with Condor › GCB has two programs Install GCB gcb_broker: The “big brains” of GCB gcb_relay_server: proxy for private net › GCB was written independently of Condor Can’t read condor_config directly So create environment in condor_config GCB reads from environment to private net communication www.cs.wisc.edu/condor Install GCB › GCB should be on computer with no other services GCB can use lots of ports, so avoid port › GCB needs to be on edge of network On public network and private network At least one GCB per private network www.cs.wisc.edu/condor competition with other programs Using GCB can slow down communication, so keeping GCB on its own computer helps speed Configure GCB › To run from condor_master: # Specify that you only want the master # and the broker running DAEMON_LIST = MASTER, GCB_BROKER # Define the path to the broker binary # for the master to spawn GCB_BROKER=$(RELEASE_DIR)/libexec/gcb_broker www.cs.wisc.edu/condor › GCB expects configuration in environment. Sample: GCB_BROKER_ENVIRONMENT = Configure GCB ## Providefull pathfull gcb_relay_server the to the path to the gcb_relay_server Provide the GCB_BROKER_ENVIRONMENT GCB_RELAY_SERVER=$(GCB_RELAY) GCB_BROKER_ENV = = GCB_RELAY_SERVER=$(GCB_RELAY) # Tell GCB to write all log files into the # Tell GCB to write all log files into the Condor log ## Condor logmore configuration options are available. directory Note: directory GCB_BROKER_ENVIRONMENT=(GCB_BROKER_ENVIRONMENT);GCB_LOG_DIR=$(LOG) GCB_BROKER_ENV=$(GCB_BROKER_ENV);GCB_LOG_DIR=$(LOG) # Tell GCB it can connect to private network See manual for details # Tell GCB it can connect to private network GCB_BROKER_ENV = GCB_BROKER_ENVIRONMENT=$(GCB_BROKER_ENVIRONMENT);GCB_ACTIVE_TO_CLIENT=yes $(GCB_BROKER_ENV);GCB_ACTIVE_TO_CLIENT=yes ## Set public IPfor GCB broker for GCB broker address Set public IP address GCB_BROKER_ARGS = -i 123.123.123.123 GCB_BROKER_ARGS = -i 123.123.123.123 www.cs.wisc.edu/condor › In condor_config: Configure Condor to Use GCB Turn on GCB: NET_REMAP_ENABLE = true NET_REMAP_SERVICE = GCB # Point to GCB NET_REMAP_INAGENT = 123.123.123.123 # Routing Table NET_REMAP_ROUTE = /full/path/gcbroutes www.cs.wisc.edu/condor Set Up Routing Table Public Network 123.123.123.* Private Network 192.168.2.* GCB Broker 123.123.123.123 Routing Table 123.123.123.123/32 GCB */0 direct www.cs.wisc.edu/condor Set Up Routing Table Public Network 123.123.123.* GCB Broker 123.123.123.65 Private Network 192.168.2.* Routing Table 123.123.123.65/32 123.123.123.66/32 */0 GCB Broker 123.123.123.66 GCB GCB direct Private Network 192.168.2.* www.cs.wisc.edu/condor Security Implications › Hosts in private network look like they › › share a single IP Address (the address of the GCB broker) If you use host-based security, you can’t distinguish hosts in the private network GCB does not authenticate who it is providing its proxy service for. www.cs.wisc.edu/condor More Information › Section 3.8 of the Condor manual “Networking” › http://www.cs.wisc.edu/~sschang/firewall/gcb Thank You!!! www.cs.wisc.edu/condor