Internet Draft Andreas Papp Expiration date: November 2000 Effnet AB June 2000 Firewall Redundancy Protocol Specification Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of section 10 of RFC2026. Please send comments directly to the author . Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet- Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html . Abstract Firewalls are used to get controlled and secure connection between networks, e.g. a company's internal network and the Internet. Preferably the firewall is the only link between the networks to be able to guarantee a certain level of security. The firewall is a critical node in the network and if it would fail the result is lost connection between the networks. To ensure reliability and connectivity we add redundancy, i.e. a number of parallel firewalls are installed which will act as backups for each other. Firewall Redundancy Protocol (FRP) is a communication protocol for firewalls to make transparent redundancy possible. It makes cooperating firewalls act as if they were a single reliable firewall. The protocol includes a failure mechanism that specifies what happens if a participating firewall crashes and the responsibility for filtering traffic falls on another firewall. Firewall Redundancy Protocol uses multicast and alive messages to detect failures, TCP connections to transfer data and IP/MAC addresses to move traffic between participating firewalls. Priorities are used to ensure stability and convergence in the system. Table of Contents 1.0 ..................... Introduction 2.0 ......................... Overview 3.0 .................. Sample Scenario 4.0 ................... FRP Mechanisms 5.0 ................ FRP State Machine 6.0 ........................ Take Over 7.0 ... Synchronization Message Format 8.0 ........... Control Message Format 9.0 .......... Security Considerations 10.0 ....................... References 11.0 ................. Acknowledgements 12.0 ................. Author's Address 1.0 Introduction The Firewall Redundancy Protocol, FRP, defines how synchronization of a number of backup firewalls with a master firewall works. Furthermore FRP defines what happens in the situation when a failure occurs on the current master and one of the backup firewalls is elected to become the new master. This document starts with an overview and a sample scenario. The scenario intends to give the reader a model picture of the situation where FRP should be used. Details are then presented beginning with the common parameters of cooperating firewalls and the mechanisms of FRP. 1.1 Definitions Firewall Table - All active connections through a firewall are remembered as states in a firewall table. FRP - Firewall Redundancy Protocol. FRP Cooperation - A number of firewalls that cooperate using FRP are called a FRP Cooperation. Rule set - The set of rules that a firewall checks, before deciding if a connection should be let through or not. State - All characteristic parameters of a connection are collected in a state. By adding this state to the firewall table the connection gets a hole through the firewall. Another meaning of "state" is also used, as in "the state machine" where a firewall is in the backup state or master state. Which meaning is intended should be clear by the context. 2.0 Overview Firewalls are used to get a controlled and secure connection between networks, e.g. a company's internal network and the Internet. Preferably the firewall is the only link between the networks, which enables it to guarantee a certain level of security. The firewall is a critical node in the network and if it would fail the result is lost connection between the networks. To ensure reliability and connectivity we add redundancy, i.e. a number of parallel firewalls are installed which will act as backups for each other. The protocol, Firewall Redundancy Protocol (FRP) is a communication protocol for firewalls to make transparent redundancy possible. It makes the cooperating firewalls act as if they were a single reliable firewall. Figure 1 shows an example configuration. ISP | ---+----+---+--- Outside net | | +--+--+ +--+--+ | FW1 | | FW2 | +--+--+ +--+--+ | | -+-+---+----+-+- Inside net | | | Host1 Host2 Host3 Figure 1. Example of firewall constellation with FRP. The outside network is connected to the company's Internet provider, in the middle two firewalls cooperate using FRP and on the inside three users have their machines. Firewalls using FRP are placed in parallel between networks. One of the firewalls is active, i.e. filtering traffic, and called master. The others are not filtering traffic and called backups. FRP cooperating firewalls act as if they were (from an user's point of view) a single reliable firewall with cooperation specific IP and MAC addresses on the inside and on the outside. These addresses are the official addresses of the FRP cooperation. If a failure occurs, the master goes down and a backup machine takes over the official addresses, the users will never notice the change (except for a delay). The users will continue to send and receive to/from the same official IP and MAC addresses, but now there is actually another firewall doing the filtering. (Each firewall has also own IP and MAC addresses which in this document is called private addresses). The current master of a FRP cooperation filters all traffic to the cooperation's official addresses and sends synchronization data to the backups when it is needed. Synchronization data is sent over the inside net (or a special designated net) using a TCP-connection for each backup. Communication between firewalls inside the FRP cooperation uses the multicast channel or the private IP and MAC addresses of the firewalls, but never the official addresses. The master sends alive messages with regular time intervals, these are also sent over the inside network but using UDP on a multicast channel. If a failure occurs in the master firewall and it stops sending alive messages, the backups will (after a specified time interval) consider the current master down and try to elect a new master. Election of a new master depends on priorities, the firewall with the highest FRP priority will take over after the shortest time interval and thereby "elect" itself. When a backup takes over the master role, it starts to send alive messages over the multicast channel to avoid that another backup tries to become master. It also starts filtering traffic (for the cooperation's official addresses), waits for backup firewalls to connect and then starts to send synchronization data to all connected backups. 3.0 Sample Scenario A company has a network connected to an ISP via a firewall. They decide to add redundancy by connecting another firewall in parallel with the first firewall and use FRP. The first firewall that gets connected starts to listen for alive messages from any other firewall(s) belonging to the same FRP cooperation. No such alive messages will arrive and therefore it considers itself to be alone on the network and becomes master of the FRP cooperation. As master, the firewall filters traffic on the behalf of the FRP cooperation as well as sending alive messages with regular intervals. Then the next firewall is connected to the networks and starts to listen for alive messages. Soon one message is received and the newcomer knows that there is a firewall that belongs to this FRP cooperation and currently acts as master. It connects to the master with a TCP connection and downloads the complete firewall table. When this is done the newcomer becomes a backup firewall ready to take over in the case of failure. The backup receives synchronization data and continues to be updated. At the same time as it waits for synchronization data it also waits for alive messages from the master. If no alive message is received for a certain interval, the backup considers the master to have failed and tries to become master. Another thing could happen and that is if the second firewall realizes (when it initialized and becomes backup) that itself has higher priority than the current master. It then becomes master itself, which forces the first firewall to resign and become backup. 4.0 FRP Mechanisms The election of a new master and take over are similar to Virtual Router Redundancy Protocol, VRRP, which can be found at [VRRP]. Message formats and data transfer are completely different though. 4.1 FRP Cooperation The firewalls that cooperate using FRP are (together) called a FRP cooperation. A cooperation has a number of common parameters: - a common filter rule set, i.e. the same security level must apply to both the master and the backups. This is not synchronized in FRP version 2, which means that the participating firewalls must be configured with the same rule set. Otherwise the FRP cooperation will have the security level of weakest link, due to the fact that holes through the firewall cooperation will be synchronized. - an ID, which is a number between 0 - 255 unique for each FRP cooperation on the network. This is used to distinguish between different FRP cooperations when a message is received on the multicast channel. - an inside network IP address that all inside users of the FRP cooperation are connected to. This address will be active on exactly one participating firewall (the master) at a time which will do all filtering for the users. All participating firewalls have their own IP addresses as well, which are used to connect the backups via TCP to the master and to enable other access to specific firewalls. - an inside MAC address 00-00-5E-00-02-xx, where xx is the FRP cooperation ID (8 bits gives 255 possible IDs). These are not assigned nor reserved by any other protocol yet. Organizationally Unique Identifier, OUI, 00-00-5E is owned by IANA and may be used for unicast address assignments or other special purposes. [IANA Eth.no] 00-00-5E-00-00-00 to 00-00-5E-00-00-FF is reserved by IANA, 00-00-5E-00-01-00 to 00-00-5E-00-01-FF is assigned to VRRP, 00-00-5E-00-02-00 to 00-00-5E-FF-FF-FF not yet assigned. - an outside IP address that outside sources know about, this address will be active on exactly one participating firewall (the master), which is doing all filtering, at a time. - an outside MAC address 00-00-5E-00-02-xx, where xx is the FRP cooperation ID. This is the same as the inside MAC address. To use a not globally unique address implies that collisions may occur, then the ID could be changed and thereby also the colliding MAC address. - a multicast address 224.0.0.33. This is not assigned nor reserved by any other protocol yet. 224.0.0.0 - 224.0.0.255 is assigned as link local multicast addresses and 224.0.0.0 - 224.0.0.32 is already reserved, for example VRRP uses 224.0.0.18. - UDP port number 2925 for traffic over the multicast channel. - TCP port number 2925 for synchronization data traffic. - a FRP version number. This specification describes version 2. Those parameters are known by all participating firewalls. 4.2 Priorities Each firewall that runs FRP has a priority which is a value between 1 - 255. Priority zero (0) is reserved for a master that want to resign. The firewall that has the highest priority at the time will act as master of the FRP cooperation. If two firewalls has the same priority the one with the highest IP address becomes master. The default value is 100. Priorities are used to avoid conflicts when electing a new master and to get stability and predictability in the system. 4.3 Timer Intervals Three timer intervals are used: alive_interval, skew_interval and master_down_interval. Alive messages are sent with an alive_interval seconds time separation. The default value of alive_interval is one (1) second. If no alive message has arrived from the master in master_down_interval seconds the backup considers the master down. The master_down_interval is calculated as: (3 * alive_interval + skew_interval). The skew_interval is used when a new master is elected. It depends on the specific firewall's priority. A high priority firewall calculates a short skew_interval and will thereby react faster if a failure occurs on the master than a low priority firewall. The skew_interval is calculated as: ((255 - priority)/255) in seconds. 5.0 Protocol State Machine A firewall that runs FRP can be in one of five states: Initialize, Init Download, Backup, Master or Master Download. Transition between these states occur when certain criteria are fulfilled. +------+ --alone-on-network--------------> +--------+ | Init | <-----------------------failure-- | Master | | | +--------+ -take-over-> | | +------+ <-failure-| Backup | <---resign-- +--------+ | ^ | | | ^ v | +--------+ v | +------+ ^ +------+ | Down | | | Down | | load | --Ready------+ | load | +------+ +------+ Figure 2. State Machine with state and transition names 5.1 Initialize A firewall always starts in the initialize state. To be ready to act as a backup firewall the complete firewall table must be downloaded from the master. Actions: o It starts the timer with master_down_interval. o It listens on the multicast channel. - If an alive message is received, a TCP connection is setup with the master and the current firewall table should be downloaded. Transition to Init Download. - If an alive message is received with zero priority the current master wants to resign. This firewall is not ready yet so it does not participate in the election of a new master, i.e. the timer is reset to master_down_interval. o If the timer calls this firewall is alone and should become master. First the IP and MAC addresses are set on the interfaces, gratuitous ARPs for the cooperations addresses are sent and an alive message is sent. Transition to master. 5.2 Backup When a firewall is in the backup state it keeps up to date with the master and listens for alive messages from the master. Actions: o It listens on the TCP-connection for synchronization data and updates the local firewall table for each received data. o It listens on the multicast channel for alive messages and resets the timer to master_down_interval for each alive message it receives. o If an alive message is sent from another firewall than the last master, a take over has occurred and this firewall disconnects and connects to the new master. Transition to Init Download. o If an alive message is received and this firewall's priority is higher than the priority of the master, this firewall could become master (if the master resigns with a zero priority alive message there should be an election). The timer is set to skew_interval and the TCP connection with the master is set as the first backup connection if this firewall becomes master. o If the timer calls, this firewall becomes master of the FRP cooperation. First an alive message is sent, IP and MAC addresses are set on the interfaces, gratuitous ARPs for the cooperations addresses are sent and the connection to the master is rearranged to be the first backup connection. If the master connection is not complete (i.e. half closed due to failing master) it is disconnected. Last the timer is set to alive_interval and state is set to master. 5.3 Master The master filters all traffic for the FRP cooperation, sends synchronization data to the backups, waits for new backups to connect and sends alive messages over the multicast channel. Actions: o If the firewall table changes, the firewall tables in the backups need to be updated. It is done by sending a synchronization message to each connected backup. If the firewall participates in several FRP cooperations it has to know which FRP cooperation the change is related to. (The firewall table changes when a new connection is setup through the firewall or when a connection is closed.) o When the timer calls an Alive message is sent on the multicast channel and the timer is reset to alive_interval. o It listens on the multicast channel: - If an alive message is received with lower priority than the own, it is discarded and an own alive message is sent. - If an alive message is received with higher priority there has been a take over and this firewall becomes backup. The FRP cooperation's official IP and MAC addresses are removed, all connections are closed and the timer is set to master_down_interval. The states that no longer are valid should be removed from the firewall table. Transition to backup. o It has a listening socket for new backups. When a new backup connects the firewall makes a transition to Master Download. 5.4 Init Download The initializing firewall has a connection with the master and is downloading a firewall table. Actions: o Receives messages on the TCP connection from the master. - If it is download messages, the information is inserted in to the table. - If it is a synchronization message or an empty download message the download is finished. Insert the data and make transition to backup state. o Waits for alive messages. - If the master has changed or is resigning destroy the table and make a transition to Initialize state. Otherwise just restart the timer with alive_interval. o If the timer calls, the master is down. Destroy the incomplete table and make transition to Initialize state. The timer is restarted with alive_interval (to avoid an election where this machine without a state table takes over instead of a lower priority backup with a complete state table) and waits for another backup to take over to be able to download the firewall table. 5.5 Master Download The master is downloading the firewall table to one or many initializing firewalls. All new state changes from the kernel are queued. When the complete table is downloaded, the queue is sent to the backups and the master makes a transition back to master state. Actions: o Pack the firewall table in to download messages and send them to the (participating) firewalls which are in Init Download state. Finish with an empty download message if there are no queued states. Make transition to Master. o Send alive messages on timeout. o Listen for alive messages on the multicast channel. - If an alive message is received with a higher priority, downloading is stopped and it makes a transition to Master to act according to the same resign procedure. - If the message has lower priority then send an own alive message. 6.0 Take Over By moving IP and MAC addresses from the failing (ex-)master to the new master, other machines on the network will not notice the change and do not need any updates. One network component will need to change and that is a learning bridge, if there is one between the ex-master and the new master. This learning bridge has to realize that after the take over the MAC address of the FRP cooperation has moved to another interface. This is done by sending a gratuitous ARP from the new master as soon as possible, after take over. Take over means that the new master sets the FRP cooperation's official MAC address on each interface concerned with FRP. 6.1 Another Example Another example that explains the function of FRP. Figure 3 shows the example configuration. Net A (outside): 10.0.0.0/24 Common FRP Parameters Table -----+---------------+-------- FRP ID : 7 | | FRP MAC: 00-00-5E-00-02-07 |.0.1 |.0.2 FRP net: Net B +----+----+ +----+----+ | MACA | | MACB | FRP IP on Net A : 10.0.0.36 | | | | FRP IP on Net B : 10.0.1.36 | FW 1 | | FW 2 | FRP IP on Net C : 10.0.2.36 | | | | FRP priority FW1: 100 |MACC MACD| |MACE MACF| FRP priority FW2: 255 +-+----+--+ +-+----+--+ .1.1| |.2.1 .2.2| |.1.2 | | | | --+----)----------)----+-- Net B (market): 10.0.1.0/24 | | ----+----------+-- Net C (production): 10.0.2.0/24 Figure 3. Example configuration. Two firewalls use FRP with the common parameters as shown in the table. This constellation has two inside networks, Net B connected to the market department and Net C to the production department. All machines on the market dep. have their default route set to be the FRP cooperation (IP 10.0.1.36) and in their ARP tables there is an entry: "IP 10.0.1.36 is found at 00-00-5E-00-02-07". Machines on Net A and Net C have, respectively, ARP entries like "10.0.0.36 found at 00-00-5E-00-02-07" and "10.0.2.36 found at 00-00-5E-00-02-07". This leads to that outward going user traffic is on Net C sent to IP 10.0.2.36 on FRP MAC 00-00-5E-00-02-07. During normal operation FW2, which has the highest priority of the two cooperating firewalls, will be master. This means that FW2 answers ARP requests for FRP IP on all networks (notice: one IP for each network) with FRP MAC (the same on all networks), filters traffic that arrives on FRP MAC, sends and listens for alive messages multicasted over Net B, listens on the multicast address on Net B and sends synchronization data over an TCP connection on Net B from FW2 to FW1. It is always possible to reach FW1 on IP .1, which is not changed by FRP. If FW2 would crash, FW1 will notice and become master. FW1 takes over by sending an alive message on the multicast channel on Net B, then it starts to listen for backup TCP connections on Net B and adding FRP MAC on all interfaces. This includes sending a gratuitous ARP on all interfaces. FW1 also starts to respond to ARP requests for FRP IP and of course filter traffic arriving at FRP MAC. 7.0 Synchronization Message Format Synchronization data is sent as messages over the TCP connection between a master and a backup firewall. It uses the private IP addresses of the master and backup and port 2925. There are three message types: synchronization data add, synchronization data delete and firewall table download. The complete set of parameters for a connection through the firewall is called a state. When a connection is setup through the firewall all information about it is collected into a state, which is inserted in to the firewall table. The state is then sent as a synchronization data message to the backups. Adding a state to the firewall table is equivalent to making a hole through the firewall for a certain connection and deleting the state closes the hole. Figure 4 describes the message format. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | Data | | . | | . | | . | +---------------------------------------------------------------+ Figure 4. The synchronization message header and data field (sent over a TCP connection). Type: 1 Synchronization data Add. This message is sent from the master to the backup with information about a new connection that has been setup through the firewall. 2 Synchronization data Delete. This message is sent from the master to the backup with information about a disconnection. 3 Download. This message is sent from the master to the backup and contains a part of the firewall table. Length: The length of the data in bytes. With types 1,2 and 3 the length divided by the length of a state (28 bytes) gives the number of states contained in the message. A zero length download message indicates that the download is complete. Data: Depends on the type. A packet of type 1 or 2 can contain one or many states of 28 bytes. The state format is described in figure 5. The next figure shows a synchronization message transporting one state but it is possible to send several states in one message (to reduce overhead). 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Type | Length | +---------------+---------------+-------------------------------+ | Source IP address | +---------------------------------------------------------------+ | Destination IP address | +---------------------------------------------------------------+ | Redirect IP address | +---------------------------------------------------------------+ | NAT IP address | +-------------------------------+-------------------------------+ | Source port | Destination port | +-------------------------------+-------------------------------+ | Redirect port | NAT port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---------------+---------------+ | Flags | Protocol | Direction | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---------------+---------------+ Figure 5. The synchronization message with one state in the data field. Address and port fields: These are the addresses and ports that defines the connection through the firewall. Flags: State status flags. Implementation dependent. Protocol: The protocol number for traffic that is going through the firewall. Direction: Implementation dependent. 8.0 Control Message Format Control messages are sent as UDP packets over the multicast channel on IP address 224.0.0.33 and port 2925. There is one packet type and it is the alive message. The alive message is used for failure detection and to distribute the IP address for synchronization data TCP connection requests. When discovering a new master the backup will connect to the IP address and port to download the current firewall table and then get continuous synchronization data. Figure 6 describes the message format. 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Version | Type | Id | Priority | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | IP address | +---------------------------------------------------------------+ Figure 6. The multicast message format (inside an UDP packet) Version: Version number is 2. Type: Only one type is specified. Type 1 is the alive message. ID: Firewall Cooperation ID. Priority: This is the priority of the sender or zero if the sender is the current master and wants to resign. IP address: This is the IP address on which the current master wishes to get requests for TCP connections, i.e. the private address. It also makes it possible for the backups to realize when a take over has occured and handle priority conflicts. The control and synchronization traffic could be on a separate net to avoid disturbing the ordinary traffic. 9.0 Security Considerations The security risks occur during and after a take over. There will be a gap before the new master takes over after the last master have failed. If any connection setups are done during this time, they will not be recognized, i.e. never reach their destination. Then the end application will have to try again until it works. This is not a security problem. If any disconnects are done during take over time, they will not be recognized and the holes can remain. These will remain until they times out and will also be synchronized to backups. This is a security problem. Ordinary packets that get lost during the take over will be handled by the flow control of the applications and not affect the users with other than delay, which is unavoidable when something has crashed. During download the new states are stored in a queue, which means that if the master fails or another firewall takes over before the download is finished a number of states will be lost. This leads to the same consequences as above. All control and synchronization traffic should be sent on a separate network, that none other than the FRP daemon can send and receive traffic on. With access to that network it is very easy to attack the firewalls. One way to handle this is to insert authentication on control traffic, but it is not included in this version of FRP. Another reason to use a special net is to eliminate the risk of disturbances from ordinary traffic on the control traffic. Heavy traffic implies high packet losses which could cause a take over or worse multiple take overs and loss of information. 10.0 References [IANA Eth.no] Internet Assigned Numbers Authority, IANA. ``Protocol Numbers and Assignment''; Online living document available from http://www.iana.org/numbers.html [VRRP] Knight S. et. al. 1998. ``RFC 2338, Virtual Router Redundancy Protocol''; available from http://www.ietf.org/rfc/rfc2338.txt 11.0 Acknowledgements A lot of the failure detection and take-over mechanisms have been build upon work in VRRP (RFC2338). Thanks to Tomas Jyde and other colleagues at Effnet who have contributed with discussions and comments. 12.0 Author's Address Andreas Papp, Effnet AB, frp@effnet.com Box 150 40, 167 15 Bromma, Sweden