| Network Working Group | J. Fu |
| Internet-Draft | X. Zhu |
| Intended status: Informational | M. Ramalho |
| Expires: January 5, 2016 | W. Tan |
| Cisco Systems | |
| July 4, 2015 |
Evaluation Test Cases for Interactive Real-Time Media over Wi-Fi Networks
draft-fu-rmcat-wifi-test-case-00
Wireless network is becoming the dominiant mode of Internet access nowadays. According to statisics gathered by Cisco regarding usage of interactive real-time interactive media (e.g., video conferencing calls) across different types of clients, 76.12% of the calls are over Wi-Fi networks, 23.84% are over cellular, and only 0.04% are over Ethernet. [Editor's note: need some clarification/reference regarding source of this data]. It is therefore important to evaluate candidate congestion control schemes designed for real-time interactive media over test cases that include Wi-Fi access links. This draft serves such a purpose, and is complementary to [I-D.ietf-rmcat-eval-test] and [I-D.draft-sarker-rmcat-cellular-eval-test-cases]
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 5, 2016.
Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Given the prevalence of Internet acess links over Wi-Fi, it is important to evaluate candidate RMCAT congestion control solutions over Wi-Fi test cases. Such evaluations should also highlight the inherent different characteristics of Wi-Fi networks in contrast to Wired networks:
As we can see here that Wi-Fi network has different senarios which has impact on the network performance, and affects packet delay and packet loss which real-time multimedia congestion control rely on.
Since Wi-Fi network normally connects to a wired infrastructure, either the wired network or the Wi-Fi network could be the bottleneck. In the following section, we describe basic test cases for these two scenarios separately.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described RFC2119 [RFC2119].
The test scenarios below are intended to mimic the set up of video conferencing over Wi-Fi connections from the home. Typically, the Wi-Fi home network is not congested and the bottleneck is present over the wired home access link. Although it is expected that test evaluation results from this section are similar to those from the all-wired test cases (draft-sarker-rmcat-eval-test), it is worthwhile to run through these tests as sanity checks.
This test case is designed to measure the responsiveness of the candidate algorithm when the Wi-Fi hop of the connection is uncongested.
Figure 1 illustrates topology of this test.
uplink
+-------------->+
+----+ +----+ +----+ +----+
| S |))))))))))| AP |==========| B |=========| R |
+----+ +----+ +----+ +----+
Figure 1: Single Flow over Wired Bottleneck
Testbed attributes:
Expected behavior: the candidate algorithm is expected to detect the path capacity constraint, converges to bottleneck link's capacity and adapt the flow to avoid unwanted oscillation when the sending bit rate is approaching the bottleneck link's capacity. Oscillations occur when the media flow(s) attempts to reach its maximum bit rate, overshoots the usage of the available bottleneck capacity, to rectify it reduces the bit rate and starts to ramp up again.
This test case is designed to evaluate performance of the candidate algorithm when lack of enough feedback.
+----+ +----+
| R1 |)))))) /=====| S1 |
+----+ )) uplink // +----+
)) +-------------->+ //
+----+ +----+ +----+ +----+
| S2 |))))))))))| AP |==========| B |========| R2 |
+----+ +----+ +----+ +----+
+<--------------+
downlink
Figure 2: Two Bidirectional Flows over Wired Bottleneck
Testbed attributes:
Expected behavior: It is expected that the candidate algorithms is able to cope with the lack/noise of feedback information and adapt to minimize the performance degradation of media flows in the forward channel.
This test case is designed to measure the performance of the candidate algorithm when lack of enough feedback.
+----+ +----+
| S1 |)))))) /=====| R1 |
+----+ )) uplink // +----+
)) +-------------->+ //
+-------+ +----+ +----+ +-------+
| S_tcp |))))))))))| AP |==========| B |========| R_tcp |
+-------+ +----+ +----+ +-------+
+<--------------+
downlink
Figure 3: RMCAT vs. TCP over Wired Bottleneck
Testbed attributes:
Expected behavior: the candidate algorithm should be able to avoid congestion collapse, and get fair share of the bandwidth. In the worst case, the media stream will fall to the minimum media bit rate.
These test cases assume that the wired portion along the media path are well-provisioned. The bottleneck is in the Wi-Fi network over wireless. This is to mimic the enterprise/coffee-house scenarios. Currently the widely used IEEE 802.11 standards are 802.11g and 802.11n, 802.11ac. IEEE 802.11b is legency standard, and 802.11a is not widely adopted.
Since morden IEEE 802.11 standards supports far higher data rates than the maximum requirements of individual RMCAT flows, in this test the legacy standard 802.11b is chosen to test the single RMCAT flow case. 802.11b Adaptive rate selection can operate at 11 Mbps in terms of PHY-layer transmission rate, and falls back to 5.5 Mbps, 2 Mbps, and 1 Mbps when the wireless client moves away from the access point.
[Editor's Note: we may want to move this section to Section 5.4 instead.]
uplink
+-------------->+
+----+ +----+ +----+ +----+
| S |))))))))))| AP |==========| B |=========| R |
+----+ +----+ +----+ +----+
Figure 4: One RMCAT Flow over Wireless Bottleneck
Testbed attributes:
+---------------------+----------------+------------+----------------+ | Variation pattern | Path direction | Start time | PHY-layer rate | | index | | | | +---------------------+----------------+------------+----------------+ | One | Forward | 0s | 5.5 Mbps | | Two | Forward | 40s | 2 Mbps | | Three | Forward | 60s | 1 Mbps | | Four | Forward | 80s | 2 Mbps | +---------------------+----------------+------------+---------------+
Figure 5: Adaptive rate variation pattern for uplink direction
Expected behavior: The rate adaptation algorithm run at application level should follow the adaptation in 802.11 mac layer.
This test case is for studying the impact of contention on competing RMCAT flows. Specifications for IEEE 802.11g with a physical-layer transmission rate of 54 Mbps is chosen. The total application-layer throughput (reasonable distance, low interference and small number of contention stations) for 802.11g is around 20 Mbps. Consequently, a total of 16 RMCAT flows are needed for saturating the wireless interface in this experiment.
uplink
+----------------->+
+----+ +----+
| S1 |)))))) /=====| R1 |
+----+ )) // +----+
)) //
+----+ +----+ +----+ +----+
| S2 |))))))))))| AP |==========| B |========| R2 |
+----+ +----+ +----+ +----+
...... ......
)) \\
+----+ )) \\ +----+
|S16 |)))))) \=====|R16 |
+----+ +----+
+<-----------------+
downlink
Figure 6: Multiple RMCAT Flows Sharing the Wireless Uplink
Testbed attributes:
Expected behavior: All RMCAT flow should get fair share of the bandwidth, and the overall bandwidth usage should no less than same case with TCP flows (use TCP as performance benchmark). The delay and loss should be in acceptable range for real-time multimedia flow (might need rtp circuit breaker to guarantee that?).
This test case is different with the uplink one in the previous section because here, most of the time the access point (AP) is transmission, whereas individual stations only need to send feedback messages. As a result, this test case will incur less contention than the uplink case.
uplink
+----------------->+
+----+ +----+
| R1 |)))))) /=====| S1 |
+----+ )) // +----+
)) //
+----+ +----+ +----+ +----+
| R2 |))))))))))| AP |==========| B |========| S2 |
+----+ +----+ +----+ +----+
...... ......
)) \\
+----+ )) \\ +----+
|R16 |)))))) \=====|S16 |
+----+ +----+
+<-----------------+
downlink
Figure 7: Multiple RMCAT Flows Sharing the Wireless Downlink
Testbed attributes:
Expected behavior: All RMCAT flow should get fair share of the bandwidth. Overall bandwidth usage should be no less than same case with TCP flows (using TCP as performance benchmark). The delay and loss should be within acceptable range for real-time multimedia flow.
This one differs with previous contention cases because Wi-Fi share bandwdith between uplink and downlink.
uplink
+----------------->+
+----+ +----+
| R1 |)))))) /=====| S1 |
+----+ )) // +----+
)) //
...... ......
+----+ +----+ +----+ +----+
| R8 |))))))))))| AP |==========| B |========| S8 |
+----+ +----+ +----+ +----+
...... ......
)) \\
+----+ )) \\ +----+
|S16 |)))))) \=====|R16 |
+----+ +----+
+<-----------------+
downlink
Figure 8: Multiple Bi-Directional RMCAT Flows Sharing the Wireless Bottleneck
Testbed attributes:
Expected behavior: All (uplink/downlink) RMCAT flow should get fair share of the bandwidth, and the overall bandwidth usage should no less than same case with TCP flows (use TCP as performance benchmark). The delay and loss should be in acceptable range for real-time multimedia flow (might need rtp circuit breaker to guarantee that?).
This case having both long lived TCP and RMCAT sharing the uplink at the same time. This is for testing how RMCAT competing with long lived TCP flow in a congested Wi-Fi network.
uplink
+----------------->+
+----+ +----+
| S1 |)))))) /=====| R1 |
+----+ )) // +----+
)) //
...... ......
+----+ +----+ +----+ +----+
| S8 |))))))))))| |==========| |========| R8 |
+----+ | | | | +----+
| AP | | B |
+--------+ | | | | +--------+
| S1_tcp |))))))| | | |========| R1_tcp |
+--------+ +----+ +----+ +--------+
...... ......
)) \\
+--------+ )) \\ +--------+
| S8_tcp |)))) \=====| R8_tcp |
+--------+ +--------+
+<-----------------+
downlink
Figure 9: Multiple RMCAT and TCP Flows Sharing the Wireless Uplink
Testbed attributes:
Expected behavior: All RMCAT flows should get comparable share of the network bandwidth with respect to competing TCP flows. The overall bandwidth usage should no less than same case with TCP flows (use TCP as performance benchmark). The delay and loss should be in acceptable range for real-time multimedia flow (might need rtp circuit breaker to guarantee that?).
This case having both long lived TCP and RMCAT on the downlink at the same time. This is for testing how RMCAT competing with long lived TCP flow in crowed Wi-Fi network. This differs from test scenario in the previous section becauase less contention on the Wi-Fi network because most media data is sent from AP to stations.
uplink
+----------------->+
+----+ +----+
| R1 |)))))) /=====| S1 |
+----+ )) // +----+
)) //
...... ......
+----+ +----+ +----+ +----+
| R8 |))))))))))| |==========| |========| S8 |
+----+ | | | | +----+
| AP | | B |
+--------+ | | | | +--------+
| R1_tcp |))))))| | | |========| S1_tcp |
+--------+ +----+ +----+ +--------+
...... ......
)) \\
+--------+ )) \\ +--------+
| R8_tcp |)))) \=====| S8_tcp |
+--------+ +--------+
+<-----------------+
downlink
Figure 10: Multiple RMCAT and TCP Flows Sharing the Wireless Downlink
Testbed attributes:
Expected behavior: All RMCAT flows should get comparable share of the network bandwidth with respect to competing TCP flows. The overall bandwidth usage should no less than same case with TCP flows (use TCP as performance benchmark). The delay and loss should be in acceptable range for real-time multimedia flow (might need rtp circuit breaker to guarantee that?).
This case having both long lived TCP and RMCAT on the both direction at the same time. This is for testing how RMCAT competing with long lived TCP flow in a congested Wi-Fi network. This differs from previouss cases as both uplink and downlink flows share the same wireless bottleneck.
uplink
+----------------->+
+----+ +----+
| S1 |)))))) /=====| R1 |
+----+ )) // +----+
)) //
...... ......
+----+ +----+ +----+ +----+
| R8 |))))))))))| |==========| |========| S8 |
+----+ | | | | +----+
| AP | | B |
+--------+ | | | | +--------+
| S1_tcp |))))))| | | |========| R1_tcp |
+--------+ +----+ +----+ +--------+
...... ......
)) \\
+--------+ )) \\ +--------+
| R8_tcp |)))) \=====| S8_tcp |
+--------+ +--------+
+<-----------------+
downlink
Figure 11: Multiple Bi-Directional RMCAT and TCP Flows Sharing the Wireless Bottleneck
Testbed attributes:
Expected behavior: All RMCAT flows should get comparable share of the network bandwidth with respect to competing TCP flows. The overall bandwidth usage should no less than same case with TCP flows (use TCP as performance benchmark). The delay and loss should be in acceptable range for real-time multimedia flow (might need rtp circuit breaker to guarantee that?).
Wi-Fi roaming need scanning, authentication and re-association which will cause packet drops and delay, and interrupt the network connection. RMCAT congestion control algorithms should at least recover (if affected by the transition process) after roaming.
The phone can switch automatically between Wi-Fi and Cellular network when the other is not available, and some phones like "Samsung Galaxy" have smart network switch to switching to network has better connectivity automatically. Unlike Wi-Fi Roaming, such kind of switch might or might not interrupt the network connection, and might change the route. RMCAT congestion control should be able to cope with the changes.
EDCA/WMM is prioritized QoS with four traffic classes (or Access Categories) with differing priorities, RMCAT flow should have better performance (lower delay, less loss) when EDCA/WMM enabled.
When there is 802.11b devices connected to modern 802.11 network, it may affect the performance of the whole network. Additional test cases can be added to evaluate the affects of legancy devices on the performance of RMCAT congestion control algorithm.
There are no IANA impacts in this memo.
| [RFC2119] | Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. |
| [I-D.ietf-avtcore-rtp-circuit-breakers] | Perkins, C. and V. Singh, "Multimedia Congestion Control: Circuit Breakers for Unicast RTP Sessions", Internet-Draft draft-ietf-avtcore-rtp-circuit-breakers-10, March 2015. |
| [I-D.ietf-rmcat-eval-criteria] | Singh, V. and J. Ott, "Evaluating Congestion Control for Interactive Real-time Media", Internet-Draft draft-ietf-rmcat-eval-criteria-03, March 2015. |
| [I-D.ietf-rmcat-eval-test] | Sarker, Z., Singh, V., Zhu, X. and M. Ramalho, "Test Cases for Evaluating RMCAT Proposals", Internet-Draft draft-ietf-rmcat-eval-test-01, March 2015. |
| [I-D.ietf-rmcat-cc-requirements] | Jesup, R. and Z. Sarker, "Congestion Control Requirements for Interactive Real-Time Media", Internet-Draft draft-ietf-rmcat-cc-requirements-09, December 2014. |
| [I-D.draft-sarker-rmcat-cellular-eval-test-cases] | Sarker, Z., "Evaluation Test Cases for Interactive Real-Time Media over Cellular Networks" |
| [I-D.ietf-rtcweb-use-cases-and-requirements] | Holmberg, C., Hakansson, S. and G. Eriksson, "Web Real-Time Communication Use-cases and Requirements", Internet-Draft draft-ietf-rtcweb-use-cases-and-requirements-16, January 2015. |