Internet DRAFT - draft-ooki-lmap-internet-measurement-system
draft-ooki-lmap-internet-measurement-system
Internet Engineering Task Force M. Ooki
Internet-Draft S. Kamei
Intended status: Informational NTT Communications
Expires: June 23, 2016 December 21, 2015
Internet Measurement System
draft-ooki-lmap-internet-measurement-system-03
Abstract
This document describes an experience of Japanese Internet
measurement system to measure end-to-end performance of user's
experience. We have developed the system toward the enhancement of
the network performance in an ISP since October 2013. The systems
and the considerations about the Internet measurement are introduced
along with our current status. This document is expected to be
useful for the standardization of Internet measurements.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 23, 2016.
Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
Ooki & Kamei Expires June 23, 2016 [Page 1]
Internet-Draft Internet Measurement System December 2015
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Motivation of Internet Measurement . . . . . . . . . . . . . 3
3. Framework of Internet Measurement System . . . . . . . . . . 4
3.1. Measurement Agent . . . . . . . . . . . . . . . . . . . . 4
3.1.1. Specification of the MA . . . . . . . . . . . . . . . 5
3.1.2. Configuration . . . . . . . . . . . . . . . . . . . . 5
3.1.3. Location of the MA . . . . . . . . . . . . . . . . . 6
3.2. Controller Server . . . . . . . . . . . . . . . . . . . . 6
3.2.1. Control of MAs . . . . . . . . . . . . . . . . . . . 6
3.2.2. Control of the Assigned ISP . . . . . . . . . . . . . 7
3.2.3. Setting the Measurement Task and Measurement Schedule 8
3.2.4. Receiving the Requests . . . . . . . . . . . . . . . 8
3.3. Collector Server . . . . . . . . . . . . . . . . . . . . 9
3.4. Architecture . . . . . . . . . . . . . . . . . . . . . . 9
4. Operation of Internet Measurement System . . . . . . . . . . 11
4.1. Measurement Performance Metrics . . . . . . . . . . . . . 11
4.2. Measurement Target Contents . . . . . . . . . . . . . . . 12
4.3. Measurement Schedule . . . . . . . . . . . . . . . . . . 13
4.4. Applications of Measurement . . . . . . . . . . . . . . . 13
5. Issues of Internet Measurement System . . . . . . . . . . . . 13
5.1. Architecture Issue . . . . . . . . . . . . . . . . . . . 14
5.2. Operation Issue . . . . . . . . . . . . . . . . . . . . . 14
5.3. Security Issue . . . . . . . . . . . . . . . . . . . . . 15
6. Security Considerations . . . . . . . . . . . . . . . . . . . 16
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 16
8.1. Normative References . . . . . . . . . . . . . . . . . . 16
8.2. URL References . . . . . . . . . . . . . . . . . . . . . 16
1. Introduction
In Japan, it is common to use a high speed Internet such as 100Mbps
and 1Gbps as an ISP's customer connection. Users only know the
maximum bandwidth of the last one mile for the ISP connection. The
maximum bandwidth value is ranging from 100Mbps to 2Gbps in ISP's
price plan as a FTTH connection.
Of course the end-to-end performance of actual Internet connection is
below the bandwidth value. Internet users can obtain actual
performance depends on various ISP conditions such as congestions.
Internet users don't know the performance of the actual network.
Ooki & Kamei Expires June 23, 2016 [Page 2]
Internet-Draft Internet Measurement System December 2015
On the other hand, ISPs also don't know the quality that Internet
users experience. For the ISP's point of view, it is important to
understand the service quality for its customers in order to design
its network properly. For this reason, it is necessary to measure
the actual performance of typical Internet users.
The Large-Scale Measurement of Broadband Performance (LMAP) working
group is formed to standardize a large scale measurement system to
measure broadband network performance. The current LMAP WG focus on
the information model, data model language, the protocols in a
certain ISP network. However, the LMAP WG does not focus on the
measurement of the global end-to-end performance at the moment. We
believe that either way someday it will be necessary to establish a
method for the Internet measurement and the standardization of the
end-to-end performance measurement, that is not closed to a certain
ISP.
This document describes the Internet measurement system and our
considerations for the end-to-end measurement. Our measurement
requirements can be useful for LMAP framework. We have measured the
end-to-end performance by using Internet measurement system we have
been operating since 2011. We expect the experience of our case can
contribute to the standardizations in LMAP WG and the enhancement of
network operation from ISP's perspective.
2. Motivation of Internet Measurement
The LMAP WG describes some use cases for the Large-scale Measurement
of Broadband Performance [RFC7536][I-D.deng-lmap-collaboration].
There are three reasons that we, ISPs, need to measure the end-to-end
performance of user experience of its access services.
First, ISPs want to keep the customer satisfaction. Typically ISPs
provide the list of maximum bandwidth and the service prices, such as
the estimated total fee and the discount rate after the result of a
cash back campaign. Japanese users select a ISP based on only those
information without knowing the end-to-end performance results. The
poor performance causes the lower customer satisfaction.
On the other hand, the Ministry of Internal Affairs and
Communications in Japan discuss the ideal measurement methods of the
end-to-end performance about the mobile network in the research
society. The organization has yet to be discussed fixed network.
The researchers are planning to measure the mobile network
performance on the 1,500 measurement points in the main Japanese
area. They use the tool Federal Communications Commission developed
for measurement of the end-to-end performance. The mobile network
Ooki & Kamei Expires June 23, 2016 [Page 3]
Internet-Draft Internet Measurement System December 2015
operator in Japan may be required to publish the actual performance
in addition to the best effort performance.
Second, contents providers are beyond the control of ISPs. The
traffic volume of Contents Delivery Network (CDN) providers such as
AKAMAI and LEVEL3 is increasing in the Internet in recent years. How
much users are connecting to which contents providers impact the end-
to-end performance. ISPs need to understand their behavior to decide
ISP's strategies and operation.
Third, we would like to support the public evaluations of ISPs. Some
contents service provider e.g., Google or Sandvine[google][sandvine],
presented the reports about Internet traffic and ISP performance
based on each criterion recently. The Google report presents the
results of multiple ISPs measured in for each locations in USA.
People in the world can browse the reports on the Internet. These
reports will have huge impact on user's choice of the ISP selection.
We would like to double-check by using our performance data in order
to confirm whether the reports can be reliable or not. If we can
find the difference of the performance results between our data and
the reports, we might be able to review whether our measurement
methods are mistaken or not. It is also better for an ISP to
investigate and comprehend the status of end-to-end performance
between ISPs. So, we have to measure the end-to-end performance by
ourself.
Hence, ISPs should measure the end-to-end performances from end users
to multiple content providers accurately while comparing with other
ISPs' performance. Then, ISPs can show a performance of the actual
network to build brand value compared with other providers.
3. Framework of Internet Measurement System
We introduce the framework of Internet Measurement System in this
section. The words, such as Measurement Agent, Controller, and
Collector conforms to the glossary of the LMAP document[I-D.ietf-
lmap-framework].
3.1. Measurement Agent
The MA has the functions that receive instructions from Controller
Server (described below), performs measurement tasks, and sends the
measurement results to the Collector Server.
Ooki & Kamei Expires June 23, 2016 [Page 4]
Internet-Draft Internet Measurement System December 2015
3.1.1. Specification of the MA
We used a Japanese product, called OpenBlocks [plathome], which is
the Linux box with Dual Core Marvell ARMADA XP 1.33GHz, 1GB SDRAM
memory. We selected the box as the MA because of the affordable
price, software stability, small form factor, flexible functionally,
and extendability. The MA needs some CPU power in order to connect
PPPoE access line and download tens of contents on the Internet. The
OpenBlocks stacks CPU enough to archive them.
3.1.2. Configuration
We introduce information configured on the MA in this section.
o MA's ID
We have to setup the MA's ID. The ID has to be a unique among MAs in
order for Controller server to distinguish MAs. The information is
described in the "/etc/hostname" on Linux File System. The naming
rule is based on the location of MA, the types of line, and the plan,
etc.
On the other hand, MA doesn't have a group ID. The ID in our
measurement system is under only Controller Server.
o HTTP Get Tasks
MA automatically gets the measurement tasks from Controller Server
every five minutes. MA sends the request about the tasks and
schedule to Controller Server by HTTP. Controller Server returns the
tasks decided based on SCHEDULE Table to the MA.
o Convert measurement results
MA automatically converts from raw data to the type of JSON data.
JSON is a lightweight data-interchange format. We selected it as our
measurement data model language because of readability, simple
format, and easy data cleaning to analyze the measurement data. On
the other hand, the LMAP WG selected YANG Data Model [I-D.ietf-lmap-
yang]. We need to consider which we should use YANG data model or
not. The convert process is executed every one minute. The
converted JSON data is written in the measurement result files. An
example of JSON data type of the results of wget raw data is as
follows.
{"host":"tokyo-xXx01","filename":"tokyo-xXx01_ISP_target_wget_2014120
3235011.log.ok","result":"ok","line":"2","message":"2014-12-03
23:50:18 (10.1 MB/s) - `/dev/null' saved [67206439/67206439]"}
Ooki & Kamei Expires June 23, 2016 [Page 5]
Internet-Draft Internet Measurement System December 2015
o Data Collector and Submitting
MA automatically and efficiently collects and submits the measurement
data. To make it realized, we selected the fluentd which is an open
source data collector, The software lets Collector Server unify the
data collection and understanding of data. The reason we selected it
is that the software is reliable, stable, and simple of
implementation enough to control hundreds of MAs. The data
collecting tool of many products and system in Japan is implemented
by fluentd. The above measurement result files are submitted to
Collector Server by the process as soon as the files are created. An
example of the running flutend process is as follows.
/usr/bin/ruby1.9.1 /usr/local/bin/fluentd --daemon /var/lib/fluent/
fluentd.pid --user fluent --group fluent --config /etc/fluent/
fluent.conf --log /var/log/fluent/fluent.log -vv
o Self Check
The MAs check whether the above processes for measurement service are
down or not on the regular interval. If a process is down, the MA
transmits the message about the message logs to Controller Server.
3.1.3. Location of the MA
We have distributed MAs on many places all over Japan. The number of
locations is approximately 150 in June 2015. The number of our MAs
will be increasing in approximately 200 by the end of this year. MAs
are located in houses where the residents can respond our requests
(e.g., not turning off the power to constantly perform the
measurement) to manipulate the device.
3.2. Controller Server
The Controller Server is a Linux server. The Controller Server has
the functions to instruct the MAs and receives the HTTP GET requests
from MAs. The Controller Server has 3 tables of database implemented
by MySQL to instruct MAs.
3.2.1. Control of MAs
The Controller Server manage MAs by using two tables (MA Table and
GROUP Table). A example of the MA Table is as follows. MA_ID is a
key identifying MA. TYPE is a kind of network type. MODE expresses
the type of the measurement.IF MODE is 0, it means MA is the
measurement mode. The mode is the status that MAs are performing a
measurement task. If MODE is 1, it means MA is the maintenance mode.
The mode is the status that MA stop performing a measurement task.
Ooki & Kamei Expires June 23, 2016 [Page 6]
Internet-Draft Internet Measurement System December 2015
In case of MODE 1, the MA automatically connect to Controller Server
by using ssh protocol. We can login to the MA of MODE 1 from
Controller Server and change the configuration. We can manage the
behavior for MAs by switching the MODE.
+-------------+-------+-------+---------+------+--------------------+
| MA_ID | TYPE | AREA | OS_TYPE | MODE | GET_SCHEDULE_TIME |
+-------------+-------+-------+---------+------+--------------------+
| tokyo-nFh04 | flets | tokyo | Debian | 1 | 2014-12-08 |
| | | | 7 | | 23:21:00 |
| osaka-nFs01 | flets | osaka | Debian | 1 | 2014-12-08 |
| | | | 7 | | 23:22:00 |
+-------------+-------+-------+---------+------+--------------------+
Table 1: MA Table
A example of the GROUP Table is as follows. The GROUP_ID is a key
record grouping MAs. MAs are sure to belong one group at least. MA,
tokyo-nFh04, belongs to the group-id1. The GROUP_INFO is the
remarks. We can set the information of the group which MAs belong in
the column
+-----------+-------------+------------+
| GROUP_ID | MA_ID | GROUP_INFO |
+-----------+-------------+------------+
| group-id1 | tokyo-nFh04 | Group 01 |
| group-id2 | osaka-nFs01 | Group 02 |
+-----------+-------------+------------+
Table 2: GROUP Table
3.2.2. Control of the Assigned ISP
We set the information of ISP accounts to assign MAs on the
Controller Server. MAs automatically download the information from
Controller Server. Of course, assigned ISP account is unique of all
ISP accounts for the measurements. The table includes the column of
multiple assigned information so that the duplicate use does not
happen. A example of the ISP Table is as follows. ASSIGN_ID is a
key record within the table. MEASURE_ISP is the service name of ISP.
ISP_ID is the unique ID to connect to the ISP network. PASSWORD is
the password of the ISP_ID. ASSIGN_STATUS, ASSING_MA, and
ASSIGN_TIME are the assigned information at that time. If the
ASSIGN_STATUS is 1, that means a MA use the ISP_ID.
Ooki & Kamei Expires June 23, 2016 [Page 7]
Internet-Draft Internet Measurement System December 2015
+-------+--------+-----------+-------+----------+---------+---------+
| ASSIG | MEASUR | ISP_ID | PASSW | ASSIGN_S | ASSIGN_ | ASSIGN_ |
| N_ID | E_ISP | | ORD | TATUS | MA | TIME |
+-------+--------+-----------+-------+----------+---------+---------+
| 1 | OCN | abc123@oc | abc12 | 1 | tokyo- | 2014-12 |
| | | n.ne.jp | 3def | | nFh04 | -08 23: |
| | | | | | | 21:05 |
| 2 | OCN | ghi456@oc | ghi45 | 0 | | |
| | | n.ne.jp | 6jkl | | | |
+-------+--------+-----------+-------+----------+---------+---------+
Table 3: ISP Table
3.2.3. Setting the Measurement Task and Measurement Schedule
We set the measurement tasks to instruct MAs on the Controller
Server. MAs automatically download the task from Controller Server
by the fixed time. We need to set a measurement schedule with the
measurement task at the same time. A example of the MEASUREMENT
SCHEDULE Table is as follows. SCH_ID is a key record within the
table. LINE_TYPE is a type of network provided by a network service.
ISP is a Internet service provider to perform the measurement tasks.
SCRIPT is the script file of the measurement tasks described by some
programming languages. PARAM is a parameter file required for
performing measurement tasks. START_TIME is the time when MAs start
performing a measurement task. END_TIME is the time when MAs stop
performing a measurement task.
+------+--------+--------+----+----------+-------+---------+--------+
| SCH_ | GROUP_ | LINE_T | IS | SCRIPT | PARAM | START_T | END_TI |
| ID | ID | YPE | P | | | IME | ME |
+------+--------+--------+----+----------+-------+---------+--------+
| 1 | group- | flets | IS | measure1 | param | 00:00:0 | 00:00: |
| | id1 | | P1 | .sh | 1 | 0 | 00 |
| 2 | group- | flets | IS | measure2 | param | 00:00:0 | 00:00: |
| | id2 | | P2 | .sh | 2 | 0 | 00 |
+------+--------+--------+----+----------+-------+---------+--------+
Table 4: MEASUREMENT SCHEDULE Table
3.2.4. Receiving the Requests
On the Controller Server, a httpd program is running as a daemon that
executes continuously in the background to handle requests. The
Controller Server returns the appropriate measurement tasks and
measurement schedules to MAs in response to HTTP GET requests. The
MA which complete own measurement task receives a new measurement
Ooki & Kamei Expires June 23, 2016 [Page 8]
Internet-Draft Internet Measurement System December 2015
task continuously. MAs can start performing the next measurement
tasks continuously.
3.3. Collector Server
The Collector Server receives the measurement results from MAs
through fluentd process. The fluentd process is running as a daemon
that executes continuously in the background to handle the
measurements data. The details of the measurement results received
by fluentd process are listed below.
20141214230628+0900 measure.tokyo-nFh04 {"host":"tokyo-
nFh04","filename":"tokyo-nFh04_ISP_DEST_wget_20141214230450.log.ok","
result":"ok","message":"2014-12-14 23:06:18 (745 KB/s) - `/dev/null'
saved [67206439/67206439]"}
20141214230902+0900 measure.tokyo-nFh04 {"host":"tokyo-
nFh04","filename":"tokyo-nFh04_ISP_DEST_wget_20141214230731.log.ok","
result":"ok","message":"2014-12-14 23:08:52 (811 KB/s) - `/dev/null'
saved [67206439/67206439]"}
A example of the directory structure of the stored measurement
results is as follows. A MA's measurement result file is created by
the day.
/data/MA's-ID/measure_result_MA's-ID.DATETIME.log
When the Collector Server receives the measurement results, the
server creates the directory of the MA-ID of MA and the measurement
result files. The measurement results are stored in the data
directory.
3.4. Architecture
The architecture of the measurement system is composed of MAs,
Controller Server, and, Collector Server.
+----------------+ +----------------+
| Controller | | Collector |
| | | |
| Server | | Server |
+----------------+ +----------------+
^ ^
| |
| Get Measurement Sending |
| Tasks and Measurement the Results |
| Schedules |
| +-------------+ |
Ooki & Kamei Expires June 23, 2016 [Page 9]
Internet-Draft Internet Measurement System December 2015
| | Measurement | |
+---------- | | -----------+
| Agents |
+-------------+
| ^
Perform Only | | Obtain
Active Measurement | | Results
v |
+---------------------------------------------------+
| |
| FTTH Access Line |
| |
+---------------------------------------------------+
| ^
| |
| |
v |
+---------------------------------------------------+
| |
| ISP network |
| |
+---------------------------------------------------+
| ^
| |
| |
v |
+---------------------------------------------------+
| |
| Internet |
| |
+---------------------------------------------------+
| ^
| |
| |
v |
+----------------------+
| Target Contents |
+----------------------+
Figure 1: Architecture of the Internet Measurement System
We need to import Controller Server the record of MA's configuration
in MA Table, GROUP Table, and MEASUREMENT SCHEDULE Table before the
MA is powered on.
When a MA is powered on, it tries to establish the FTTH access PPPoE
connection with the ISP. After obtaining an IP address, it
Ooki & Kamei Expires June 23, 2016 [Page 10]
Internet-Draft Internet Measurement System December 2015
automatically connect the Controller Server and gets the
configuration by HTTP. If the value of MODE column in MA Table is 1,
the MA automatically gets the maintenance mode. If the value of that
is 0, the MA automatically gets the measurement mode and start
downloading the measurement tasks that is configured in MEASUREMENT
SCHEDULE Table.
The MA prepares for the measurement tasks, performs the tasks for
Measurement Target Contents actively, and collects the measurement
results.
After the completion of the measurement tasks, the MA sends the
measurement results to Collector Server using fluentd process. and
submits the request for downloading the next measurement tasks to
Controller Server using HTTP.
When the specification of the LMAP WG's protocol and framework is
finished, we will deploy the protocol in our measurement system.
4. Operation of Internet Measurement System
We introduce the operation of Internet Measurement System we have
been operating since October 2013 in this section.
4.1. Measurement Performance Metrics
The MAs perform only active measurements for Target Contents.
Examples of the Target Contents include Video Streaming files, OS
update files, and the test server for performance measurement in a
local ISP network.
In our measurements, the measurement performance metrics are as
follows.
o Round Trip Time (RTT)
This is the response time between the submission of the ICMP echo
request packet and the reception of the ICMP echo reply packet. The
metric can also be regarded as the round-trip delay time. This is
measured by the ping command. We take the min/avg/max time and the
loss rate based on measuring this metrics one hundred times by the
measurement task. An example of the command MAs execute is as
follows.
MA-ID $ ping -i 0.05 -c 100 {contents_ip_address}
o Hop Count and Network Path
Ooki & Kamei Expires June 23, 2016 [Page 11]
Internet-Draft Internet Measurement System December 2015
This metric refers to the number of intermediate devices (like
routers) through which the data must pass between the MA and the
Target Contents. This metric is regarded as the network distance
between the MA and the Target Contents. This is measured by the
traceroute command. Actually, MA submits the ICMP echo requests
three times. Afterwards, by checking the hop counts and network
path, we can find the change of the network routing on the Internet.
An example of the command MAs execute is as follows.
MA-ID $ traceroute -nIq 3 {contents_ip_address}
o Throughput
This metric refers to how much data can be transferred from the MA to
the Target Contents in a given amount of time. This is regarded as
the bandwidth. We can understand how fast we can get the contents
the on the network. Currently, this is measured by the wget command.
A MA receives URL of the measurement targets and start downloading
the contents using HTTP GET. When the download is completed, the
value divided the contents size by the download complete time is
regarded as the performance metrics of throughput. An example of the
command MAs execute is as follows.
MA-ID $ wget -T 300 -dvO /dev/null {wgetopts} {contents_url}
In addition to above three performance metrics, we are studying the
change of destination IP address of the Internet contents distributed
by some contents service providers. It is important for ISPs to know
the mechanism of the contents delivery networks. MAs resolve some
FQDN and gets the destination IP address. We are studying the
mechanism of the contents delivery networks based on the response
results.
4.2. Measurement Target Contents
The selection of the Target Contents is important for the Internet
measurement; the type, the length, and the number of the contents.
We need to measure the representative contents on the Internet. In
order to find such contents, we have selected contents based on two
viewpoints.
One is the volume of transferred data of network traffic in an ISP.
We obtained partial traffic data on multiple prefectures in Japan.
We selected the Target Contents which were higher in the transferred
traffic volume ranking. Examples of such target Contents are Youtube
Video Streaming files and Mac OS update file on AKAMAI and so on.
Ooki & Kamei Expires June 23, 2016 [Page 12]
Internet-Draft Internet Measurement System December 2015
Second is the number of access to the contents in the Internet. For
example, the portal sites such as Google or Yahoo!, etc. and the
shopping sites such as Amazon and iTunes, etc. are always the higher
in the number of access to the contents in the Internet.
In another viewpoint, we need to change the target contents according
to the purpose of analysis. If our purpose is to measure an event
traffic, e.g., the download traffic concerning iOS update or the
access traffic concerning the special winning sale of the
professional baseball team, etc., we need to measure the related
contents.
4.3. Measurement Schedule
On receiving the measurement tasks from Controller Server, MAs start
measurement tasks. MAs used to perform the measurement task by
thirty minutes. When the measurement completes, MAs wait the next
scheduled time and do not perform the next measurement tasks. On the
contrary, when the measurement does not complete before the next
scheduled time, the MA kills the measurement process and moves to the
next measurement. The current system in that point is flexible
because the MA can start the next task as soon as a measurement task
is completed. We can collect more kinds of data than before.
4.4. Applications of Measurement
Using the data collected by our measurement system we have studied
how to comprehend and analyze end-to-end performance more accurately
than ever before. A example of analysis is the difference of network
performance between Japanese ISPs, based on the combinations of
Target Contents, measurement time, and areas. We took the
measurement results in consideration to ISP network design and ISP
operation as a reference information.
Furthermore, we have studied the analysis method based on the
combination of our customer feedbacks, remarks on social network
service, and customer voices on our callcenters in addition to the
collected measurement data. As the result of combined analysis, we
expect to find new and useful knowledge we have never found before.
5. Issues of Internet Measurement System
We introduce the issues of Internet Measurement System we have been
operating in this section. The issue section is divided into three
parts: Architecture Issue, Operation Issue, and Security Issue.
Ooki & Kamei Expires June 23, 2016 [Page 13]
Internet-Draft Internet Measurement System December 2015
5.1. Architecture Issue
o Scalability
The Controller Server is connected to receive HTTP GET requests from
multiple MAs. This means that the Controller Server needs to process
as many HTTP GET requests as the number of MAs. The number of MAs
can easily grow beyond the number of HTTP GET requests that a
Controller Server can process. If we place hundreds of MAs all over
Japan, we will need to improve the scalability of our system.
o IPv6 Support
IPv6 network is constructed totally independently from IPv4 network.
Hence, the performance of the IPv6 network is highly likely different
from that of the IPv4 network.
Although the IPv6 network is not the majority yet, it is growing.
NTT EAST and WEST provided only 2.7% in NGN (Next Generation Network)
on December 2013. The rate of IPv6 enabled network in Japan is 27%
in June 2014[IPv6-Promotion Council]. NTT EAST and WEST presented
the IPv6 support in PPPoE connection on March 2014. All CPE devices
for NTT access line already support IPv6 tunneling, allowing users to
adapt IPv6 easily.
In order to achieve the broad applicability of our measurement
results, we will need to investigate the IPv6 performance also.
o Data Reliability
We need many kinds of data in order to improve the reliability of
data analysis. If there are many kinds of data, the reliability of
our analysis results will be improved and the analysis results might
be statistically significant. We need to develop the architecture to
collect as many different types data as possible. When a MA
completed all the instructed measurement tasks, we are creating the
measurement system that the MA performs other measurement tasks being
high priority as soon as possible when a MA completed all the
instructed measurement tasks.
5.2. Operation Issue
o Selection of the Products as Measurement Agent
We used a Japanese product, called OpenBlocks. However, some issues
happen by using this product. The product sometimes generates high
heat (e.g., a certain hot day). The heat is hot enough to feel like
getting burned. In fact, the maximum degree of the heat reaches
Ooki & Kamei Expires June 23, 2016 [Page 14]
Internet-Draft Internet Measurement System December 2015
about eighty degrees Celsius. Some people were afraid to set up MA
in their house. We are looking for more efficient product satisfying
our requirements than this model.
o Selection of Measurement Target Contents
It is difficult to decide what contents should be measured to present
the representative performance. There are many kinds of contents on
the Internet.
This time we have selected the Target Contents based on the volume of
transferred data at some points in an ISP. However, there are more
metrics to consider, such as the number of accesses to that contents,
rather than the transferred volume. Other metrics are not studied in
this document.
o Stable Operation
We had experiences where the measurement results were not sent
immediately, and the measurements for some Target Contents were
failed. Although the actual causes of these difficulties vary (e.g.,
accidentally disconnected LAN cable or power cable), we could easily
respond to those issues using informations (e.g., time and place)
contained in the centralized logs in the Collector Server. Another
difficulty is the change in the settings of the contents provider.
For example, wget command for a video content has not worked due to a
change in a setting in the contents provider. This issue is
difficult to tackle and is left for future work.
5.3. Security Issue
o Measurement between ISPs and CSPs
If we continuously measure the performance about the contents in the
Internet, from the point of the contents service provider, it can be
obstacle to provide the stable service due to the traffic volume for
measurement. However, we need real situations for the customer to
measure the performance as correctly as possible. The service
provider may request the limitation, e.g., volume and the number of
access, for measurement to MA. In terms of the combination between
ISPs and CSPs, we may need the condition for measurement.
o DDoS Attack
We placed approximately 150 MAs all over Japan. These MAs may become
DDoS attackers by wrong commands from the Controller Server. From
this reason, the list of commands MAs can perform should be
restricted. And also, the MAs must deny illegal accesses and logins.
Ooki & Kamei Expires June 23, 2016 [Page 15]
Internet-Draft Internet Measurement System December 2015
MAs should permit only access through instruction from the Controller
Server.
6. Security Considerations
As described in Setion 5.3, security consideration for Internet
measurement must be considered.
7. IANA Considerations
No need to describe any request regarding number assignment.
8. References
8.1. Normative References
[RFC7536] Linsner, M., Eardley, P., Burbridge, T.,and Sorensen, F.,
"Large-Scale Broadband Measurement Use Cases, May 2015", .
[I-D.deng-lmap-collaboration]
Deng., L., Huang, R.,and Duan, S., "Use-cases for
Collaborative LMAP, draft-deng-lmap-collaboration-05 (work
in progress), June 2015", .
[I-D.ietf-lmap-framewark]
Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
Aitken, P., and Akhter, P., "A framework for large-scale
measurement platforms(LMAP),draft-ietf-lmap-framework-14
(work in progress), April 2015", .
[I-D.ietf-lmap-yang]
Schoenwaelder, J.,and Bajpai, V., "A YANG Data Model for
LMAP Measurement Agents, draft-ietf-lmap-yang-00 (work in
progress), April 2015", .
8.2. URL References
[MIC] MIC Report, "http://www.soumu.go.jp/main_sosiki/kenkyu/
speed_measurement/", .
[sandvine]
Sandvine Report, "https://www.sandvine.com/pr/2014/5/14/
sandvine- report-netflix%E2%80%99s-british-invasion.html",
.
[google] Google Report,
"http://www.google.com/get/videoqualityreport/", .
Ooki & Kamei Expires June 23, 2016 [Page 16]
Internet-Draft Internet Measurement System December 2015
[plathome]
OpenBlocks,
"http://openblocks.plathome.com/products/ax3/", .
[IPv6-Promotion-Council]
Japanese IPv6-Promotion,
"http://v6pc.jp/jp/spread/ipv6spread_02.phtml", .
Authors' Addresses
Motoyuki Ooki
NTT Communications
GranPark 16F
3-4-1 Shibaura, Minato-ku, Tokyo
108-8118,Japan
EMail: m.ooki@ntt.com
Satoshi Kamei
NTT Communications
GranPark 16F
3-4-1 Shibaura, Minato-ku, Tokyo
108-8118,Japan
EMail: skame@nttv6.jp
Ooki & Kamei Expires June 23, 2016 [Page 17]
