COIN I. Kunze Internet-Draft J. Rueth Intended status: Informational K. Wehrle Expires: January 5, 2020 RWTH Aachen University July 4, 2019 Industrial Use Cases for In-Network Computing draft-kunze-coin-industrial-use-cases-00 Abstract Cyber-physical systems and the Industrial Internet of Things are characterized by diverse sets of requirements which can hardly be satisfied using standard networking technology. One example are latency-critical computations which become increasingly complex and are consequently outsourced to more powerful cloud platforms for feasibility reasons. The intrinsic physical propagation delay to these remote sites can, however, already be too high for given requirements. The challenge is to develop techniques that bring together these requirements. Utilizing available computational capabilities within the network can be a solution to this challenge which makes in-network computing concepts a promising starting point. This document discusses select industrial use cases to demonstrate how in-network computing concepts can be applied to the industrial domain and to point out essential requirements of industrial applications. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on January 5, 2020. Kunze, et al. Expires January 5, 2020 [Page 1] Internet-Draft Industrial Use Cases July 2019 Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. In-Network Control / Time-sensitive applications . . . . . . 4 2.1. Characterization and Requirements . . . . . . . . . . . . 5 2.1.1. Approaches . . . . . . . . . . . . . . . . . . . . . 5 3. Large Volume Applications/ Traffic Filtering . . . . . . . . 6 3.1. Characterization and Requirements . . . . . . . . . . . . 6 3.2. Approaches . . . . . . . . . . . . . . . . . . . . . . . 7 3.2.1. Traffic Filters . . . . . . . . . . . . . . . . . . . 7 3.2.2. In-Network (Pre-)Processing . . . . . . . . . . . . . 8 4. Industrial Safety (Dead Man's Switch) . . . . . . . . . . . . 9 4.1. Characterization and Requirements . . . . . . . . . . . . 9 4.1.1. Approaches . . . . . . . . . . . . . . . . . . . . . 9 5. Security Considerations . . . . . . . . . . . . . . . . . . . 10 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 7. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 10 8. Informative References . . . . . . . . . . . . . . . . . . . 11 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 1. Introduction The Internet is based on a best-effort network that provides limited guarantees regarding the timely and successful transmission of packets. This design-choice is suitable for general Internet-based applications, but specialized industrial applications demand a number of strict performance guarantees, e.g., regarding real-time capabilities, which cannot be provided over regular best-effort networks. Enhancements to the standard Ethernet such as Time-Sensitive- Networking [TSN] try to achieve the requirements on the link layer by statically reserving shares of the bandwidth. These concepts are Kunze, et al. Expires January 5, 2020 [Page 2] Internet-Draft Industrial Use Cases July 2019 well-suited for traditional industrial settings where the communication paths are encapsulated at the respective factory sites and where the communication patterns are well understood. Following the vision of the Industrial Internet of Things (IIoT), more and more parts of the industrial production domain are interconnected. This increases the complexity of the industrial networks, making them more dynamic and creating more diverse sets of requirements. Furthermore, process control is imagined to be exercised from remote clouds for feasibility reasons which is why solutions on the link layer alone are not sufficient in these scenarios. Common components of the IIoT can be divided into three categories as illustrated in Figure 1. Following [I-D.draft-mcbride-edge-data-discovery-overview-01], EDGE DEVICES, such as sensors and actuators, constitute the boundary between physical and digital world. They communicate the current state of the physical world to the digital world by transmitting sensor data or let the digital world interact with or manipulate the physical world by executing actions after receiving (simple) control information. The processing of the sensor data as well as the creation of the control information is done on COMPUTING DEVICES. They range from small-powered controllers in close proximity to the EDGE DEVICES, to more powerful edge or remote clouds in larger distances. The connection between the EDGE and COMPUTING DEVICES is established by NETWORKING DEVICES. In the industrial domain, they range from standard devices, e.g. typical Ethernet switches, which can interconnect all Ethernet-capable hosts, to proprietary equipment with proprietary protocols which only supports hosts of specific vendors. The challenge is to develop concepts which can include off-premise entities (such as distant cloud platforms) as well as proprietary hosts into the communication and still satisfy the performance requirements of modern industrial networks. The in-network computing paradigm presents a promising starting point because (pre-)processing data within the network can speed up the communication, e.g., by reducing the amount of transmitted data and thus congestion. Flexibly distributing the computation tasks across the network helps to manage dynamic changes. Specifying general requirements for the different application scenarios is difficult due to the mentioned diversity. In an effort to showcase potential requirements for the domain of industrial production, we characterize and analyze three distinct scenarios to illustrate how in-network computations can be helpful. Kunze, et al. Expires January 5, 2020 [Page 3] Internet-Draft Industrial Use Cases July 2019 -------- |Sensor| ------------| ~~~~~~~~~~~~ ------------ -------- ------------- { Internet } --- |Remote Cloud| . |Access Point|--- ~~~~~~~~~~~~ ------------ -------- ------------- | | |Sensor| ----| | | | -------- | | -------- | . | | |Switch| ---------------------- . | | -------- | . | | ------------ | ---------- | |----------------- | Controller | | |Actuator| ------------ ------------ | ---------- | -------- ------------ . |----|Switch|---------------------------| Edge Cloud | ---------- -------- ------------ |Actuator| ---------| ---------- |-----------| |------------------| |-------------------| EDGE DEVICES NETWORKING DEVICES COMPUTING DEVICES Figure 1: Industrial networks show a high level of heterogeneity. 2. In-Network Control / Time-sensitive applications The control of physical processes and components of a production line is a cornerstone of the industrial domain. It is essential for the growing automation of production and ideally allows for a consistent quality level. Traditionally, the control has been exercised by control software running on programmable logic controllers (PLCs) located directly next to the controlled process or component. This approach is best-suited for settings with a simple model that is focussed on a single or few controlled components. Modern production lines and shop floors are characterized by an increasing amount of involved devices and sensors, a growing level of dependency between the different components, and more complex control models. A centralized control is desirable to manage the large amount of available information which often has to be pre-processed or aggregated with other information before it can be used. PLCs are not designed for this array of tasks and computations could theoretically be moved to more powerful devices. These devices are no longer in close proximity to the controlled objects and induce additional latency. It is worthwhile to investigate whether the outsourcing of control functionality to distant computation platforms is viable, because these platforms have a high level of flexibility and scalability. In Kunze, et al. Expires January 5, 2020 [Page 4] Internet-Draft Industrial Use Cases July 2019 the following, we describe the requirements and characteristics of the control setting in more detail. 2.1. Characterization and Requirements A control process consists of two main components as is illustrated in Figure 2: a system under control and a controller. In feedback control, the current state of the system is monitored, e.g., using sensors, and the controller influences the system based on the difference between the current and the reference state to keep it close to this reference state. Apart from the control model, the quality of the control primarily depends on the timely reception of the sensor feedback, because the controller can only react if it is notified about changes in the system state. Depending on the dynamics of the controlled system, the control can be subject to tight latency constraints, often in the single digit millisecond range. While low latencies are important, there is an even greater need for stable and deterministic levels of latency, because controllers can generally cope with different levels of latency if they are designed for them, but they are significantly challenged by dynamically changing or unstable latencies. This is especially true if off-premise cloud platforms are included due to the unpredictable latency of the Internet. The main requirements for the industrial control scenario are low and stable latencies to ensure that processes can work continuously and that no machines are damaged. reference state ------------ -------- Output ----------> | Controller | ---> | System | ----------> ^ ------------ -------- | | | | observed state | | --------- | -------------------| Sensors | <----- --------- Figure 2: Simple feedback control model 2.1.1. Approaches Control models in general can become complex but there is a variety of control algorithms that are composed of simple computations such as matrix multiplication. As these are supported by programmable network devices, it is a possibility to compose simplified approximations of the more complex algorithms and deploy them in the network. While the simplified versions induce a more inaccurate Kunze, et al. Expires January 5, 2020 [Page 5] Internet-Draft Industrial Use Cases July 2019 control, they allow for a quicker response and might be sufficient to operate a basic tight control loop while the overall control can still be exercised from the cloud. The problem, however, is that networking devices typically only allow for integer precision computation while floating point precision is needed by most control algorithms. Early approaches like [RUETH] have already shown the general applicability of such ideas, but there are still a lot of open research questions not limited to the following: o How can one derive the simplified versions of the overall controller? * How complex can they become? * How can one take the limited computational precision of networking devices into account when making them? o How does one distribute the simplified versions in the network? o How does the overall controller interact with the simplified versions? 3. Large Volume Applications/ Traffic Filtering In the IIoT, processes and machines can be monitored more effectively resulting in more available information. This data can be used to deploy machine learning techniques and consequently help to find previously unknown correlations between different components of the production which in turn helps to improve the overall production system. Newly gained knowledge can be shared between different sites of the same company or even between different companies. Traditional company infrastructure is neither equipped for the management and storage of such large amounts of data nor for the computationally expensive training of ML approaches. Similar to the considerations in Section 2, off-premise cloud platforms offer cost- effective solutions with a high degree of flexibility and scalability. While the unpredictable latency of the Internet is only a subordinate problem for this use case, moving all data to off- premise locations primarily poses infrastructural and security challenges which are presented in more detail in the following. 3.1. Characterization and Requirements Processes in the industrial domain are monitored by distributed sensors which range from simple binary (e.g., light barriers) to complex sensors measuring the system with varying degrees of resolution. Sensors can further serve different purposes, as some Kunze, et al. Expires January 5, 2020 [Page 6] Internet-Draft Industrial Use Cases July 2019 might be used for the time-critical process control while others are only used as redundant fall back platforms. Overall, there is a high level of heterogeneity which makes managing the sensor output a challenging task. Depending on the deployed sensors and the complexity of the observed system, the resulting overall data volume can easily be in the range of several Gbit/s [GLEBKE]. Using off-premise clouds for managing the data requires uploading or streaming the growing volume of sensor data using the companies' Internet access which is typically limited to a few hundred of Mbit/s. While large networking companies can simply upgrade their infrastructure, most industrial companies rely on traditional ISPs for their Internet access. Higher access speeds are hence tied to higher costs and, above all, subject to the supply of the ISPs and consequently not always available. A major challenge is thus to devise methodology which is able to handle such amounts of data over limited access links. Another aspect is that business data leaving the premise and control of the company further comes with security concerns, as sensitive information or valuable business secrets might be contained in it. Typical security measures such as encrypting the data makes in- network computing techniques hardly applicable as they typically work on unencrypted data. Adding security to in-network computing approaches, either by adding functionality for handling encrypted data or devising general security measures, is thus a very promising field for research. 3.2. Approaches While there is no work on the question of security yet, there are at least two concepts which might be suitable for reducing the amount of transmitted data in a meaningful way: 1. filtering out redundant or unnecessary data 2. aggregating data by applying preprocessing steps within the network Both concepts require detailed knowledge about the monitoring infrastructure at the factories and the purpose of the transmitted data. 3.2.1. Traffic Filters Sensors are often set up redundantly, i.e., part of the collected data might also be redundant. Moreover, they are often hard to configure or not configurable at all which is why their resolution or Kunze, et al. Expires January 5, 2020 [Page 7] Internet-Draft Industrial Use Cases July 2019 sampling frequency is often larger than required. Consequently, it is likely that more data is transmitted than is actually needed or desired. A trivial idea for reducing the amount of data is thus to filter out redundant or undesired data before it leaves the premise using simple traffic filters that are deployed in the on-premise network. In this context, the following research questions can be of interest: o How can traffic filters be designed? o How can traffic filters be coordinated and deployed? o How can traffic filters be changed dynamically? 3.2.2. In-Network (Pre-)Processing There are manifold computations that can be performed on the sensor data in the cloud. Some of them are very complex or need the complete sensor data during the computation, but there are also simpler operations which can be done on subsets of the overall dataset or earlier on the communication path as soon as all data is available. One example is finding the maximum of all sensors values which can either be done iteratively on each intermediate hop or at the first hop, where all data is available. Using expert knowledge about the exact computation steps and the concrete transmission path of the sensor data, simple computation steps can be deployed in the on-premise network to reduce the overall data volume and potentially speed up the processing time in the cloud. Related work has already shown that in-network aggregation can help to improve the performance of distributed machine learning applications [SAPIO]. Investigating the applicability of stream data processing techniques to programmable networking devices is also interesting, because sensor data is usually streamed. In this context, the following research questions can be of interest: o Which (pre-)processing steps can be deployed in the network? * How complex can they become? o How can applications incorporate the (pre-)processing steps? o How can the programming of the techniques be streamlined? Kunze, et al. Expires January 5, 2020 [Page 8] Internet-Draft Industrial Use Cases July 2019 4. Industrial Safety (Dead Man's Switch) Despite increasing automation in production processes, human workers are still often necessary. This gives safety measures a high priority to ensure that no human life is endangered. In traditional factories, the regions of contact between humans and machines are well-defined and interactions are simple. Simple safety measures like emergency switches at the working positions are enough to provide a decent level of safety. Modern factories are characterized by increasingly dynamic and complex environments with new interaction scenarios between humans and robots. Robots can either directly assist humans or perform tasks autonomously. The intersect between the human working area and the robots grows and it is harder for human workers to fully observe the complete environment. Additional safety measures are important to prevent accidents and support humans in observing the environment. The increased availability of sensor data and the detailed monitoring of the factories can help to build additional safety measures if the corresponding data is collected early at the correct position. 4.1. Characterization and Requirements Industrial safety measures are typically hardware solutions, because they have to pass rigorous testing before they are certified and deployment-ready. Common measures include safety switches, which need to be triggered manually, and light barriers. Additionally, the working area can be explicitly divided into 'contact' and 'safe' areas, indicating when workers have to watch out for interactions with machinery. These measures are static solutions, potentially relying on special hardware, and are challenged by the increased dynamics of modern factories. Software solutions offer a higher flexibility as they can dynamically respect new information gathered by the sensor systems. Depending on the corresponding occupational safety laws, the software has to satisfy very strict requirements which cannot be satisfied by regular best-effort networks. 4.1.1. Approaches Software-based solutions can take advantage of the large amount of available sensor data. Different safety indicators within the production hall can be combined within the network so that programmable networking devices can give early responses if a potential safety breach is detected. A rather simple possibility Kunze, et al. Expires January 5, 2020 [Page 9] Internet-Draft Industrial Use Cases July 2019 could be to track the positions of human workers and robots. Whenever a robot gets too close to a human in a non-working area or if a human enters a certain safety zone, robots are stopped to prevent injuries. More advanced concepts could also include image data or combine arbitrary sensor data. In this context, the following research questions can be of interest: o How can the software give guaranteed safety over best-effort networks? o Which sensor information can be combined and how? 5. Security Considerations N/A 6. IANA Considerations N/A 7. Conclusion In-network computing concepts have the potential to improve industrial applications. There are at-least three scenarios for which in-network processing can be beneficial, each having a unique set of requirements. In the control scenario, tight latency constraints in the single digit millisecond range have to be satisfied despite the use of cloud platforms and the corresponding unstable latency of the Internet. In a second scenario, large amounts of data have to be transmitted to cloud platforms for further evaluation. One important task here is to reduce the amount of data that needs to be transmitted as the available Internet access speed is most likely non-sufficent. Apart from that, security measures have to be implemented as business data is transmitted to the Internet. Regarding safety, software-based measures often lack the required guarantees and do not withstand the testing for certification. In- network processing with its potential for early responses can be a solution by combining different sensor outputs early and acting quickly. Kunze, et al. Expires January 5, 2020 [Page 10] Internet-Draft Industrial Use Cases July 2019 8. Informative References [GLEBKE] Glebke, R., "A Case for Integrated Data Processing in Large-Scale Cyber-Physical Systems", DOI: 10125/60162, in HICSS, January 2019. [I-D.draft-mcbride-edge-data-discovery-overview-01] McBride, M., Kutscher, D., Schooler, E., and C. Bernardos, "Overview of Edge Data Discovery", draft-mcbride-edge- data-discovery-overview-01 (work in progress), March 2019. [RUETH] Rueth, J., "Towards In-Network Industrial Feedback Control", DOI: 10.1145/3229591.3229592, in ACM SIGCOMM NetCompute, August 2018. [SAPIO] Sapio, A., "Scaling Distributed Machine Learning with In- Network Aggregation", 2019, . [TSN] "Time-Sensitive Networking (TSN) Task Group", 2019, . Authors' Addresses Ike Kunze RWTH Aachen University Ahornstr. 55 Aachen D-50274 Germany Phone: +49-241-80-21422 Email: kunze@comsys.rwth-aachen.de Jan Rueth RWTH Aachen University Ahornstr. 55 Aachen D-50274 Germany Phone: +49-241-80-21417 Email: rueth@comsys.rwth-aachen.de Kunze, et al. Expires January 5, 2020 [Page 11] Internet-Draft Industrial Use Cases July 2019 Klaus Wehrle RWTH Aachen University Ahornstr. 55 Aachen D-50274 Germany Phone: +49-241-80-21401 Email: wehrle@comsys.rwth-aachen.de Kunze, et al. Expires January 5, 2020 [Page 12]