Internet DRAFT - draft-rokui-5g-ietf-network-slice

draft-rokui-5g-ietf-network-slice







Individual                                                      R. Rokui
Internet-Draft                                                     Nokia
Intended status: Informational                                  S. Homma
Expires: May 6, 2021                                                 NTT
                                                               X. de Foy
                                                       InterDigital Inc.
                                                           LM. Contreras
                                                              Telefonica
                                                              P. Eardley
                                                                      BT
                                                            K. Makhijani
                                                      Futurewei Networks
                                                               H. Flinck
                                                                   Nokia
                                                           R. Schatzmayr
                                                        Deutsche Telekom
                                                            A. Tizghadam
                                                TELUS Communications Inc
                                                                 C. Janz
                                                                   H. Yu
                                                           Huawei Canada
                                                        November 2, 2020


           IETF Network Slice for 5G and its characteristics
                  draft-rokui-5g-ietf-network-slice-00

Abstract

   5G Network slicing is an approach to provide separate independent
   end-to-end logical network from User Equipment (UE) to various mobile
   applications where each network slice has its own Service Level
   Agreement (SLA) and Objectives (SLO) requirements.  Each end-to-end
   network slice consists of a multitude of contexts across RAN, Core
   and transport domains each with its own controller.  To provide
   automation, assurance and optimization of the 5G the network slices,
   a 5G E2E network slice orchestrator is needed which interacts with
   controllers in RAN, Core and Transport network domains.  The
   interfaces between the 5G E2E network slice orchestrator and RAN and
   Core controllers are defined in various 3GPP technical
   specifications.  However, 3GPP has not defined a similar interface
   for transport network.

   The aim of this document is to describe E2E network slicing and its
   relation to "IETF network slice" for 5G use-case.  It also provides
   an information model for control and mangement of IETF network slices
   for 5G.




Rokui, et al.              Expires May 6, 2021                  [Page 1]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on May 6, 2021.

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
     1.1.  Definition of Terms . . . . . . . . . . . . . . . . . . .   4
   2.  Architecture of a 5G end-to-end network slice . . . . . . . .   5
   3.  Typical flow for fulfillment of a 5G E2E network slice  . . .   8
   4.  Definition of 5G IETF Network Slice . . . . . . . . . . . . .  11
     4.1.  5G IETF Network Slices in Distributed RAN deployment  . .  12
     4.2.  5G IETF Network Slices in Centralized RAN deployment  . .  13
     4.3.  5G IETF Network Slices in Cloud RAN (C-RAN) . . . . . . .  13
     4.4.  5G IETF Network Slice as a set of Connection Groups . . .  14
   5.  IETF Network Slice Controller NBI for 5G  . . . . . . . . . .  16
     5.1.  Relationship between 5G IETF Network Slice NBI and
           various IETF data models  . . . . . . . . . . . . . . . .  17
   6.  5G IETF Network Slice NBI Information Model . . . . . . . . .  18
   7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  22



Rokui, et al.              Expires May 6, 2021                  [Page 2]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   8.  Security Considerations . . . . . . . . . . . . . . . . . . .  22
   9.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  22
   10. Informative References  . . . . . . . . . . . . . . . . . . .  23
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  25

1.  Introduction

   Network slicing offers network operators a mechanism to allocate
   dedicated infrastructure, resources and services from a shared
   operator's network to a customer for specific use-case.  As discussed
   in draft [I-D.nsdt-teas-ietf-network-slice-definition], there are a
   number of use-cases benefiting from network slicing including:

   o  5G network slicing (See [TS.23.501-3GPP])

   o  Network wholesale services

   o  Network sharing among operators

   o  NFV connectivity and Data Center Interconnect

   It is important to note that the concept of network slicing is not
   only limited to 5G but other use-cases can also benefit from it as
   shown above.  However, the 5G use case is one of the important use
   cases for network slicing.  This memo will discuss the 5G use-case in
   more details.  In specific, 5G network slicing is a mechanism which a
   mobile network operator can use to allocate dedicated
   infrastructures, resources and services from a shared mobile and
   transport network to a 5G customer for specific 5G use-case.

   A 5G network slice is inherently an E2E concept and is composed of
   multiple logical independent networks in a common operator's network
   from a user equipment to various 5G applications.  In particular, 5G
   network slicing receives attention due to factors such as diversity
   of services and devices in 5G each with its own SLA requirements.
   Each 5G E2E network slice consists of multiple 5G RAN, 5G Core and
   transport network domains each with its own controller (See
   [TS.28.531-3GPP].

   To enable automation, assurance and optimization of 5G network
   slices, an E2E network slice orchestrator is needed which interacts
   with 5G RAN, 5G Core and Transport network controllers.  The
   interfaces between the E2E network slice orchestrator and RAN and
   Core slice controllers are defined in various 3GPP technical
   specifications.  However, 3GPP has not defined the same interface
   towards the transport network.  Draft
   [I-D.wd-teas-transport-slice-yang] addresses the object model of such
   interface for all network slice use-cases.  However, for 5G network



Rokui, et al.              Expires May 6, 2021                  [Page 3]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   slicing, the current model shall be augmented to address the specific
   characteristics of the 5G network slices.  The aim of this document
   is to provide characteristics of 5G network slices and how it relares
   to "IETF network slice".  It also provides the IETF network slice
   interface specifications and its information model to be used for
   automation, monitoring and optimization of IETF network slices for
   5G.  See [I-D.contreras-teas-slice-nbi].

1.1.  Definition of Terms

   Please refer to [I-D.nsdt-teas-ietf-network-slice-definition] and
   [I-D.homma-slice-provision-models] as well.

   Tenant:  Also known as Customer.  A network slice tenant is a person
      or group that rents and occupies an instance of the network slice
      from network provider.

   5G End-to-end Network Slice:  A logical end-to-end network provided
      by a 5G network slice provider that has the functionality and
      performance to support a specific 5G service.  It spans multiple
      network domains (e.g. radio, transport and core) and in some cases
      more than one administrative domain.  It may well support dynamic
      modification or it might be long-lasting i.e. only change on
      commercial timescales.

   5G IETF Network Slice:  We will use the term "5G IETF network slice"
      throughout this draft.  It simply refers to IETF network slice
      define in [I-D.nsdt-teas-ietf-network-slice-definition] applicable
      to 5G.

   RAN Slice:  Also known in 3GPP as RAN Sub-Slice or RAN Slice-Subnet.
      The context and personality created on RAN network functions to
      address the 5G radio portion of a 5G E2E network slice.

   Core Slice:  Also known in 3GPP as Core Sub-Slice or Core Slice-
      Subnet.  The context and personality created on Core network
      functions to address the 5G Core portion of a 5G E2E network
      slice.

   S-NSSAI:  Single-Network Slice Selection Assistance Information,
      defined by 3GPP which is the identification of a 5G E2E Network
      Slice

   gNB:  The radio portion of a 5G E2E network slice and in a
      distributed radio deployment (called Cloud-RAN), it incorporates
      two major modules; Central Unit (CU) and Distribute Unit (DU)





Rokui, et al.              Expires May 6, 2021                  [Page 4]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   DU:  Distributed Unit: This logical unit includes a subset of gNB
      real-time functions.  Its operation is controlled by the CU.

   CU:  Central Unit: It is a logical unit that includes the gNB non-
      realtime functions.

   UE:  User Equipment such as vehicle infotainment unit, cell phone,
      IoT sensor and etc.

   RAN:  Radio Access Network is the part of a mobile system that
      connects individual devices to other parts of a network through
      radio connections.  It provides connection between user equipment
      (UE) and mobile core network.

   Transport Domain:  Transport domain is a network domain implemented
      by the deployment of IETF network technologies.

2.  Architecture of a 5G end-to-end network slice

   To demonstrate the concept of 5G E2E network slice and the role of
   various controllers, consider a typical 5G network shown in Figure 1
   where a mobile network operator Y has two customers C1 and C2.  The
   boundaries of administrative domain of the operator includes RAN,
   transport, Core and mobile application domains.  Customer C1 and C2
   request to have one or more logical independent E2E networks from UEs
   (e.g. vehicle infotainment, mobile phone, IoT meters etc.) to the 5G
   application servers, each with its own distinct SLO.

   Each of these independent networks is called a 5G E2E network slice.
   Each E2E network slice comprised of three componets RAN slice, IETF
   network slice, and Core slice, respectively representing RAN,
   transport and core domain portions of the slice.



















Rokui, et al.              Expires May 6, 2021                  [Page 5]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


    <---------------- 5G End to End Network Slice ----------------->

    <-- RAN --> <-- IETF Network --> <- Core -> <-- IETF Network -->
        Slice       Slice 1             Slice        Slice 2

      ......... ................... ............ .................
      :       : :                 : :          : :               :
      :       : :                 : :          : :               :
NS1  ----------------------------------------------------------------
NS2  ================================================================
NS3  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
      :       : :                 : :          : :               :
NS4  - - - - - - - - - -  - - - - - - - - - - - - - - - - - - - - - -
      :       : :                 : :          : :               :
      :       : :                 : :          : :               :
      :.......: :.................: :..........: :...............:

       RAN         Transport          Core        Transport      App
       Network     Network 1          Network     Network 2      Servers

Legend:
 ----- NS1: 5G E2E NS 1 for customer C1, service type Infotainment
 ===== NS2: 5G E2E NS 2 for customer C1, service type Autonomous Driving
 +++++ NS3: 5G E2E NS 3 for customer C1, service type HD Map
 - - - NS4: 5G E2E NS 4 for customer C2, service type CCTV


    Figure 1: High level architecture of a 5G end-to-end network slice

   In Figure 1 mobile network operator Y has created four 5G E2E network
   slices, NS1, NS2, NS3 and NS4, each with its own RAN, Core and IETF
   network slices.  To create a RAN slice, the RAN network function s
   such as eNB and gNB should be programmed to have a context for each
   5G E2E network slice.  This context means that the RAN network
   functions should allocate certain resources for each 5G E2E network
   slice they belong to such as air interface, various schedulers,
   policies and profiles to guarantee the SLO requirement for that
   specific network slice.  By the same token, the Core slices will be
   created which means that the mobile network operator will create the
   context for each 5G E2E network slice on Core network functions.

   For each 5G E2E network slice NS1, NS2, NS3 and NS4, after creation
   of RAN and Core slices, they should be connected to each other and be
   connected to mobile application servers to form the 5G E2E network
   slice.  As defined in [I-D.nsdt-teas-ietf-network-slice-definition],
   the set of connections are referred to "IETF Network Slices" and
   specifically for 5G they are referred to "5G IETF Network Slices".




Rokui, et al.              Expires May 6, 2021                  [Page 6]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   Referring to Figure 1, for each 5G E2E network slice, the following
   5G IETF network Slices are needed:

   o  5G IETF Network Slice 1: To connect RAN slice to Core slice in
      Transport Network 1

   o  5G IETF Network Slice 2: To connect Core slice to Mobile
      Application Servers in Transport Network 2.  This might be needed
      if the mobile application servers are connected to core network
      functions through a transport network.  For example, if the Core
      slice, which is realized on VNFs, and mobile application servers
      are in the same data center, the 5G IETF Network Slice 2 is not
      needed.  In this case the transport network 2 does not exist.

   Note that as we will see later in Section 4.1, Section 4.2 and
   Section 4.3, the number of "5G IETF network slices" might be more
   than two which depends on some factors such as RAN deployment:

   After creation of RAN, Core and 5G IETF network slices, they will be
   associated together to form a working 5G E2E network slice identified
   by an ID referred as to S-NSSAI.  Please refer to [TS.23.501-3GPP]
   for more info on S-NSSAI.

   To support fully automated enablement and assurance of 5G E2E network
   slices, multiple controllers are needed to perform the life cycle of
   5G E2E network slices in RAN, Core and Transport domains.  As shown
   in Figure 2 each RAN, Core and Transport domain needs its own
   controller called RAN Slice Controller, Core Slice Controller and
   IETF Network Slice Controller.  In addition, an E2E network slice
   orchestrator is needed to provide coordination and control of network
   slices from an E2E perspective.

   In summary, a 5G E2E network slice will involve several domains, each
   with its own controller; 5G RAN, 5G Core and transport domains need
   to be coordinated in order to deliver an E2E mobile service.  Note
   that in this context a service is not an IP/MPLS service but rather
   customer facing services such as CCTV service, eMBB service,
   Infotainment service and so on.













Rokui, et al.              Expires May 6, 2021                  [Page 7]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


    |---------------------------------------------------------|
    |              E2E Network Slice Orchestrator             |
    |---------------------------------------------------------|

    |----------------| |-------------------| |----------------|
    |   RAN Slice    | |  IETF Network     | |   Core Slice   |
    |   Controller   | |  Slice Controller | |   Controller   |
    |----------------| |-------------------| |----------------|

    .......... ................... ........... ................
    :        : :                 : :         : :              :
    :        : :                 : :         : :              :
    : RAN    : :   Transport     : : Core    : :   Transport  :  Mobile
    : Network: :   Network 1     : : Network : :   Network 2  :  App
    :        : :                 : :         : :              :  Servers
    :        : :                 : :         : :              :
    :........: :.................: :.........: :..............:


       Figure 2: Various controllers for 5G end-to-end network slice

3.  Typical flow for fulfillment of a 5G E2E network slice

   Figure 3 provides a typical flow across various controllers,
   orchestrator, NFVO and RAN/Transport/Core networks to achieve the
   automatic creation of a 5G E2E network slices such as NS1, NS2, NS3
   or NS4 shown in Figure 1.  Below are typical steps from the time a
   customer sends its request for a 5G E2E network slice creation to the
   operators network until the network slice is created and ready to be
   used by the customer.  It is important to note that in practice some
   of these steps can be combined or re-ordered.

   1.   The customer C1 requests, from operator Y, the creation of a 5G
        E2E network slice NS1 for Infotainment service type and SLO of
        10 [Mbps]

   2.   The 5G E2E network slice orchestrator receives this request and
        using its pre-defined network slice blueprints (a.k.a. network
        slice templates), creates a network slice profile (a.k.a.
        network slice instance) containing all the network functions in
        RAN and core which should be part of this E2E network slice.  It
        then goes through decomposition of this profile and triggers
        various actions towards RAN, Core and transport domains.

   3.   A request for creation of 5G RAN slice will be sent to RAN Slice
        Controller.





Rokui, et al.              Expires May 6, 2021                  [Page 8]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   4.   If new instances of virtual RAN network functions are needed,
        the RAN Slice Controller triggers the creation of new VNFs in
        RAN (using for example ETSI interface Os-Ma-nfvo)

   5.   NFVO manages the life cycle of virtual RAN network functions

   6.   Since both physical and virtual RAN network functions which are
        part of 5G E2E network functions are known to the RAN slice
        controller, it triggers the creation of a RAN slice by
        programming 5G RAN network functions

   7.   Similary to previous step (3), a request for creation of a Core
        slice will be sent to the Core Slice Controller.

   8.   If new instances of virtual Core network functions are needed,
        the Core Slice controller triggers the creation of new 5G core
        VNFs (using for example ETSI interface Os-Ma-nfvo) and NFVO
        manages the life cycle of virtual Core network functions

   9.   Since both physical and virtual 5G Core network functions as
        components of 5G E2E network functions are known to the Core
        slice controller, it triggers the creation of Core slice by
        programming 5G Core network functions

   10.  In this step, the creation of various 5G IETF network slices
        will be triggered.  Each 5G IETF network slice contains one or
        more connections between RAN network functions, Core network
        functions and 5G mobile applications.  For example, connectivity
        between 5G RAN and 5G Core slices, connectivity between 5G RAN
        network functions (such as DU to CU) or connectivity between 5G
        core slice and mobile applications.  Note that this step can be
        triggered by E2E network slice orchestrator, RAN slice
        controller, Core slice controller, or a combination of them.

   11.  [Optional] If the realization of a 5G IETF network slice
        involves creation of new VNFs (e.g.  Firewall, security gateway
        etc.), 5G IETF network slice controller triggers the creation of
        those VNFs (using for example ETSI interface Os-Ma-nfvo)

   12.  Various 5G IETF network slices will be realized in transport
        network.  Note that interface (10) is technology-agnostic
        whereas interface (12) is technology-specific

   13.  The E2E network slice orchestrator associates RAN slice, Core
        slice and 5G IETF network slices together to form a single 5G
        E2E network slice NS1





Rokui, et al.              Expires May 6, 2021                  [Page 9]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   14.  At the end, when the E2E network slice is created, the E2E
        network slice orchestrator will allocate a unique network slice
        id (called S-NSSAI) and eventually, during the UE network
        attach, UEs will be informed about the existence of this newly
        created E2E network slice.  UEs can request it using the 3GPP 5G
        signaling procedures.

   Note that the interfaces 3 and 7 between 5G E2E network slice
   orchestrator and RAN and Core slice controllers along with their
   information models are defined in various 3GPP technical
   specifications.  However, 3GPP has not defined the same interface for
   transport network (i.e. interface 10).

   The aim of this document is to define specific attributes related to
   5G network slices which will be used later to augment
   [I-D.wd-teas-transport-slice-yang].


   |----------------------------------------------------|
   |                 Customer portal                    |
   |----------------------------------------------------|
                            |(1)
                            V
   |----------------------------------------------------|
   | Generate NS Profile (aka NSI) using NS Blueprints  | 5G E2E
   |                        |(2)                        | Network Slice
   |                        V                           | Orchestrator
   |  Decompose NS Profile and trigger various actions  |
   |----------------------------------------------------|
         |(3)                 |(10)             |(7)
         |         |--------| | |------|        |
         V         |        V V V      |        V
   --------------- | ----------------- | ----------------
   | RAN         | | | IETF          | | |  Core        | Domain
   | Slice       |-| | Network Slice | |-|  Slice       | Controllers
   | Controller  |   | Controller    |   |  Controller  |
   ---------------   -----------------   ----------------
     (6)|  |(4)        (12)|  |(11)          (8)|  |(9)
        |  |               |  |--------|        |  |
        |  |-------------- | --------| | |------|  |
        |                  |         V V V         |
        |                  |     |----------|      |
        |                  |     |   NFVO   |      |
        |                  |     |----------|      |
        |                  |          |            |
   ..............  .................  |  .................
   :   RAN      :  : IETF Network  :  |  :     Core      :
   :   Slice    :  : Slice         :  |  :     Slice     :



Rokui, et al.              Expires May 6, 2021                 [Page 10]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   :............:  :...............:  |  :...............:
   .... | ................ | ........ | .......... | ..... 5G E2E
   :    |                  |          |            |     : Network Slice
   :... | ................ | ........ | .......... | ....: (13)
        |                  |          |            |
     (6)|              (12)|       (5)|            |(9)
        V                  V          V            V
   |-----------------------------------------------------|
   |    ,--------.     ,-------------.    ,--------.     | Network of
   |   /   RAN    \   /   Transport   \  /   Core   \    | Mobile
   |   \  Network /   \   Network     /  \  Network /    | Operator "Y"
   |    `--------'     `-------------'    `--------'     |
   |-----------------------------------------------------|

   Legend:
     NS: 5G E2E Network Slice
     NSI: Network Slice Instance
     NFVO: NFV Orchestrator


     Figure 3: Typical flow for fulfillment of a 5G E2E network slice

4.  Definition of 5G IETF Network Slice

   Referring to [I-D.nsdt-teas-ietf-network-slice-definition], the IETF
   network slice is define as follows:

   An IETF network slice is a logical network topology connecting a
   number of endpoints with a set of shared or dedicated network
   resources.  These resources are used to satisfy specific Service
   Level Objectives (SLOs).

   The 5G IETF Network slice specification is technology-agnostic.
   Using the above-mentioned definition, the 5G IETF network slice is
   define as follows:

   5G IETF network slices are sets of connections among the following
   network functions and mobile applications:

   o  5G RAN slice and 5G Core slice

   o  5G Core slice and mobile application server

   o  Among 5G RAN network functions DU to CU

   o  Among 5G RAN network functions RU to DU





Rokui, et al.              Expires May 6, 2021                 [Page 11]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   In Section 4.1, Section 4.2 and Section 4.3, the details of various
   5G IETF network slices for following RAN deployment will be provided:

   o  Distributed RAN

   o  Centralized RAN

   o  Cloud RAN (C-RAN)

4.1.  5G IETF Network Slices in Distributed RAN deployment

   Distributed RAN is the most common deployment of 4G and 5G RAN
   networks whereas shown in Figure 4, the RAN network is connected to
   Core network using the transport network 1.  Optionally the mobile
   applications can be also connected to the Core network using another
   transport network 2.

   In this case, a single 5G E2E network slice contains not only 5G RAN
   and 5G Core slices but two 5G IETF network slices INS_1 and INS_2.
   INS_1 connects the RAN slice to Core slice and INS_2 connects Core
   slice to mobile application servers (if needed).

         <------------- 5G E2E Network Slice  ------------->
         <--- RS -->              <-- CS -->
                    <--- INS_1 -->           <--- INS_2 --->
   ..................
   : RAN            :
   :                : .............           .............
   : |----| |-----| : :           : |------|  :           : |-----|
   : | RU | | BBU | : : Transport : | Core |  : Transport : | App |
   : |----| |-----| : : Network 1 : |------|  : Network 2 : |-----|
   :                : :...........:           :...........:
   :................:

   Legend
     INS: 5G IETF Network Slice
     RS: RAN Slice
     CS: Core Slice
     RU: Radio Unit
     BBU: BaseBand Unit
     App: Mobile Application Servers


      Figure 4: 5G IETF network slices in distributed RAN deployment







Rokui, et al.              Expires May 6, 2021                 [Page 12]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


4.2.  5G IETF Network Slices in Centralized RAN deployment

   The RAN consists of two functional units: the baseband unit (BBU) and
   the radio unit (RU).  The BBU processes the signal and is connected
   to the transport network.  The RU transmits and receives the carrier
   signal that is transmitted over the air to the end user equipment
   (UE).  In Centralized RAN as depicted in Figure 5, the RU and BBU are
   separated by a network called fronthaul network.

   In this deployment a single 5G E2E network slice contains not only 5G
   RAN and 5G Core slices but three 5G IETF network slices INS_1, INS_2
   and INS_3 where INS_1 and INS_2 are identical to their counterparts
   in distributed RAN deployment case (see Figure 4) and a new INS_3
   connects the Radio Units (RU) to the BBUs.

    <-------------------- 5G E2E Network Slice  -------------------->
    <-------- RS -------->              <-- CS -->
    <--- INS_3 --->        <--- INS_1 --->          <---- INS_2 ---->

...........................
:  RAN                    :
:        ........         : .............          .............
: |----| :      : |-----| : :           : |------| :           : |-----|
: | RU | :  FN  : | BBU | : : Transport : | Core | : Transport : | App |
: |----| :      : |-----| : : Network 1 : |------| : Network 2 : |-----|
:        :......:         : :...........:          :...........:
:                         :
:.........................:

Legend
  INS: 5G IETF Network Slice
  RS: RAN Slice
  CS: Core Slice
  FN: Fronthaul network
  RU: Radio Unit
  BBU: BaseBand Unit
  App: Mobile Application Servers



      Figure 5: 5G IETF network slices in Centralized RAN deployment

4.3.  5G IETF Network Slices in Cloud RAN (C-RAN)

   In Cloud-RAN deployment, the baseband unit (BBU) is further
   disaggregated into real-time and non-real-time components.  The
   former is deployed close to antenna to manages the real-time air
   interface resources while the non-real-time control functions are



Rokui, et al.              Expires May 6, 2021                 [Page 13]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   hosted centrally in the cloud.  In 5G, BBU is split into two parts
   called CU (Central Unit) and DU (Distributed Unit) as shown in
   Figure 6 where these entities are connected by a new network called
   Midhaul network.

   In this deployment a single 5G E2E network slice contains not only 5G
   RAN and 5G Core slices but four 5G IETF network slices INS_1, INS_2,
   INS_3 and INS_4 where INS_1, INS_2 and INS_3 are identical to their
   counterparts in centralized RAN deployment case (see Figure 5) and a
   new 5G IETF network slice INS_4 connects the DUs to CUs.

    <--------------------- 5G E2E Network Slice  --------------------->
    <-------------- RS --------------->          <- CS ->
    <--- INS_3 --->    <-- INS_4 -->  <-- INS_1 -->     <--- INS_2 --->
 ......................................
 :  RAN                               :
 :        ......        ......        : ........          ......
 :|----|  :    : |----| :    : |----| : :      : |------| :    : |-----|
 :| RU |  : FN : | DU | : MN : | CU | : : TN1  : | Core | :TN2 : | App |
 :|----|  :    : |----| :    : |----| : :      : |------| :    : |-----|
 :        :....:        :....:        : :......:          :....:
 :                                    :
 :....................................:

Legend
  INS: 5G IETF Network Slice
  RS: RAN Slice
  CS: Core Slice
  FN: Fronthaul network
  MN: Midhaul network
  TN: Transport network
  DU: Distributed Unit
  CU: Central Unit
  RU: Radio Unit
  App: Mobile Application Servers


         Figure 6: 5G IETF network slices in Cloud RAN deployment

4.4.  5G IETF Network Slice as a set of Connection Groups

   As discussed in [I-D.nsdt-teas-ietf-network-slice-definition], an
   IETF network slice contains one or more connections between various
   network functions, application and devices.  These connections can be
   grouped into various Connection Groups where each group has its own
   SLA/SLO.





Rokui, et al.              Expires May 6, 2021                 [Page 14]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   To further explore this concept in 5G E2E network slicing, consider
   Figure 7, where the details of 5G IETF network slice INS_1 introduced
   in Figure 4 is illustrated.  The 5G IETF network slice INS_1 is
   between 5G RAN and Core slices and has multiple connections between
   5G RAN network functions BBU1 and BBU2 and 5G Core network functions
   AMF1 and UPF1.  In particular, it contains the following connection
   groups, each with its own SLO where SLO-C and SLO-U might be
   different (e.g. they might be control and user plans SLOs):

   o  "Connection group C" connects BBU1 and BBU2 to AMF1 with SLO-C

   o  "Connection group U" connects BBU1 and BBU2 to UPF1 with SLO-U

   The combination of two connection groups will form the 5G IETF
   network slice INS_1.  Note that the definition of 5G IETF network
   slice INS_1 does not specify how these connections should be realized
   in transport network 1.  Although it is optionally possible, it is
   not necessarily mandatory for the definition of a 5G IETF network
   slice to state which technology (e.g.  IP, MPLS, Optics, PON etc.) or
   tunnel type (e.g.  RSVP-TE, SR-TE etc.) should be employed for
   realization.  As discussed in
   [[I-D.nsdt-teas-ietf-network-slice-definition], any of these
   technologies may be used by the IETF Network Slice Controller (NSC)
   to realize an IETF network slice.

   In summary, a 5G IETF network slice is a distinct set of technology-
   agnostic connection groups between various 5G network functions, 5G
   devices or 5G applications each with its own deterministic SLO which
   can be realized by any suitable technology, tunnel type and service
   type.





















Rokui, et al.              Expires May 6, 2021                 [Page 15]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


               <------- IETF Network Slice INS_1 ------->

   ......................   .....................   ...................
   :                    :   :                   :   :                 :
   :  --------- NSE1    :   :                   :   : NSE1 ---------  :
   :  |       | <--------------------------------------->  |       |  :
   :  | BBU1  | <+++++  :   :              /------------>  |  AMF1 |  :
   :  |       | NSE2  + :   :             /     :   :      |       |  :
   :  ---------        +:   :            /      :   :      ---------  :
   :                    :+  :           /       :   :                 :
   :                    : + :          /        :   :                 :
   :  --------- NSE1    :  +          /         :   :                 :
   :  |       | <-----------+--------/          :   : NSE1 ---------  :
   :  | BBU2  |         :    +++++++++++++++++++++++++++>  |       |  :
   :  |       | <+++++++++++++++++++++++++++++++++++++++>  |  UPF1 |  :
   :  --------- NSE2    :   :                   :   :      |       |  :
   :                    :   :                   :   :      ---------  :
   :....................:   ....................:   :.................:
          RAN                    Transport             Core
          Network                Network 1             Network


  5G IETF Network Slice INS_1:
      {"Connection group C" + "Connection group U"}
  Connection Group C {from BBU1.NSE1, BBU2.NSE1 to AMF1.NSE1 with SLO-C}
  Connection Group U {from BBU1.NSE2, BBU2.NSE2 to UPF1.NSE1 with SLO-U}

  Legend
    BBU: BaseBand Unit
    AMF: Access and Mobility Management Function
    UPF: User Plane Function
    NSE: 5G IETF Network Slice Endpoint
    ---- Connection group C
    ++++ Connection group U


     Figure 7: Details of 5G IETF Network Slice as a set of Connection
                                  Groups

5.  IETF Network Slice Controller NBI for 5G

   As discussed in [I-D.nsdt-teas-ietf-network-slice-definition] and
   [I-D.nsdt-teas-ns-framework], to fulfill (i.e. create, modify,
   delete) any IETF network slice and perform monitoring on it, an
   entity called IETF Network Slice Controller (NSC) is required to take
   abstract requests for 5G IETF network slices and realize them using
   suitable underlying technologies.  An IETF Network Slice Controller
   is the key building block for control and management of the transport



Rokui, et al.              Expires May 6, 2021                 [Page 16]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   slice.  It provides the creation/modification/deletion, monitoring
   and optimization of transport Slices in a multi-domain, a multi-
   technology and multi-vendor environment.

   Figure 8 shows the NSC and its NBI interface for 5G.  Draft
   [I-D.wd-teas-transport-slice-yang] addresses the base data model of
   the NSC NBI interface for all network slicing use-cases.  However,
   for 5G network slicing, the current model shall be augmented to
   include the specific characteristics of the 5G network slices for
   this interface.  The details of NSC NBI interface for 5G provided in
   Section 6.

            +------------------------------------------+
            |            5G Customer (Tenant)          |
            +------------------------------------------+
                               A
                               |
                               V
            +------------------------------------------+
            |    5G E2E Network Slice Orchestrator     |
            +------------------------------------------+
                               A
                               | NSC NBI
                               V
            +------------------------------------------+
            |    IETF Network Slice Controller (I-NSC)   |
            +------------------------------------------+
                               A
                               | NSC SBI
                               V
            +------------------------------------------+
            |          Network Controller(s)           |
            +------------------------------------------+


            Figure 8: IETF Network Slice Controller NBI for 5G

5.1.  Relationship between 5G IETF Network Slice NBI and various IETF
      data models

   As discussed in [I-D.nsdt-teas-ns-framework], the main task of the
   IETF Network Slice Controller is to map abstract IETF network slice
   requirements from NBI to concrete technologies on SBI and establish
   the required connectivity, and ensure that required resources are
   allocated to IETF network slice.  There are a number of different
   technologies that can be used on SBI including physical connections,
   MPLS, TSN, Flex-E, PON etc.  If the undelay technology is IP/MPLS/




Rokui, et al.              Expires May 6, 2021                 [Page 17]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   Optics, any IETF models can be used during the realization of IETF
   network slice.

   There are no specific mapping requirements for 5G.  The only
   difference is that in case of 5G, the NBI interface contains
   additional 5G specific attributes such as customer name, mobile
   service type, 5G E2E network slice ID (i.e.  S-NSSAI) and so on (See
   Section 6).  These 5G specific attributes can be employed by IETF
   Network Slice Controller during the realization of 5G IETF network
   slices on how to map NBI to SBI.  They can also be used for assurance
   of 5G IETF network slices.  Figure 9 shows the mapping between NBI to
   SBI for 5G IETF network slices.


                         | (1) NBI: Request to create/modify/delete
                         |          5G IETF Network Slice
                         V
             +----------------------+
             |  IETF Network Slice  | (2) Mapping between technology
             |    Controller (NSC)  |     agnostics NBI to technology
             +----------------------+     specific SBI
                       ^ ^ ^
                       | | |
                   |---| | |---|  (3) SBI: Realize 5G IETF Network Slice
                   |     |     |      by using various IETF models for
                   V     V     V      services, tunnels and paths
             +----------------------+
             |       Network        |-+
             |     Controller(s)    | |-+
             +----------------------+ | |
               +----------------------+ |
                 +----------------------+


     Figure 9: Relationship between transport slice interface and IETF
                     Service/Tunnels/Path data models

6.  5G IETF Network Slice NBI Information Model

   Based on the definition of 5G IETF Network slices (see Section 4),
   the high-level information model of northbound interface of IETF
   Network Slice Controller (NSC) for 5G IETF network slices should
   conform with Figure 10:


 module: 5g-ietf-network-slices
  +--rw 5g-ietf-network-slice
      +--rw 5g-ietf-network-slice-info



Rokui, et al.              Expires May 6, 2021                 [Page 18]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


        +--rw ins-id
        +--rw ins-name
        +--rw ins-plmn
        +--rw ins-hierarchical-tenant-id
     +--rw 5g-network-slice-info [s-nssai]
        +--rw s-nssai (i.e. 5G E2E network slice id)
        +--rw 5g-customer (i.e. 5G tenant)
        +--rw 5g-mobile-service-type (e.g. CCTV, infotainment etc)
     +--rw 5g-connection-group* [connection-group-id ]
        +--rw connection-group-id
        +--rw connection-group-name
        +--rw connection-group-type  (e.g., P2P, MP2MP, etc.)
        +--rw connection-group-status
           +-- admin-status
           +-- operational-status
        +--rw connection-group-member* [member-id]
           +--rw member-id
           +--rw member-name
           +--rw member //Ref. to 5G-ietf-connection-group-member
           +--ro member-slo-monitoring
             +--ro latency?
             +--ro jitter?
             +--ro loss?
        +--rw connection-group-slo-policy
           +--rw policy-id
           +--rw slo attributes
        +--rw connection-group-realization-policy //Optional
           +--rw policy-id
           +--rw realization-attributes //Optional
                                        //Technology-specific attributes
        +--rw connection-group-monitoring-policy //Optional
           +--rw policy-id
           +--rw monitoring-attributes //Optional. Such as if monitoring
                                       //is needed, frequency of
                                       //monitoring and how often send
                                       //them to NBI etc.
  +--rw 5g-ietf-network-slice-endpoint* [ep-id]
     +--rw ep-id
     +--rw ep-name
     +--rw domain-id
     +--rw node-id
     +--rw transport-port-id
     +--rw transport-vlan-id
     +--rw transport-id (e.g. IP address of the transport)
     +--rw transport-label (For future use)
     +--rw transport-bsid (For future use)
   +--rw 5G-ietf-connection-group-member* [member-id]
     +--rw member-id



Rokui, et al.              Expires May 6, 2021                 [Page 19]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


     +--rw member-endpoint-a  //Ref. to 5g-ietf-network-slice-endpoint
     +--rw member-endpoint-b  //Ref. to 5g-ietf-network-slice-endpoint


   Figure 10: Information model of NSC NBI interface for 5G IETF Network
                                  Slices

   The proposed information model should include the following building
   blocks:

   o  5g-ietf-network-slice-info: All attributes related to 5G IETF
      Network Slice.  It contains information such as 5G IETF network
      slice name, 5G IETF network slice ID, PLMN and hierarchical tenant
      ID etc.

   o  5g-network-slice-info: A list of all E2E network slices mapped to
      this 5G IETF network slice.  As discussed in Section 3, a request
      for creation of a 5G IETF network slice is sent from 5G E2E
      network slice orchestrator to IETF Network Slice Controller (NSC)
      for a customer and certain service type (e.g.  CCTV, Infotainment,
      URLLC, etc.).  It is NSC's decision to either create a new
      transport slice or use one of the existing ones.  As a result, the
      mapping between 5G E2E network slice and IETF network slice is
      many to one, i.e. one 5G IETF network slice can be used with
      multiple 5G E2E network slices.  The attributes of each 5G E2E
      network slices are included here.  The 5g-network-slice-info
      contains the list of E2E network slices which are mapped to a 5G
      IETF network slice with all relevant attributes such as S-NSSAI,
      customer name and service type.

   o  5g-connection-group: A 5G IETF network slice contains one or more
      connection groups each with its own SLA/SLO.  Each connection
      group contains:

      *  connection-group-attributes: A list of attributes for each 5g-
         connection-group such as connection-group-id, connection-group-
         name and connection-group-status

      *  connection-group-member: A list of members.  Each member is a
         connection between two endpoints.  A connection group can
         contain one or more members.  For example, the connections
         BBU1-UPF1 and BBU2-UPF1 are 2 members of "connection group U"
         in Figure 7.

      *  connection-group-slo-policy: This is a mandatory policy.  The
         connection-group-slo-policy represents in a generic and
         technology-agnostics fashion the SLO requirement needed to
         realize members of a connection group.  It contains SLOs such



Rokui, et al.              Expires May 6, 2021                 [Page 20]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


         as bounded latency, bandwidth, reliability, security etc.  Note
         that all members of a connection group must have the same SLO.

      *  connection-group-realization-policy: This is an optional
         policy.  In some scenarios, the 5G E2E network slice
         orchestrator might be able to influence the IETF Network Slice
         Controller on how to realize a 5G IETF network slice by
         providing some technology-specific information.

      *  connection-group-monitoring-policy: This is an optional policy.
         The 5G E2E network slice orchestrator can influence the IETF
         Network Slice Controller on how to perform monitoring,
         analytics and optimization on 5G IETF Network Slices.  It
         contains, the type of assurance needed, time interval,
         frequency of how often to inform the 5G E2E Network Slice
         Orchestrator etc.

   o  5g-ietf-network-slice-endpoint: It contains the list of all
      endpoints along with their attributes which belong to a 5G IETF
      network slice.  See Figure 11

   o  5G-ietf-connection-group-member: It contains the list of all
      members of connectin groups along with their attributes which
      belong to a 5G IETF network slice.



























Rokui, et al.              Expires May 6, 2021                 [Page 21]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


                         transport-port-id
                         transport-vlan-id
                         transport-id (e.g. IP address of the transport)
                         transport-label (For future use)
                         transport-bsid (For future use)
                              |
                  DAN         |
           |----------------| |
           |                | |          |--------------|
           |      o         | V          |   Transport  |
           |     NSE        |------------|   Network    |
           |                |            |              |
           |                |            |--------------|
           |----------------|
                  ^
                  |
                  |
                ep-id (e.g. the IP address)
                ep-name
                domain-id
                node-id

    Legend:
       DAN: Device, application and/or network function
       NSE: IETF Network Slice Endpoint


         Figure 11: Details of the 5G IETF network slice endpoints

7.  IANA Considerations

   This memo includes no request to IANA.

8.  Security Considerations

   TBD

9.  Acknowledgments

   The authors would like to thank the following people for their
   contribution to this draft:

   o  Ryan Hoffman, Telus








Rokui, et al.              Expires May 6, 2021                 [Page 22]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


10.  Informative References

   [I-D.boucadair-connectivity-provisioning-protocol]
              Boucadair, M., Jacquenet, C., Zhang, D., and P.
              Georgatsos, "Connectivity Provisioning Negotiation
              Protocol (CPNP)", draft-boucadair-connectivity-
              provisioning-protocol-15 (work in progress), December
              2017.

   [I-D.contreras-teas-slice-nbi]
              Contreras, L., Homma, S., and J. Ordonez-Lucena, "IETF
              Network Slice use cases and attributes for Northbound
              Interface of controller", draft-contreras-teas-slice-
              nbi-03 (work in progress), October 2020.

   [I-D.homma-slice-provision-models]
              Homma, S., Nishihara, H., Miyasaka, T., Galis, A., OV, V.,
              Lopez, D., Contreras, L., Ordonez-Lucena, J., Martinez-
              Julia, P., Qiang, L., Rokui, R., Ciavaglia, L., and X.
              Foy, "Network Slice Provision Models", draft-homma-slice-
              provision-models-00 (work in progress), February 2019.

   [I-D.ietf-i2rs-yang-network-topo]
              Clemm, A., Medved, J., Varga, R., Bahadur, N.,
              Ananthakrishnan, H., and X. Liu, "A Data Model for Network
              Topologies", draft-ietf-i2rs-yang-network-topo-20 (work in
              progress), December 2017.

   [I-D.ietf-teas-actn-vn-yang]
              Lee, Y., Dhody, D., Ceccarelli, D., Bryskin, I., Yoon, B.,
              Wu, Q., and P. Park, "A Yang Data Model for VN Operation",
              draft-ietf-teas-actn-vn-yang-04 (work in progress),
              February 2019.

   [I-D.king-teas-applicability-actn-slicing]
              King, D. and Y. Lee, "Applicability of Abstraction and
              Control of Traffic Engineered Networks (ACTN) to Network
              Slicing", draft-king-teas-applicability-actn-slicing-04
              (work in progress), October 2018.

   [I-D.nsdt-teas-ietf-network-slice-definition]
              Rokui, R., Homma, S., Makhijani, K., Contreras, L., and J.
              Tantsura, "Definition of IETF Network Slices", draft-nsdt-
              teas-ietf-network-slice-definition-00 (work in progress),
              October 2020.






Rokui, et al.              Expires May 6, 2021                 [Page 23]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   [I-D.nsdt-teas-ns-framework]
              Gray, E. and J. Drake, "Framework for Transport Network
              Slices", draft-nsdt-teas-ns-framework-04 (work in
              progress), July 2020.

   [I-D.qiang-coms-netslicing-information-model]
              Qiang, L., Galis, A., Geng, L.,
              kiran.makhijani@huawei.com, k., Martinez-Julia, P.,
              Flinck, H., and X. Foy, "Technology Independent
              Information Model for Network Slicing", draft-qiang-coms-
              netslicing-information-model-02 (work in progress),
              January 2018.

   [I-D.wd-teas-transport-slice-yang]
              Bo, W., Dhody, D., Han, L., and R. Rokui, "A Yang Data
              Model for Transport Slice NBI", draft-wd-teas-transport-
              slice-yang-02 (work in progress), July 2020.

   [RFC7297]  Boucadair, M., Jacquenet, C., and N. Wang, "IP
              Connectivity Provisioning Profile (CPP)", RFC 7297,
              DOI 10.17487/RFC7297, July 2014,
              <https://www.rfc-editor.org/info/rfc7297>.

   [RFC8049]  Litkowski, S., Tomotaki, L., and K. Ogaki, "YANG Data
              Model for L3VPN Service Delivery", RFC 8049,
              DOI 10.17487/RFC8049, February 2017,
              <https://www.rfc-editor.org/info/rfc8049>.

   [RFC8453]  Ceccarelli, D., Ed. and Y. Lee, Ed., "Framework for
              Abstraction and Control of TE Networks (ACTN)", RFC 8453,
              DOI 10.17487/RFC8453, August 2018,
              <https://www.rfc-editor.org/info/rfc8453>.

   [RFC8466]  Wen, B., Fioccola, G., Ed., Xie, C., and L. Jalil, "A YANG
              Data Model for Layer 2 Virtual Private Network (L2VPN)
              Service Delivery", RFC 8466, DOI 10.17487/RFC8466, October
              2018, <https://www.rfc-editor.org/info/rfc8466>.

   [TS.23.501-3GPP]
              3rd Generation Partnership Project (3GPP), "3GPP TS 23.501
              (V16.2.0): System Architecture for the 5G System (5GS);
              Stage 2 (Release 16)", September 2019,
              <http://www.3gpp.org/ftp//Specs/
              archive/23_series/23.501/23501-g20.zip>.







Rokui, et al.              Expires May 6, 2021                 [Page 24]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   [TS.28.530-3GPP]
              3rd Generation Partnership Project (3GPP), "3GPP TS 28.530
              V15.1.0 Technical Specification Group Services and System
              Aspects; Management and orchestration; Concepts, use cases
              and requirements (Release 15)", December 2018,
              <http://ftp.3gpp.org//Specs/
              archive/28_series/28.530/28530-f10.zip>.

   [TS.28.531-3GPP]
              3rd Generation Partnership Project (3GPP), "3GPP TS 28.531
              V16.2.0 Technical Specification Group Services and System
              Aspects; Management and orchestration; Provisioning;
              (Release 16)", June 2019, <http://ftp.3gpp.org//Specs/
              archive/28_series/28.531/28531-g20.zip>.

   [TS.28.540-3GPP]
              3rd Generation Partnership Project (3GPP), "3GPP TS 28.540
              V16.0.0 Technical Specification Group Services and System
              Aspects; Management and orchestration; 5G Network Resource
              Model (NRM); Stage 1 (Release 16)", June 2019,
              <https://www.3gpp.org/ftp/Specs/
              archive/28_series/28.540/28540-g00.zip>.

   [TS.28.541-3GPP]
              3rd Generation Partnership Project (3GPP), "3GPP TS 28.541
              V16.1.0 Technical Specification Group Services and System
              Aspects; Management and orchestration; 5G Network Resource
              Model (NRM); Stage 2 and stage 3 (Release 16)", June 2019,
              <http://www.3gpp.org/ftp//Specs/
              archive/28_series/28.541/28541-g10.zip>.

Authors' Addresses

   Reza Rokui
   Nokia
   Canada

   Email: reza.rokui@nokia.com


   Shunsuke Homma
   NTT
   3-9-11, Midori-cho
   Musashino-shi, Tokyo  180-8585
   Japan

   Email: shunsuke.homma.ietf@gmail.com




Rokui, et al.              Expires May 6, 2021                 [Page 25]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   Xavier de Foy
   InterDigital Inc.
   Canada

   Email: Xavier.Defoy@InterDigital.com


   Luis M. Contreras
   Telefonica
   Spain

   Email: luismiguel.contrerasmurillo@telefonica.com


   Philip Eardley
   BT
   UK

   Email: philip.eardley@bt.com


   Kiran Makhijani
   Futurewei Networks
   US

   Email: kiranm@futurewei.com


   Hannu Flinck
   Nokia
   Finland

   Email: hannu.flinck@nokia-bell-labs.com


   Rainer Schatzmayr
   Deutsche Telekom
   Germany

   Email: rainer.schatzmayr@telekom.de


   Ali Tizghadam
   TELUS Communications Inc
   Canada

   Email: ali.tizghadam@telus.com




Rokui, et al.              Expires May 6, 2021                 [Page 26]

Internet-Draft        draft-rokui-5G-network-slice         November 2020


   Christopher Janz
   Huawei Canada
   Canada

   Email: christopher.janz@huawei.com


   Henry Yu
   Huawei Canada
   Canada

   Email: henry.yu1@huawei.com







































Rokui, et al.              Expires May 6, 2021                 [Page 27]