DetNet R. Liu Internet Draft Z. Cheng Intended status: Informational H3C Expires: November 24, 2024 May 24, 2024 Resilient Cycle Queuing and Forwarding draft-liu-detnet-rcqf-00.txt Abstract The document proposes a solution called Resilient Cycle Queuing and Forwarding (RCQF) for DetNet extremely low data loss rates, bounded latency and minimal jitter. RCQF is a technological advancement built upon Cycle Queuing and Forwarding (CQF), designed to provide a capability for the delivery of end-to-end DetNet flows. RCQF extends the capabilities of CQF and makes it suitable for large scaling networks. The elasticity of RCQF includes adaptive bonded latency, minimal jitter, high bandwidth, large packet size, interface speed, and only requires frequency synchronization. This document aims to introduce the concept of RCQF and its potential in providing resilient and adaptable solutions for Deterministic Networks. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may not be modified, and derivative works of it may not be created, and it may not be published except as an Internet-Draft. This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may not be modified, and derivative works of it may not be created, except to publish it as an RFC and to translate it into languages other than English. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified Liu, et al Expires November 24, 2024 [Page 1] Internet-Draft RCQF May 2024 outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet-Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on November 24, 2024. Copyright Notice Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Table of Contents Liu, et al. Expires November 24, 2024 [Page 2] Internet-Draft RCQF May 2024 1. Introduction...................................................3 1.1. Requirements Language.....................................4 2. Problem Statement..............................................4 3. Network Model..................................................4 3.1. Explicit Path.............................................4 3.2. Queue Mapping.............................................5 3.3. Periodic Forwarding.......................................5 3.4. Elastic Scheduling........................................5 3.5. Edge-to-Edge Collaboration................................5 4. Scheduling cycle...............................................6 5. Dynamic Queue Sharing..........................................7 6. OAM............................................................9 7. Security Considerations.......................................10 8. IANA Considerations...........................................10 9. References....................................................10 9.1. Normative References.....................................10 9.2. Informative References...................................11 10. Acknowledgments..............................................11 1. Introduction As an extension of CQF technology, RCQF has the following specific features: o Queue Flexibility: RCQF allows for flexible definition of queue parameters such as queue quantity, queue length, and cycle width, enabling adjustment of queue parameters according to requirements. o Dynamic Load Sharing: In the event of uneven packet distribution within the queue, dynamic shaping and load-sharing operations enable smooth packet transmission to prevent congestion within the queue. o Latency Compensation: During the process of data forwarding, the latency and jitter of data transmission constantly fluctuate. By utilizing compensation mechanism, it is possible to maintain end- to-end packet latency and jitter within a baseline value, thereby enabling controlled upper limits for latency and jitter. o Loose Coupling: RCQF only requires frequency synchronization without the need for phase synchronization. As a result, it reduces the system's dependency on clock synchronization technology and subsequently lowers the difficulty of deterministic coordination. Liu, et al. Expires November 24, 2024 [Page 3] Internet-Draft RCQF May 2024 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 2. Problem Statement Deterministic Networking presents several challenges in terms of planning and scheduling. Firstly, the diversity of traffic models, including variations in traffic burstiness, packet sizes, quantities, and inter-packet intervals, poses significant challenges for planning and scheduling. Secondly, there are challenges associated with forwarding multiple services on a single device due to variations in device types and performance, such as differences in crystal oscillator accuracy, port rates, buffer resources, and system performance, which make it challenging to provide timely and accurate forwarding for different services. Lastly, achieving efficient collaboration across the entire network is also a challenge, given the differences in nodes throughout the network, including variations in clock synchronization capabilities, interconnection, and adaptation, all of which pose significant challenges for achieving a unified and efficient end-to-end deterministic transmission system. While the periodic forwarding mechanism of CQF can meet the low- latency and low-jitter requirements of deterministic services, it is too complex in terms of queue control and configuration. CSQF requires the prior specification of mapped queues. In the event of topological or traffic changes resulting in burst traffic or an overload of packets, the pre-specified queues may overflow. There is a lack of load balancing between queues, which makes it inflexible and unsuitable for upper-layer services and network planning. 3. Network Model RCQF guarantees end-to-end deterministic latency and jitter through the following mechanisms, while also possessing elasticity and dynamic adjustment capabilities to optimize overall system performance. 3.1. Explicit Path By collecting network topology and delay information, RCQF computes end-to-end data forwarding paths that satisfy different application latency requirements. Liu, et al. Expires November 24, 2024 [Page 4] Internet-Draft RCQF May 2024 3.2. Queue Mapping By embedding forwarding node, interface, queue, and other information in the SRv6 SID, RCQF establishes a mapping relationship between deterministic service flows and queues to guide deterministic traffic forwarding through specific queues. 3.3. Periodic Forwarding Network nodes perform queue periodic forwarding according to specified interfaces, queues, and periods based on relevant information in the SRv6 SID. 3.4. Elastic Scheduling RCQF achieves elasticity and dynamic adjustment through dynamic load balancing between queues, delay compensation, and other operations. 3.5. Edge-to-Edge Collaboration Through control plane coordination, the controller facilitates client-side and network-side collaboration to dynamically adjust forwarding paths, queues, windows, and other resources in the network, thereby improving the efficiency of network data transmission. Liu, et al. Expires November 24, 2024 [Page 5] Internet-Draft RCQF May 2024 4. Scheduling cycle Quences +---------------+ Sector number | Quece0 | +---------------+ +--+--+--+--+--+--+--+--+ +---------------+ | | | | | | | | | | Quece1 | |Q0|Q1|Q2|Q3|Q4|Q5|Q6|Q7| +---------------+ | | | | | | | | | +---------------+ +--+--+--+--+--+--+--+--+ | Quece2 | +---------------+ +---------------+ A Cycle | Quece3 | +---------------+ ---- +---------------+ ///- -\\\ | Quece4 | / \ +---------------+ / \ +---------------+ | | | Quece5 | | --------| +---------------+ | | +---------------+ | | | Quece6 | \ / +---------------+ \ / +---------------+ \\\- -/// | Quece7 | ---- +---------------+ Figure 1 Scheduling Cycle As shown in the above figure, one circle represents one scheduling cycle, and one sector represents one time slot which can scheduling the queue. The entire scheduling process can be broken down into the following parameters: 1. Plan multiple queues based on the characteristics of the flows, such as the quantity and bandwidth of flows, i.e., pre-define the number of queues. 2. Set the queue length based on the attributes of the flows (e.g., flow size) and available resources (e.g., buffer resources). 3. Set the sectors of a cycle based on the number of queues; typically, the number of queues and the sector of a cycle are equivalent. Liu, et al. Expires November 24, 2024 [Page 6] Internet-Draft RCQF May 2024 4. Define the time of the sector based on the business requirements (e.g., end-to-end jitter accuracy of less than 10 microseconds) and hardware conditions (e.g., crystal oscillator and scheduling accuracy). This refers to the time duration occupied by each sector (time slot), such as 10 microseconds. If set to 10 microseconds, it means that deterministic packets can be sent within 10 microseconds. For instance, for a 10G interface, approximately 12,000 bytes of packets can be sent within 10 microseconds. Once these parameters are set, a deterministic flow will be mapped to one or multiple queues. These queues are associated with sectors, with each sector being a part of the entire scheduling cycle (i.e., a time slot). The scheduling cycle is continuously rotating, and the change of cycles is alternated. For example, if the scheduling cycle is divided into 8 parts, then there will be 8 sectors. If the width of a single sector is 10 microseconds, then one complete rotation of the scheduling cycle will take 80 microseconds. In other words, within 1 second, the scheduling cycle can complete 12.5 rotations. From the operational mechanism described above, it can be concluded that by adjusting parameters such as "queue length," "queue quantity," "sector quantity," and "sector width," the scheduling strategy can be changed to accommodate different application requirements. For instance, if the end-to-end jitter requirement of an application is low, the time of a single sector can be longer, enabling more packets to be sent within a single sector. This approach has two advantages: 1. The system's forwarding efficiency increases because the frequency of cycle switching within a unit of time decreases, thereby reducing the overhead caused by cycle switching; 2. More flows and packets can be aggregated, as a longer sector width allows for the transmission of more packets within a single sector, thus enabling more flows and packets to be aggregated for forwarding within a single sector. Furthermore, by increasing the number of queues and sectors, different flows can be mapped to different queues to better support a greater number of flows and provide dedicated queues for specific flows. 5. Dynamic Queue Sharing Packets are received from the incoming port, processed internally, and then forwarded from the outgoing port. To achieve deterministic flow, the process is going to involve: 1. entering the RCQF Queue, 2. mapping the RCQF Queue to the corresponding sector, and 3. the Liu, et al. Expires November 24, 2024 [Page 7] Internet-Draft RCQF May 2024 scheduler cyclically schedules (scheduling cycle) according to deterministic scheduling. In this process, some queues may be idle while others are busy, and this mapping to the cycle schedule results in some sectors being idle while others are busy. To address the uneven scheduling of cycles, this patent introduces a method of mutual sharing as follows: 1. Set a "Sector threshold," for example, the threshold value can be set as 50% of the maximum capacity of a single sector for forwarding. 2. Determination of exceeding the threshold: When it is determined that the threshold is exceeded, the sector is traced back to the queue, and based on the occupancy of the queue, it is determined whether the threshold is exceeded, and the corresponding queue is marked if the threshold is exceeded. 3. Mutual sharing of scheduling between sectors: Based on the above exceeding criteria, mutual sharing of scheduling between adjacent sectors is performed. 4. For the sharing in the third step, it can be moved forward. While forwarding Sector 0, if it does not exceed the threshold, the status of Queue 1 is evaluated. If Queue 1 exceeds the threshold, and there is remaining time after forwarding the packets from Queue 0, the packets are taken from Queue 1 and sent externally. This helps alleviate the burden on Sector 1. 5. In addition to forward and backward shifting, the stepping can also be adjusted. If the step size is set to 2, it can be shifted two sectors forward or backward. As shown in the diagram, if it is set to shift two sectors forward, when Sectors 4 is scheduled, it can take into account the threshold mark for Queue 6 (6 = 4 + 2). Since Queue 6 exceeds the threshold and has a large number of packets, after scheduling the packets from Queue 4, the remaining time can be used to schedule the packets from Queue 5, thereby reducing the pressure on Sector 5 and allowing more Queues 6 packets to be scheduled later in Sector 5. 6. By adjusting and optimizing the above methods such as thresholds, sharing, forward and backward shifting, and step size, it is possible to achieve queue shaping, sharing between sectors, and smoother output of packets, thereby improving overall forwarding efficiency and resource utilization. Liu, et al. Expires November 24, 2024 [Page 8] Internet-Draft RCQF May 2024 RCQF first proposed the Cycle-level sharing scheme, which has higher forwarding efficiency and resource utilization. It is more friendly to control plane resource planning, topology, and traffic changes. Additionally, RCQF's sharing processing occurs during the cycle scheduling period, rather than during the queue entry period. 6. OAM The core function of Operations, Administration, and Maintenance (OAM) is to detect time slot deviations, where forwarding nodes map time slot deviations to the egress interface queues of service packets, thereby guiding packet forwarding. Deterministic networking technology uses maximum time slot deviations to guide forwarding, ultimately achieving deterministic latency jitter. +----------------------------------+ | | +-----+ SDN Controller +-------+ | | | | | +----------+-----------------+-----+ | | | | | | | | | | | | | t1=Qm2-Qm1 t2=Qn-Qm2 t3=Qx-Qn| t4=Qy-Qx | | | | +---+----+ +----+---+ +----+---+ +---+----+ Qm1 | | Qm2 | |Qn | |Qx | | |Qy + PE1 +------+ P1 +--------+ P2 +-----+ PE2 + | | | | | | | | +--------+ +--------+ +--------+ +--------+ Figure 2 DetNet OAM As shown in Figure, the workflow of OAM detecting the maximum time slot deviation between the source node and adjacent nodes is as follows: The ingress node of the SRv6 path periodically simulates the generation of OAM detection packets for traffic, with 10 detections performed per period. The detection packets are forwarded along the Segment List. The ingress node on the Segment List detects the time slot deviations between the egress and ingress interface queues of this device for the detection packets (t1) and other nodes calculate the time slot deviations for egressing the detection packets from this node to the Liu, et al. Expires November 24, 2024 [Page 9] Internet-Draft RCQF May 2024 upstream node (t2, t3, and t4) and upload the maximum, minimum, and average time slot deviation during the period to the controller. The controller selects the maximum time slot deviation within the detection period, generates a list of time slot deviations (t1, t2, t3, t4), and sends the list of time slot deviations to the ingress node. The ingress node encapsulates the list of time slot deviations in the SRv6 packet header to guide packet forwarding. [I-D.guo-detnet-jitter-reduction-mechanism] describes a compensation mechanism designed to reduce end-to-end jitter and meet the requirements of applications with tightly bounded jitter. The compensation mechanism can be a part of RCQF and could achieve on time. 7. Security Considerations TBD 8. IANA Considerations This document has no IANA actions. 9. References 9.1. Normative References [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", RFC 8655, DOI 10.17487/RFC8655, October 2019, . [RFC8938] Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S. Bryant, "Deterministic Networking (DetNet) Data Plane Framework", RFC 8938, DOI 10.17487/RFC8938, November 2020, . Liu, et al. Expires November 24, 2024 [Page 10] Internet-Draft RCQF May 2024 9.2. Informative References [I-D.ietf-detnet-scaling-requirements] Liu, P., Li, Y., Eckert, T. T., Xiong, Q., Ryoo, J., Yin, Z., and Geng, X., "Requirements for Scaling Deterministic Networks", Work in Progress, Internet-Draft, draft-ietf-detnet-scaling-requirements-05, 20 November 2023, . [I-D.chen-detnet-sr-based-bounded-latency] Chen, M., Geng, X., Li, Z., Joung, J., and J. Ryoo, "Segment Routing (SR) Based Bounded Latency", Work in Progress, Internet-Draft, draft-chen- detnet-sr-based-bounded-latency-03, 7 July 2023, . [I-D.guo-detnet-jitter-reduction-mechanism] Guo, D., Xu, S., "Jitter Reduction Mechanism for DetNet",Work in Progress, Internet- Draft, draft-guo-detnet-jitter-reduction-mechanism-01, 23 October 2023, . [I-D.guo-detnet-vpfc-planning] D. Guo, G. Wen, K. Yao, Q. Xiong, G. Peng, "Deterministic Networking (DetNet) Controller Plane - VPFC Planning Information Model Based on VPFP in Scaling Deterministic Networks", Work in Progress, Internet-Draft, draft-guo-detnet-vpfc-planning-03, 5 January 2024, . 10. Acknowledgments TBD Liu, et al. Expires November 24, 2024 [Page 11] Internet-Draft RCQF May 2024 Authors' Addresses Rubing Liu New H3C Technologies Co., Ltd Hangzhou, China Email: liurubing@h3c.com Zuopin Cheng New H3C Technologies Co., Ltd Hangzhou, China Email: czp@h3c.com Liu, et al. Expires November 24, 2024 [Page 12]