IP Performance Metrics WG (ippm) Tuesday, November 19 at 17:30 - 20:00 ==================================== The meeting was moderated by the working group chairs, Matt Zekauskas and Merike Kaeo. Dave Plonka and Al Morton took notes, which were edited into these minutes by the chairs. AGENDA: 1. Agenda bashing 2. Announcements 3. Discussion on Packet Reordering 4. IPPM Reporting MIB and Metrics Registry discussion 5. Discussion on One-way Active Measurements Protocol 6. One-Way Metric Applicability Statement 7. Potential work: "IP Measurement Protocol" (IPMP) 8. Potential work: IPPM Spatial Metrics Measurement 9. Available Bandwidth Measurement in IP Networks IETF home page: http://www.ietf.org/html.charters/ippm-charter.html IPPM home page: http://www.advanced.org/IPPM/ 2. Announcements Mark Allman introduced the IRTF Internet Measurement RG. In general IMRG is a forum for measurement research that isn't yet ready for IETF. It intends to foster interaction amongst communities: IETF, operators, researchers. The IMRG is an *open* IRTF group, anyone can join the list: http://imrg.grc.nasa.gov/imrg/ (linked from irtf.org). David Martin asked if IMRG looking at control aspects - what kinds of measurement? Mark responded that it's open; lots of kinds: passive, active, lots of options to investigate. Al Morton presented on IPPM and ITU-T performance status. Since Will Leland's comparison in March, 1999, some terms have changed and concepts and definitions have gotten closer. One new definition is IP severe loss block ratio, prompted by a desire to measure the effect of re-routing. The one activity where the two groups continue to differ is in the area of specifying objectives for performance; the ITU "Numerical performance objectives" are beyond IPPM's scope. Matt asked how Al participates in ITU-T meetings - in essence why he is qualified to make such a report. Al responded that he attends ITU-T meetings, is the designated editor for recommendations in Study Group 13 where his company has interest (such as Y.1540 and Y.1541). He also contributes to the work of Study Group 12. Al has considered the possibility of an Informational RFC comparing ITU and IPPM terminology but suggests that that is becoming less and less necessary. An audience member (Hidega Tiku, from France Telecom) believed that a broader mapping is possible than just to the Study Group 13 documents (Y.1540 and Y.1541), she believes there is relevance to 30 ITU groups. Al believed that the closest parallel was SG 13, and would like to work with her to understand. Another audience member (Chuck Dvorak, vice chair of SG 12, the lead study group in all ITU-T QoS activities) confirmed that IPPM's closest working counterpart is the SG 13 work that Al described, other groups may have interest in IPPM, but do not define IP metrics. Matt invited them to work together and bring conclusions to the mailing list. 3. Discussion on Packet Reordering -- Al Morton http://www.ietf.org/internet-drafts/draft-ietf-ippm-reordering-01.txt Al reported on status of current draft and presented some examples. He expressed that more input is desired as to whether the examples are sufficient. Major issues include integrating the definition of n-reordering (clarification, description of how it fits in with non-reversing sequence), specific advice concerning duplicates, adding a metric of distance between reordering events, and unifying notation. Stanislav Shalunov pointed out that if you want to consider all duplicate packets as not out of order, this is a reasonable definition, but the implementation would require O(N) space to keep track of all the packet numbers. Thus, the metric is "fine but expensive". Al stated that we could just allow all duplicate packets to be declared out of order and consider duplicates at the end of the test. Matt Z. asked for a clarification on Al's use of "clarification" with respect to the n-reordering metric. Was he looking to change the definition? Al responded that he wasn't quite sure yet, but that the goal was to understand when particular metrics were applicable (and possibly add some caveats, but didn't want to resort to that, yet); n-reordering may better related to understanding reordering with espect to TCP, while other sample metrics in section 5, such as Late Time, may be better related to understanding reordering with respect to sizing buffers for streaming media. Jerry Perser (another author) echoed this, and a general difficulty understanding the n-reordering results. Matt Mathis noted that n=3 for reordering isn't a constant for TCP's; it's a parameter to at least one stack. He would also like a better understanding how Web100's reordering statistic is related to n-reordering, and how the metric is related to SACK blocks. 4. IPPM Reporting MIB and Metrics Registry discussion -- Emile Stephan http://www.ietf.org/internet-drafts/draft-ietf-ippm-reporting-mib-01.txt http://www.ietf.org/internet-drafts/draft-ietf-ippm-metrics-registry-01.txt Jesse Jewitt discussed changes made from IPPM reporting MIB from version 00 to 01. Henk Uijterwaal started a discussion around the use of the word "stratum" in the document - in particular, if it is the same as the NTP stratum level. It is apparently similar, but not exactly the same as NTP's usage (the exact similarities and differences were not clear from the discussion). In addition, Henk worried that any notion of stratum might not be sufficient, a notion of accuracy was also important. Matt stated that since there was confusion, the document should contain a clarification. Henk agreed to take the discussion to the mailing list. A complete list of enumerated open issues is contained in the slides. The authors are working on a prototype. Matt asked if there was anything learned from implementing the prototype. Jesse responded yes; Emile would provide input before or at the next meeting. Emile then gave update on registry. Emile is looking for some procedural guidance to obtaining a MIB root value (how one is obtained from IANA). There was a discussion about MIB OID lengths; the length would vary depending on where it was rooted. Emile was wanted to keep the lengths reasonable for comparison. Emile is also looking for guidance on suggesting procedures to specify how new draft metrics add a section to create a new registry entry. Finally, Bob Cole noted that there are four "types" of metrics mentioned in the registry: singleton, sample, statistical, and "ambiguous statistical" like percentile. He wondered if it was appropriate to have entries for statistics that required parameters. RMON points to just singleton metrics. He felt that the 'ambiguous statistical' metrics that require parameters might not be appropriate for the registry, or that maybe the registry should be split to avoid an explosion. It was noted that all the metrics require parameters; Bob would take the discussion to the mailing list to talk about details. 5. Discussion on One-Way Active Measurements Protocol -- Stanislav Shalunov http://www.ietf.org/internet-drafts/draft-ietf-ippm-owdp-05.txt Stanislav Shalunov gave update on OWAMP. Gave list of updates from version 04 to 05 that were based on implementation experience. Matt asked for a clarification on the units of accuracy; it is a scaled power of two, with a huge range but not much resolution. Al Morton noted one spot where the periodic sampling now included in the draft had not been added. The two significant discussion points were the error in timestamps when packets are lost and reported errors for the start session command. The protocol needs a way to report the error in the sending timestamps when the test packets are lost (or at least account for the error). It currently reports a time sent that isn't derived from explicit packet data, and the send error isn't specified. Al suggested setting the send time error to zero as a distinguished value; Stanislav agreed it should be set to something. Second, the reason a session start command is rejected should be stated (or enumerated). There will be at least a number that can be correlated to a specific reason. 6. One-Way Metric Applicability Statement -- Henk Uijterwaal http://www.ietf.org/internet-drafts/draft-ietf-ippm-owmetric-as-01.txt Henk gave update on the applicability statement status. There hasn't been a lot of input, and we really need input from operators. Merike queried the room -- about half the people in the room were operators. However, few people had read draft. Folks were asked to read document and send comments to mailing list. Merike mentioned that Henk gave a pitch at the last NANOG, and many there thought that this document would be useful. 7. Potential work: another iteration on "IP Measurement Protocol" (IPMP) -- Jon Bennett http://www.ietf.org/internet-drafts/draft-bennett-ippm-ipmp-00.txt Jon Bennett gave presentation on his version of "IPMP" - an IP measurement protocol, first presented last year by Tony McGregor. He talked about the motivation, and then gave an overview of changes that were modifications to make the protocol more palatable to hardware implementers (in his opinion) and add some functionality. IPMP is explicitly designed to be easily recognizable by intermediate systems, so that they can add measurement data as the packet is routed. After an initial stumble, Jon is working with the authors of the McGregor draft to create a merged document. David Martin wondered if this is essentially a DDoS engine, since it is not required to have authentication. Jon noted that there are a number of security/authentication options and other considerations to prevent it from being used as a DoS; in addition it doesn't make any new attacks available. It does allow you to redirect, but not hide the original source IP address. Matt queried to the room to see if there was more interest than was shown on the mailing list since Tony presented last year. Only a couple people admitted to reading the draft. Merike asked if there were any vendors willing to implement it, or if providers in the room were interested. Jon noted that Motorola was looking towards implementing it, which could get it in ~200,000 cable modems. Nevil Brownlee noted that Tony McGregor does have a working implementation (in end hosts), and it's deployed in the NLANR AMP infrastructure (~100 hosts); Nevil also noted that the focus on turnaround time and minimal IPMP processing was to have IPMP be a good measure of the forwarding of "real" traffic along a forwarding path. A service provider indicated that it would be nice to have, but he was concerned that IPMP is awfully complex to implement well; if a vendor can't decrement TTL properly, how would they get this right? 8. Potential work: IPPM spatial metrics measurement -- Emile Stephan http://www.ietf.org/internet-drafts/draft-stephan-ippm-spatial-metrics-00.txt Emile presented a draft on "spatial metrics" -- obtaining measurement data from intermediate points to help break apart path measurement data to component path segments. The advantage of this document would be understanding and standardizing the accuracy and uncertainty in the results. Ruediger Geib noted that this assumes that you understand packet routing a-priori, and something like load-balanced induced changes in routing would cause problems. Emile thought it could be found by combining passive measurements with the active traffic. Ruediger also wondered if the metric involved collecting many individual results, perhaps it involved more of a database and measurement system, and wasn't an individual result itself. 9. Potential work: Available Bandwidth Measurement in IP Networks -- Tricha Anjali http://www.ietf.org/internet-drafts/draft-anjali-ippm-avail-band-measurement-00.txt Tricha Anjali presented work on available bandwidth measurements. She started with explanations of the various definitions, and then talked about her technique, which involves the use of TCP, but not allowing it to reach steady state. Each measurement is a sample; the samples are massaged to appear uniform, and then a linear filter is used to make predictions. Additional data (such as SNMP measurements) can be used to provide additional accuracy. The details are in the slides and draft. Mark Allman noted that a RFC defining all the terminology used in bandwidth measurement would be useful, and he has attempted to help write one; other authors would be welcome. Mark asked the chairs if the notion of predicting the "future bandwidth" in WG scope? Matt responded that his feeling was that it was not, and it was more of a research topic. Mark Allman also questioned letting TCP go until a loss occurs, since the size of queue skews the bandwidth metric because of the way TCP bursts. Tricha would investigate. Matt asked if there are there simulations or actual measurements showing how accurate this method is; Tricha responded not yet. Nevil Brownlee comment about method: if you assume there is nothing on the link, and you use one TCP stream to test, I don't see how you can add one more TCP stream to an unknown number, and understand the affect on the other traffic. Matt Mathis noted it was a Heisenberg problem: you don't know how much the test traffic causes the other traffic to slow down. This seems to be a fundamental problem with this class of method. Mark Allman noted the measurement indeed could affect other traffic since it goes until it induces loss.