Subject: Planned experiment: A new mailing list for last-call discussions

From: Barry Leiba <[email-masked]>
Date: Thu, 12 Sep 2019 12:14:01 -0400

As we discussed in the plenary session at IETF 105 in Montréal, some
community members have suggested moving document last-call discussions
onto a dedicated "last-call" mailing list, and off of the general
<[email-masked]> list.  The latter is a high-volume list with a lot of
varied discussion, and some think that it would be useful to separate
the general discussion from the last-call discussion, to allow people
to choose which discussions (or both) to follow.  In the IETF 105
plenary, support was expressed for that separation.

The IESG agrees, and wants to try an experiment to that end.  We
propose to create <[email-masked]> and to direct last-call
comments and discussions there (the last-call announcements would
still go to <[email-masked]>, with "reply-to" set to the new
list).  That list would be monitored by volunteers recruited by the
IETF Chair, and digressions would be nudged back to <[email-masked]>,
while we would ask people having last-call discussions on this list to
please move them to the new list.  We would get the tools team
involved so that the distribution lists for directorate and
review-team reviews would be updated appropriately.

Our plan is to create the new list and pre-subscribe everyone who is
subscribed to <[email-masked]> at that time.  Of course, anyone could
unsubscribe to either or both lists immediately or later, but we think
that doing it this way would minimize the likelihood that people would
miss important stuff because of the move, and folks can choose what
they prefer from there.

After six months, we would do an initial evaluation, including getting
feedback from the community, to see how the experiment is working.  If
it seems worth continuing we would do so, and at a point that the
community decides that the experiment is a success (should it so
decide), we would start an update to BCP 45 to formally move the
location for last-call discussions, and we would update the 2007 IESG
Statement on Last Call Guidance.

We invite comments, here, on this plan, by the end of September. As I
say above, we've heard support from the community for the general
idea, and we'd like to make sure this direction is what the community
wants.

Barry, for the IESG

---

From: "Andrew G. Malis" <[email-masked]>
Date: Thu, 12 Sep 2019 13:00:50 -0400

Barry,

Sounds good, let's give it a shot.

Cheers,
Andy

---

From: Eric Rescorla <[email-masked]>
Date: Thu, 12 Sep 2019 10:06:30 -0700

+1. This seems like a great idea.

-Ekr

---

From: Ross Finlayson <[email-masked]>
Date: Thu, 12 Sep 2019 10:10:57 -0700

> On Sep 12, 2019, at 9:14 AM, Barry Leiba <[email-masked]> wrote:
[…]
> subscribed to <[email-masked]> at that time.

Wouldn’t it be better to have “[email-masked]” itself (and not its current subscribers) on the new mailing list (so that anyone who subscribes to “ietf” later will automatically get last-call announcements as well)?

	Ross.

---

From: Barry Leiba <[email-masked]>
Date: Thu, 12 Sep 2019 13:26:55 -0400

Hi, Ross.

> Wouldn’t it be better to have “[email-masked]” itself (and not its current subscribers) on
> the new mailing list (so that anyone who subscribes to “ietf” later will automatically
> get last-call announcements as well)?

First, remember that (as noted in the message) the last-call
*announcements* go to ietf-announce, with reply-to set to
<[email-masked]>; they already do not get posted to <[email-masked]>.
What would change is that the reply-to would become
<[email-masked]>.

Second, one of the points of this is to allow people to choose to
follow the last-call discussions and/or the IETF general discussion,
and to make separate choices for those -- some will want one, some the
other, some both.  If we were to subscribe one list to the other, it
wouldn't be possible to choose which to subscribe to independently.

Barry

---

From: Richard Barnes <[email-masked]>
Date: Thu, 12 Sep 2019 10:31:48 -0700

+1

I have no comments on this.  You even answered my usual question about
"experiments", namely what the success criteria are.  Ship it.

--Richard

---

From: Tim Chown <[email-masked]>
Date: Thu, 12 Sep 2019 18:08:03 +0000

Hi,

I think this is an excellent idea.

I know that at least the OPS-DIR review tool posts to [email-masked]<mailto:[email-masked]> by default, would such reviews go to last-call@ too?  They’re not always last call stage reviews, but most probably are.

Tim

On 12 Sep 2019, at 17:14, Barry Leiba <[email-masked]<mailto:[email-masked]>> wrote:

<[email-masked]<mailto:[email-masked]>> list.  The latter is a high-volume list with a lot of

The IESG agrees, and wants to try an experiment to that end.  We
propose to create <[email-masked]<mailto:[email-masked]>> and to direct last-call
comments and discussions there (the last-call announcements would
still go to <[email-masked]<mailto:[email-masked]>>, with "reply-to" set to the new
list).  That list would be monitored by volunteers recruited by the
IETF Chair, and digressions would be nudged back to <[email-masked]<mailto:[email-masked]>>,
while we would ask people having last-call discussions on this list to
please move them to the new list.  We would get the tools team
involved so that the distribution lists for directorate and
review-team reviews would be updated appropriately.

subscribed to <[email-masked]<mailto:[email-masked]>> at that time.  Of course, anyone could

Barry, for the IESG

---

From: Barry Leiba <[email-masked]>
Date: Thu, 12 Sep 2019 15:24:46 -0400

Hi, Tim,

> I know that at least the OPS-DIR review tool posts to [email-masked] by default, would
> such reviews go to last-call@ too?  They’re not always last call stage reviews, but
> most probably are.

Yes: the idea is that all the directorate and review-team reviews
would go there as well, all treated as last-call reviews even the few
that have been requested early.

Barry

---

From: Tim Wattenberg <[email-masked]>
Date: Thu, 12 Sep 2019 21:33:26 +0200

+1, let’s do this. Tim

> Am 12.09.2019 um 18:14 schrieb Barry Leiba <[email-masked]>:
>

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 13 Sep 2019 08:43:08 +1200

Three comments:

1. Slightly to my surprise, this is fully consistent with RFC 2026, which
says:
   This "Last-Call" notification shall be
   via electronic mail to the IETF Announce mailing list.  Comments on a
   Last-Call shall be accepted from anyone, and should be sent as
   directed in the Last-Call announcement.
I think there may be other places where the [email-masked] list is documented
as the venue for last calls, but for the experiment, we're fine.

2. Are we sure? Yes, this will reduce traffic on the list, but one view is
that it's the most important traffic of all.

3. Personally, I simply don't care. I will adjust my filters to direct the
displaced traffic to the same inbox as [email-masked].

Regards
   Brian

On 13-Sep-19 04:14, Barry Leiba wrote:
> .

---

From: Barry Leiba <[email-masked]>
Date: Thu, 12 Sep 2019 17:15:36 -0400

Hi, Brian,

> as the venue for last calls, but for the experiment, we're fine.

I agree that I think we're fine as an experiment.  What will have to
change eventually, if we keep it, is BCP 45 (RFC 3005), which is the
charter for the IETF discussion list.  It explicitly says that
last-call discussions are appropriate material.  If the experiment is
a success, we'd want to update that to change where last-call
discussions go.

> that it's the most important traffic of all.

We're not sure: hence, the experiment.  But the point isn't to
*reduce* traffic, but to separate it, so people can make reasonable
choices about what discussions they want to follow.  I, as you, will
certainly follow both.  Others will make different choices.

Barry

---

From: Randy Bush <[email-masked]>
Date: Thu, 12 Sep 2019 14:36:33 -0700

i think that the bikeshed should be magenta

randy

---

From: "Christopher Wood" <[email-masked]>
Date: Thu, 12 Sep 2019 15:13:38 -0700

Heavy +1.

Best,
Chris

On Thu, Sep 12, 2019, at 12:33 PM, Tim Wattenberg wrote:
> Attachments:
> * smime.p7s

---

From: Tommy Pauly <[email-masked]>
Date: Thu, 12 Sep 2019 15:26:20 -0700

+1, this will be a useful experiment!

—Tommy

---

From: John C Klensin <[email-masked]>
Date: Thu, 12 Sep 2019 22:13:30 -0400

Barry, 

I want to suggest a slightly dissenting view, but only slightly.
I have two quite separate concerns

(1) I believe that part of what has prompted the discussion of a
separate list and, I believe, the moves that led in the
direction of this experiment, is that the IETF list has, in the
several months, been more active and perhaps noisier than at any
time since I first started watching the IETF ("perhaps" because
one person's noise may be another person's really interesting
discussion).  That appears to me to have been stimulated by a
number of largely non-technical issues arising within a short
time of each other and stimulating very active discussions and
spin-offs of those discussions.  The RSE situation and Heather's
announcement that she was making and earlier-than-expected exit,
some actions by the SAAs that various people found concerning
including questions of whether the SAA function was being
weaponized to suppress unpopular questions and discussions,
questions that have been discussed as "tone" but that ultimately
may be about how we maximize form and content at the same time,
and issues about whether people who do not attend most f2f
meetings are really full members of the community (with the
recall topic in particular), come to mind immediately, but I
think the list is probably longer.   In normal times (whatever
that means), we might see one of those issues and the subsequent
long threads and branches once every six months or a year and
then they die down for a while.  

I hope the situation of the last months, one that has given many
of us a choice between trying to follow the discussion threads
and getting any substantive IETF work done at all, is not our
"new normal" and will settle down soon.   I assume the IESG
shares that wish since the traffic level has to be hitting most
or all of you as hard or harder than those of us who are just
participants and who have much more flexibility about what to
ignore.  So I would like to see the experiment initiated and run
in "normal" times.  If you start it while the IETF list is this
active and then it settles down, it will be impossible to
evaluate whether the experiment had any useful effect.

So, I think the experiment is worthwhile but that, if we expect
to learn anything from it, I think it would be much wiser to
initiate it only after the IETF list settles down (or the IESG
makes an informed decision that we are in a new steady state and
"settled down" is not going to happen).  If things settle down
before the 1st of September (and I devoutly hope they will)
then, by all means go for it (especially if you don't think the
concern below is a problem) but, if not, I suggest thinking very
carefully about start dates.

(2) As I have talked with people in other standards bodies,
people in procurement roles, regulators, and so on over the last
many years, I've become convinced that the IETF's main source of
strength and credibility has been effective cross-area review
(as well as interoperability testing in years gone by).  It is
not that we are so much smarter than everyone else, it is that
we've been able to look at a specification that is proposed for
the standards track from many different perspectives and do so
in depth.  As you know, I'm concerned that the quality of those
reviews has been deteriorating as am increasing fraction of the
reviews outside the WG or Area of the document are by people
with insufficient in-depth knowledge to understand the subject
matter and comment on it in detail.  I may be atypical, but
there have been many times I've ignored a Last Call announcement
of a document that is within my area of competence but not of
special interest, only to be lured in later by a Last Call
discussion on the IETF list.   Now, if everyone who is
interested in technical reviews of LC documents subscribes to
the new list and stays on it, there will be no difference.  But
I can easily imagine some number of IETF participants
subscribing to the new list only when a Last Call of interest
shows up on IETF-Announce and then unsubscribing when that Last
Call is over.  If the number of such people, perhaps weighted by
their depth and breath of technical knowledge, is trivial, then
there is no concern.  But, if it isn't and that [further]
weakens our ability to obtain in-depth cross-area reviews, it
won't be doing the IETF and its future any favors.   If the
experiment is run, I'm not sure that effect can be measured, but
the possibility may be worth a little thoughts.

best,
   john

--On Thursday, September 12, 2019 12:14 -0400 Barry Leiba
<[email-masked]> wrote:

> As we discussed in the plenary session at IETF 105 in
> Montréal, some community members have suggested moving
> document last-call discussions onto a dedicated "last-call"
> mailing list, and off of the general <[email-masked]> list.
> The latter is a high-volume list with a lot of varied
> discussion, and some think that it would be useful to separate
> the general discussion from the last-call discussion, to allow
> people to choose which discussions (or both) to follow.  In
> the IETF 105 plenary, support was expressed for that
> separation.
> 
> The IESG agrees, and wants to try an experiment to that end.
> We propose to create <[email-masked]> and to direct
> last-call comments and discussions there (the last-call
> announcements would still go to <[email-masked]>, with
> "reply-to" set to the new list).  That list would be monitored
> by volunteers recruited by the IETF Chair, and digressions
> would be nudged back to <[email-masked]>, while we would ask
> people having last-call discussions on this list to please
> move them to the new list.  We would get the tools team
> Our plan is to create the new list and pre-subscribe everyone
> who is subscribed to <[email-masked]> at that time.  Of course,
> anyone could unsubscribe to either or both lists immediately
> or later, but we think that doing it this way would minimize
> the likelihood that people would miss important stuff because
> of the move, and folks can choose what they prefer from there.
> 
> After six months, we would do an initial evaluation, including
> getting feedback from the community, to see how the experiment
> is working.  If it seems worth continuing we would do so, and
> at a point that the community decides that the experiment is a
> success (should it so decide), we would start an update to BCP
> 45 to formally move the location for last-call discussions,
> and we would update the 2007 IESG Statement on Last Call
> Guidance.
> 
> We invite comments, here, on this plan, by the end of
> September. As I say above, we've heard support from the
> community for the general idea, and we'd like to make sure
> this direction is what the community wants.
> 
> Barry, for the IESG
>

---

From: Rob Sayre <[email-masked]>
Date: Fri, 13 Sep 2019 00:22:19 -0700

Hi,

I think this is a good idea to try. It's obviously not an experiment,
because there is no control (as John Klensin pointed out). I also don't
think one can usefully perform an experiment when effects of a treatment
influence a global social graph, so I think the proposal is reasonable.

The core issue is whether people are using the "two purposes" described in
the list description to increase their audience, while the recipients might
have trouble distinguishing between technical traffic and "direction,
policy, and procedures":

"The IETF discussion list serves two purposes. It furthers the development
and specification of Internet technology through discussion of technical
issues, and it hosts discussions of IETF direction, policy, and procedures.
As this is the most general IETF mailing list, considerable latitude is
allowed with respect to topics discussed."[0]

It seems obvious to me that people are exploiting the dual attention of the
audience here, but reasonable people can disagree.

This idea also heads off bad behavior in a way that seems
under-appreciated: currently, people can derail a technical discussion with
a procedural issue, and remain on-topic for the list.

Ship it.

thanks,
Rob

[0] https://www.ietf.org/how/lists/

---

From: Barry Leiba <[email-masked]>
Date: Fri, 13 Sep 2019 09:56:00 -0400

Thanks, John, for the thoughtful response.

Just some brief responses from my personal point of view.

On point 1, I don't think the community request was prompted by one or
two (or three) discussions in particular, but by an overall busy IETF
list for quite some time.  And, stressing again, no one has said that
the discussions on the IETF list aren't useful or shouldn't happen...
simply that it would help to have separate lists so that people can
choose what they want to follow, and so that last-call discussions
don't get lost in the sea of other discussions that have been going on
-- and that we expect will continue to even after the three topics you
mention settle down.

On point 2, what we hope is that having the lists separated will make
it easier for people to follow the last-call discussions if that's
what they want to do.  Your argument seems to be that some people
(perhaps you, in the scenario you lay out) might decide that they want
to follow the IETF list but are not interested in being subscribed to
the last-call list, and we would thus lose last-call comments made by
someone who stumbled onto a last call that piqued her interest, but
which she would not have seen had it not been intermixed with the
other general chat.  That's possible, but I think vanishingly
uncommon.  I suspect that people such as you and I will remain
subscribed to both lists, and that it would be rare to have someone
who chooses to unsubscribe from the new list commenting on last calls
anyway.  And, yes, this will be admittedly hard to measure, though
we'll likely try a survey of some kind.

And certainly one benefit of separating the lists will be to make it
easier for people to filter the two types of discussion into separate
mailboxes -- something I already try to do by looking for "last call"
in the subject, but that isn't nearly as reliable.  (On the other
hand, Brian says he'll just filter them together into the same
mailbox, so there will be no apparent change for him.  À chacun, son
goût.)

Your comment about the timing of the experiment is well taken, and we
will consider that after the feedback period.  My personal inclination
is not to wait, but we'll see what the IESG as a whole thinks after
all the comments are in.

Barry

---

From: Bob Hinden <[email-masked]>
Date: Fri, 13 Sep 2019 09:06:40 -0700

Hi Barry,

I have read thought the discussion and have a few questions.

As I understand the proposal, the intent is to have the two lists initially have the same membership.  As you said:

After the initial list creation, will new people be required to subscribe to each list separately, or will subscribing to the [email-masked] list also cause a subscription to the last-call@ietf list?   I think this is an important issue as the value of each list is that there is a broad representation of people from the IETF community.

Is it a desirable outcome if the lists become very different in membership?   If the [email-masked] list becomes a lot smaller, is this a good outcome of the experiment, or a bad outcome?

I am thinking that both lists should have the same membership, that is, one can’t unsubscribe from only one.  This would preserve the broad community review of last calls and for community discussions, but still allow separate discussions.

How are we to evaluate the experiment?   I have have no doubt that it will achieve the goal of keeping the discussions separate.  I think we should be defining other criteria in which to evaluate the results of the experiment.

Also, a related question, how do new IETF participants know to subscribe to the IETF list these days?   Do we have any way of knowing if current active IETF participants are subscribed?   Perhaps, when registering for a meeting, the registration tool could offer to subscribe to the [email-masked] and [email-masked] lists.

Thanks,
Bob

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 13 Sep 2019 13:09:42 -0700

On Fri, Sep 13, 2019 at 9:07 AM Bob Hinden <[email-masked]> wrote:

> As I understand the proposal, the intent is to have the two lists
> initially have the same membership.  As you said:
>
> After the initial list creation, will new people be required to subscribe
> to each list separately, or will subscribing to the [email-masked] list
> also cause a subscription to the last-call@ietf list?   I think this is
> an important issue as the value of each list is that there is a broad
> representation of people from the IETF community.
>
> Is it a desirable outcome if the lists become very different in
> membership?   If the [email-masked] list becomes a lot smaller, is this a
> good outcome of the experiment, or a bad outcome?
>
> I am thinking that both lists should have the same membership, that is,
> one can’t unsubscribe from only one. This would preserve the broad
> community review of last calls and for community discussions, but still
> allow separate discussions.
>

I disagree with this. Part of the value proposition here is to allow people
to engage with last calls and avoid the... unpleasantness... which is the
ietf@ list.

-Ekr

> How are we to evaluate the experiment?   I have have no doubt that it will
> achieve the goal of keeping the discussions separate.  I think we should be
> defining other criteria in which to evaluate the results of the experiment.
>
> Also, a related question, how do new IETF participants know to subscribe
> to the IETF list these days?   Do we have any way of knowing if current
> active IETF participants are subscribed?   Perhaps, when registering for a
> meeting, the registration tool could offer to subscribe to the
> [email-masked] and [email-masked] lists.
>
> Thanks,
> Bob
>
>
>

---

From: Barry Leiba <[email-masked]>
Date: Fri, 13 Sep 2019 16:22:28 -0400

Hi, Bob.

> As I understand the proposal, the intent is to have the two lists initially have the same membership.

Right: initially, so that people don't miss anything in the transition.

> After the initial list creation, will new people be required to subscribe to each list separately,
> or will subscribing to the [email-masked] list also cause a subscription to the last-call@ietf list?

They will be separate lists.

> I think this is an important issue as the value of each list is that there is a broad representation
> of people from the IETF community.

That's not meant to change.  The IETF discussion list does not
currently have everyone subscribed.  People will choose what to
subscribe to, and that shouldn't change.  I expect both lists to have
broad representation... but not universal representation, and not the
same representation.

> Is it a desirable outcome if the lists become very different in membership?   If the [email-masked] list
> becomes a lot smaller, is this a good outcome of the experiment, or a bad outcome?

It is an expected outcome that the list memberships will be different.
If either list's membership becomes "a lot smaller", that, in itself,
is neither a good nor a bad outcome.  I would say that a bad outcome
would be significantly less last-call discussion.  A bad outcome would
also be bad community experience with the result, based on
discussion/survey after we've tried it for a while.

> Also, a related question, how do new IETF participants know to subscribe to the IETF list these
> days?   Do we have any way of knowing if current active IETF participants are subscribed?
> Perhaps, when registering for a meeting, the registration tool could offer to subscribe to the

This is a really good point, and, I think, a good suggestion for a
solution.  The IESG will discuss that as we plan registration for IETF
107.

Barry

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 14 Sep 2019 08:23:54 +1200

On 14-Sep-19 04:06, Bob Hinden wrote:
...
> Also, a related question, how do new IETF participants know to subscribe to the IETF list these days?   Do we have any way of knowing if current active IETF participants are subscribed?   Perhaps,…

I notice that the information for newcomers on the web site doesn't mention the ietf@ and ietf-announce@ lists. It would be useful to add a pointer to https://www.ietf.org/how/lists/ at https://www.ietf.org/about/participate/get-started/resources/

Barry: Also note that https://www.ietf.org/how/lists/ and https://www.ietf.org/how/lists/discussion/ will need updating for the duration of the experiment.

    Brian

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Sep 2019 02:45:21 -0400

--On Friday, September 13, 2019 13:09 -0700 Eric Rescorla
<[email-masked]> wrote:

>> I am thinking that both lists should have the same
>> membership, that is, one can't unsubscribe from only one.
>> This would preserve the broad community review of last calls
>> and for community discussions, but still allow separate
>> discussions.
>> 
> 
> I disagree with this. Part of the value proposition here is to
> allow people to engage with last calls and avoid the...
> unpleasantness... which is the ietf@ list.

Ekr,

I almost agree.  There have certainly been weeks lately in which
I would classify the bulk of the traffic on the main IETF list
as unpleasant and have wished that much of hadn't reached me.
However, we claim that the basis of what we do is "IETF
consensus".  Today, someone who opts out of the IETF list
essentially opts out of that consensus process no matter how
active they might be in, e.g., particular WGs.  If we split the
list and the membership of the two lists diverges, I wonder if
honesty and transparency require us to adjust our vocabulary to
indicate, e.g., "consensus of those who chose to participate in
the IETF's broad final review process".  

   best,
    john

---

From: Barry Leiba <[email-masked]>
Date: Sat, 14 Sep 2019 08:52:32 -0400

I don't follow this reasoning, John: it's already the case that many
IETF participants don't subscribe to the IETF discussion list, and
subscribing to it is not in any way a requirement.  Lots of people opt
out and do not think they're opting out of participation in IETF
consensus.  Last-call announcements go to ietf-announce and are
visible to people who subscribe to that, and not to this.  Last-call
discussions are often copied to the working group, as well.

If anything, separating the lists might *increase* the number of
people who explicitly subscribe to last-call discussions (but who
don't want to deal with the high volume on this list).

Barry

---

From: John C Klensin <[email-masked]>
Date: Sat, 14 Sep 2019 09:20:21 -0400

Barry,

--On Saturday, September 14, 2019 08:52 -0400 Barry Leiba
<[email-masked]> wrote:

> I don't follow this reasoning, John: it's already the case
> that many IETF participants don't subscribe to the IETF
> discussion list, and subscribing to it is not in any way a
> requirement.  Lots of people opt out and do not think they're
> opting out of participation in IETF consensus.

I have had the impression that, despite dropouts due to volume
on the list and other factors, the vast majority of active IETF
participants (those who were contributing to focused technical
mailings lists such as WG ones and/or coming to meetings and
participating in technical work) were still on the IETF list.
That impression may well be obsolete and the proportion of
participants who are on the list and paying attention to at
least the beginning and/or end of most threads is much lower
than what I thought it was a decade ago.  Or maybe I was too
optimistic and wrong even then.

If fairly low subscriptions and involvement among active
participants is the present reality, then you are quite correct
and my concern is irrelevant to the proposed experiment.

>  Last-call
> announcements go to ietf-announce and are visible to people
> who subscribe to that, and not to this.  Last-call discussions
> are often copied to the working group, as well.
> 
> If anything, separating the lists might *increase* the number
> of people who explicitly subscribe to last-call discussions
> (but who don't want to deal with the high volume on this list).

Indeed, especially if the subscription and involvement (or at
least active observation) level by IETF participants on the IETF
list is actually low, that might be a useful outcome.  It would
be good if could figure out a way to measure that, for example,
keeping track of subscriptions to the new list by people who
were not subscribed to the IETF one before the experiment
started, ideally controlling for newcomers.

I want to be clear, in case I haven't been, that I have never
been opposed to this experiment and am not now.   I have only
wanted to raise issues and concerns we should be aware of going
into it, including matters of timing and being alert to possible
side-effects.

I'll start a different thread, I hope later today, to address
the concern about the interplay between low subscription (or
involvement or at least active observation) rates and how we
think and talk about about IETF consensus.

best,
   john

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 15 Sep 2019 08:56:15 +1200

On 15-Sep-19 01:20, John C Klensin wrote:
...
> participating in technical work) were still on the IETF list.

I wonder if we could change from an impression to some data?

For example:

(A) How many people* are subscribed to ietf@ ?

(B) How many people are eligible under the current NomCom rule?

(C) How many people are "active" in some other sense (e.g. have sent mail to *any* IETF list in the past year)?

How many people are in (A) only, in (A) & (B), in (A) & (C), in (B) or (C) but not (A)?

We coud add

(D) How many people are subscribed to ietf-announce@

* people = distinct email addresses excluding the + construct.

These data exist if someone in the Secretariat could extract them.

Regards
    Brian

---

From: Barry Leiba <[email-masked]>
Date: Sat, 14 Sep 2019 18:07:54 -0400

> (A) How many people* are subscribed to ietf@ ?

About 1800.

But as I scan through the list, I see addresses that I know not to be
active any more.  And it's pretty much impossible to have any idea of
who actually looks at the messages, and so on.

> How many people are in (A) only, in (A) & (B), in (A) & (C), in (B) or (C) but not (A)?

This all might be interesting, but I think the real data that we'll be
able to collect as we experiment, and that we'll find useful, is this:

(α) How many people unsubscribe from the IETF Discussion list after
the Last Call list is active?

(β) How many people unsubscribe from the Last Call list?

(γ) How many new subscribers do we get for the two lists over time,
and how do they coordinate?

You think?

-- Barry

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 15 Sep 2019 16:42:07 +1200

On 15-Sep-19 10:07, Barry Leiba wrote:
> This all might be interesting, 

Yes, I think it's interesting in a broader context too.

> but I think the real data that we'll be

Yes, certainly. But cross-checking against ietf-announce (where the Last
Calls are sent) also seems relevant.

    Brian

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 15 Sep 2019 19:28:55 -0700

On Fri, Sep 13, 2019 at 11:45 PM John C Klensin <[email-masked]> wrote:

> active they might be in, e.g., particular WGs.

I'm not sure on what basis you make that claim, but I don't agree. The
discussion in the WG is part of the consensus proces.

-Ekr

If we split the

---

From: Melinda Shore <[email-masked]>
Date: Sun, 15 Sep 2019 18:45:07 -0800

On 9/15/19 6:28 PM, Eric Rescorla wrote:
> On Fri, Sep 13, 2019 at 11:45 PM John C Klensin <[email-masked]
> <mailto:[email-masked]>> wrote:

> discussion in the WG is part of the consensus proces.

Yes, but we do distinguish between working group consensus and IETF
consensus, and working group consensus is not sufficient to get a
document published.

Melinda

-- 
Melinda Shore
[email-masked]

Software longa, hardware brevis

---

From: Eric Rescorla <[email-masked]>
Date: Sun, 15 Sep 2019 20:16:39 -0700

> consensus, and working group consensus is not sufficient to get a

Yes, of course, but that doesn't mean that WG members need to separately
chime in on the IETF list in support or opposition of a document

-Ekr

---

From: Melinda Shore <[email-masked]>
Date: Sun, 15 Sep 2019 19:28:01 -0800

> chime in on the IETF list in support or opposition of a document

No, of course not, but in the event that something becomes
contested on the ietf@ mailing list, people who've opted out
but have strong views or particular experience don't participate
in those discussions.  It's very common for something to get
through WG last call and then run into problems later in the
process.

It is, of course, completely legit not to want to participate
in those discussions but not participating because someone feels
they've already had their say, or they don't care about IETF
consensus, or ... is not quite the same as not participating because
they find the mailing list intolerable.

There are myriad little ways in which we end up inadvertently
excluding input based on criteria that don't have much to do
with technical chops.  This is one of them.  It may be a price
we're willing to pay in order to have free-ranging and
unfettered discussions but we've never really made that
decision.

Melinda

-- 
Melinda Shore
[email-masked]

---

From: Keith Moore <[email-masked]>
Date: Sun, 15 Sep 2019 23:56:35 -0400

> unfettered discussions but we've never really made that

I think it's interesting that people keep talking about the effect of 
being inclusive, to exclude input from other people, as if this is some 
sort of inherent and unresolvable tussle.

Is there some way we can be inclusive to both sets of people?

Keith

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 16 Sep 2019 02:41:13 -0700

> they find the mailing list intolerable.

I don't think that's what I said, or at least not what I meant to say. If
people who have had their say in the WG subsequently opt not to participate
in the IETF list -- for whatever reason, even if because they find the IETF
list aversive -- that does not mean that their views are not included in
the consensus process.

-Ekr

---

From: Eliot Lear <[email-masked]>
Date: Mon, 16 Sep 2019 13:37:44 +0200

Hi, Eric,

> On 16 Sep 2019, at 11:41, Eric Rescorla <[email-masked]> wrote:
> 
> I don't think that's what I said, or at least not what I meant to say. If people who have had their say in the WG subsequently opt not to participate in the IETF list -- for whatever reason, even if because they find the IETF list aversive -- that does not mean that their views are not included in the consensus process.

If a point is raised during LC discussion that has not been addressed earlier, then that person is absenting him/herself from the consensus process at the time of LC.  If the point was already raised during WG development, then that person’s view could count.  After all, if the same point was raised, and s/he answered it once, why would that view be considered no longer valid?   As a practical matter, I don’t know how that would be taken into account. And that is not the same as the person being silent during WGLC.

Eliot

---

From: avri doria <[email-masked]>
Date: Mon, 16 Sep 2019 08:20:17 -0400

Hi,

Seems like it will be effective at reducing the cross section of the
community that will reached by a last call.

If that is what you all want.

avri

---

From: Eric Rescorla <[email-masked]>
Date: Mon, 16 Sep 2019 05:29:57 -0700

e

On Mon, Sep 16, 2019 at 4:38 AM Eliot Lear <[email-masked]> wrote:

> If a point is raised during LC discussion that has not been addressed
> earlier, then that person is absenting him/herself from the consensus
> process at the time of LC.
>

This seems like it's painting with an unnecessarily broad brush. Consider
the case where a document goes through WGLC and has strong consensus. Then
an issue is raised in IETF LC on one specific aspect of the protocol and
addressed by one of the authors but nobody else speaks up. That doesn't
generally diminish the overall consensus for the document

-Ekr

 If the point was already raised during WG development, then that person’s
> view could count.  After all, if the same point was raised, and s/he
> answered it once, why would that view be considered no longer valid?   As a
> practical matter, I don’t know how that would be taken into account. And
> that is not the same as the person being silent during WGLC.
>

---

From: Eliot Lear <[email-masked]>
Date: Mon, 16 Sep 2019 15:14:13 +0200

Hi Eric,

> 
> If a point is raised during LC discussion that has not been addressed earlier, then that person is absenting him/herself from the consensus process at the time of LC.
> 
> This seems like it's painting with an unnecessarily broad brush. Consider the case where a document goes through WGLC and has strong consensus. Then an issue is raised in IETF LC on one specific aspect of the protocol and addressed by one of the authors but nobody else speaks up. That doesn't generally diminish the overall consensus for the document

Actually, I realize that someone has made my point better than I ever possibly could.  What do you think of RFC 7282 Section 6?  All I’m saying is that I’m looking at this on an issue by issue basis with regard to a given document.  To me this is exactly what cross-area review means, and that is important because WGs often have blindspots, as I am sure you would agree.

And so that’s a good reason for people to pay attention LCs for docs they participated in developing.

Eliot