<<<
Chronological Index
>>> <<<
Thread Index
>>>
[council] RE: [gnso-wpm-dt] WPM Preliminary Status: Step 2
- To: Stéphane Van Gelder <stephane.vangelder@xxxxxxxxx>, "Ken Bour" <ken.bour@xxxxxxxxxxx>
- Subject: [council] RE: [gnso-wpm-dt] WPM Preliminary Status: Step 2
- From: "Gomes, Chuck" <cgomes@xxxxxxxxxxxx>
- Date: Thu, 10 Jun 2010 12:26:56 -0400
- Cc: <gnso-wpm-dt@xxxxxxxxx>, <council@xxxxxxxxxxxxxx>
- In-reply-to: <E81E5A50-EB2C-4502-9605-7B9161E29DE6@indom.com>
- List-id: council@xxxxxxxxxxxxxx
- References: <003201cb0812$1a080d80$4e182880$@verizon.net> <E81E5A50-EB2C-4502-9605-7B9161E29DE6@indom.com>
- Sender: owner-council@xxxxxxxxxxxxxx
- Thread-index: AcsIeklYtfC1gDLzRKqyUaNMA3Bs/wAPzWFg
- Thread-topic: [gnso-wpm-dt] WPM Preliminary Status: Step 2
Good questions Stéphane, but please note that there will not be time in today's
agenda to discuss the prioritization project. We will just have a Status
update. That discussion will start happening in Brussels. Of course, we can
discuss this on the list.
Chuck
From: owner-gnso-wpm-dt@xxxxxxxxx [mailto:owner-gnso-wpm-dt@xxxxxxxxx] On
Behalf Of Stéphane Van Gelder
Sent: Thursday, June 10, 2010 4:48 AM
To: Ken Bour
Cc: gnso-wpm-dt@xxxxxxxxx; council@xxxxxxxxxxxxxx Council
Subject: Re: [gnso-wpm-dt] WPM Preliminary Status: Step 2
Hi Ken,
Thanks for providing such a clear and concise response.
I would venture to suggest that the fundamental question we should now be
asking ourselves is whether the system, as currently proposed, would allow the
GNSO Council to decide that projects should be stricken from the to-do list
because they are rated too low in priority?
A secondary question may be, should the GNSO Council determine that there is a
finite number of projects that can be undertaken at the same time, considering
the available resources (# of Councillors, # of support Staff, etc.)?
I am copying the Council because this may be a point we want to discuss during
our teleconference tonight.
I guess what I am saying is that whilst prioritization is a good first step, to
be truly efficient, the GNSO may want to decide that it can only work on 15
projects in one given time period (e.g. a year) and that more projects can only
be added when one of the 15 is finished (the "15" is arbitrary here, I only use
it to illustrate my idea, I am not suggesting that should be the number)? I
freely admit to being inspired by Staff's batching idea for new gTLDs in this
regard ;)
Stéphane
Le 9 juin 2010 à 22:26, Ken Bour a écrit :
Stéphane:
I will try...
You wrote: "I wonder if the reason we are not getting any projects with a
lower score than 2 is that Councilors are not sufficiently aware that they can
strike projects all together should they wish to when they rate them?"
Shorter response:
1) Councilors are rating projects lower than 2 in Value; lots of "1s" have
been registered.
2) My original comment about ">2" dealt with the Range statistic, not the
individual Councilor project ratings.
I apologize for using your inquiry to springboard into a broader discussion of
central tendency measures, but I wanted to convey those thoughts to the WPM-DT
while they were fresh on my mind. Sorry, if my explanation was overly
confusing...
Ken
From: Stéphane Van Gelder [mailto:stephane.vangelder@xxxxxxxxx]
Sent: Wednesday, June 09, 2010 3:56 PM
To: Ken Bour
Cc: gnso-wpm-dt@xxxxxxxxx
Subject: Re: [gnso-wpm-dt] WPM Preliminary Status: Step 2
Hi Ken,
Well if I was confused before, I am now completely flummoxed after trying to
read your latest email :)
If there's any way you can explain it to me in a couple of sentences, I would
be grateful. If not, please just ignore my previous comment.
Stéphane
Le 9 juin 2010 à 15:18, Ken Bour a écrit :
Stephane:
I think you might have misunderstood my comment. Every project's Range
statistic is > 2, which means that, when I subtract the highest rating from the
lowest rating, that difference is larger than 2 for every project. On the
surface, that statistic indicates a wide spread among certain Councilors as to
their perception of the each project's relative Value. It also means that
there are no projects that can be automatically removed from the Brussels
discussion based upon the individual ratings step, which required a Range of 2
or less.
One of the possible improvement steps, going forward, may be to change the way
that central tendency is measured after the individual ratings round. When we
only had 5 testers, the Range looked like the best indicator; it is easy to
calculate and understand. Now that I am seeing over a dozen values, the
Standard Deviation might be a more useful statistic for determining agreement
since the population size is so much larger. For example, out of 12 ratings
for one particular project, we have the following ratings: 5, 3, 4, 3, 6, 4,
4, 4, 5, 4, 4, 4. Even though the Range is 3 (6-3), the Mode, the Median, and
the Mean are all equal to 4 and the Standard Deviation is less than 1.0
(actually .8), which indicates a very tight spread not only statistically, but
visually as well. For our current exercise, we will not be excluding this
project from discussion; but, in the future, it could be a candidate for
pre-determining agreement (in this case: Rating=4) on the basis of its low
Standard Deviation.
Here is one more example that is interesting. The 12 ratings thus far are: 6,
5, 7, 6, 5, 5, 7, 7, 4, 7, 7, 7. The Range=3, the Mode=7, Median=7, Mean=6,
and Std Dev=1.04. I think a strong case could be made for accepting 7 as the
group Value Rating although, if the rule were written such that Std Dev had to
be < 1.00, it would fail the test.
I expect to have a more fully thought-out recommendation once this part of the
process has completed and I have more time to analyze the results.
Lastly, I just want to be clear there are tons of individual project ratings
that are "1" and, for that matter, "7". From what I can discern in examining
each Councilor's spreadsheet, it doesn't appear that anyone misunderstood the
directions. Any value from 1-7 could be selected for any cell and, while some
Councilors used the entire range and others did not, I don't think there is
anything to be concluded other than that is how they perceived Value across the
range of Eligible Projects.
Ken
From: Stéphane Van Gelder [mailto:stephane.vangelder@xxxxxxxxx]
Sent: Wednesday, June 09, 2010 6:30 AM
To: Ken Bour
Cc: gnso-wpm-dt@xxxxxxxxx
Subject: Re: [gnso-wpm-dt] WPM Preliminary Status: Step 2
Thanks Ken for that update. I wonder if the reason we are not getting any
projects with a lower score than 2 is that Councillors are note sufficiently
aware that they can strike projects all together should they wish to when they
rate them?
Stéphane
Le 8 juin 2010 à 21:08, Ken Bour a écrit :
WPM-DT Members:
I thought you might appreciate receiving a brief status report concerning Step
2-Individual Councilor Ratings...
As of this afternoon, 8 June, I have received 12 Councilor ratings
spreadsheets. The deadline, as you may know, has been extended to 9 June
(tomorrow). Happily, other than a few names/dates being left off (I am saving
the emails and renaming the attachments so that I can positively ID each one),
the data aggregation process is going as planned and tested. No one, thus
far, has failed to provide a 1-7 rating for each of the Eligible Projects.
You may be interested, if not surprised, to learn that not a single project can
be excluded from discussion after the individual rating step. Every project's
Range is already > 2 and, of course, it cannot get any tighter as more results
are received. Of the 15 Eligible Projects:
· 11 or 73% have a Range >= 5
· 7 or 46% have a Range = 6 (max)
I have developed a consolidation spreadsheet, which is automatically
color-coded to reveal the top/bottom ratings and the most prevalent answer (or
Mode). Fortunately, several projects have pretty stable Mode/Median/Mean
results meaning that, while we might have a couple of 7's and 1's (thus
Range=6), most participants rated the project similarly. In a few cases, the
Mode, Median, and Mean are the identical value indicating strong central
tendency (so far)! In those instances, at least theoretically, it should be
possible to influence the small number of outliers to move closer to the
group's most common rating. Even if that is not possible, after discussion,
it will be somewhat comforting to know that there was reasonably strong
agreement statistically.
For Brussels, I estimate that we will have about 105 minutes net (if we can
hold preliminaries to 15), which leaves an average of 7 minutes per project for
discussion and polling.
I am currently drafting a letter that I plan to send out early next week (14th
or 15th) addressing as many preliminaries as possible so that the Brussels
meeting (on Saturday morning) can be quickly focused on the group ratings
discussions. This letter will cover such topics as: Councilor Preparation,
Meeting Setup, Guiding Principles, and Process Flow (briefly). I will be
encouraging participants to arrive a few minutes early so that we can speed up
the routine process of settling in...
If WPM-DT members would like to preview the letter before it goes out, please
let me know. Although I recognize that you are all very busy, I would
appreciate another set of eyes on this next communication...
Regards,
Ken Bour
<<<
Chronological Index
>>> <<<
Thread Index
>>>
|