How representative is the Forum: results

[El Salsero Gringo]

Summary

A survey of 100 recently updated threads was taken and the number of contributions from individuals was tallied. Each contributor was assesed against given criteria for their eligibility to be included in a group of 'representative' MJ customers, and the number of their respective contributions summed by category of 'representative', 'non-representative' and 'unknown'.

Of the 4192 posts surveyed, 57.0% were from 'non-representative' posters, 15.7% from 'representative' posters and 27.3% from posters whose inclusion in either of the other two categories could not be determined.

The survey concludes that more than half of the postings on the Forum come from a non-representative group of highly-experienced dancers or dancers with connections to venues and operators. Care should therefore be taken in extrapolating from the consensus of opinion on the Forum to the wider MJ world.

Method

An automated web-bot analysed the 100 most recently updated threads (as reported at 10 am, 9 March 2005 in the Let's talk about Dance forum.

An a priori assumption was made based on the marketing of MJ and Ceroc in particular as an easy-to-learn dance for non-dancers that the 'typical' MJ customer has less than 4 years of dance experience, and no personal connection to running a venue or dance institution. (The author acknowledges that this could only be verified with access to commercially sensitive data held by venue operators, and that he has no such access.)

The 50 most prolific posters among the surveyed group were PM'd to to assess themselves and reply as to whether they fell into any of the following groups:

Venue owners, franchisees, teachers, taxi-dancers, DJ's, promoters, commercial Forum members or anyone with a remote connection to running a venue or dance company - and also anyone who has four or more years of dance (not necessarily MJ) experience.

From the replies received and from other postings on the Forum (in particular the threads How Long? & By Way of Introduction) along with personal knowledge of the author, the contributors were assigned to the category of 'representative of a median MJ dancer' (those not meeting any of the above criteria) or 'unrepresentative of the same' (those meeting at least one of the above criteria.)

Results

The 100 threads yielded 4192 posts written by 241 contributors. Of the 50 top contributors contacted, 28 replies were recieved returning the requested information, three replies explicitly declining to answer and, at time of writing, the Forum software reports that 4 more have not yet read the PM requesting the information. A further 15 are therefore judged to have declined to respond.

Overall, 90.3% of the posts came from the top 100 posters, and 73.5% from the top 50 posters. The single most prolific poster had written 315 messages, or 7.5% of the total, more than double the next highest contributor. 54 posters each contributed only one message.

After analysis it was determined that 57.0% of the posts were written by 'non-representative' posters, 15.7% by 'representative' and the categorisation of the authors of the remaining 27.3% of the posts analysed could not be determined.

A cumulative frequency graph of the results in all three categories is included as an attachment to this posting. The horizontal axis marks the rank of the poster arranged in order of number of posts, the vertical axis the cumulative percentage of posts up-to and including that poster. The blue area marks 'non-representative' posters, the yellow 'representative', and the white area is posts whose authors are of category unknown.

Conclusion

Relying on an assumption that the contents of the 100 surveyed threads are a representative sample of Forum postings, and noting that more than half of the total number of posts surveyed were provided by Forum members who are highly experienced or connected with running a venue or dance organisation, the author concludes that it cannot be claimed without a case-by-case justification that the consensus of opinion expressed on the Forum matches or is even indicative of the wider MJ scene.

While there is no bar to Forum members having the same breakdown of opinions on individual issues as all MJ dancers as a whole, the demonstrated disparity between the make-up of those writing on the Forum and what might be (reasonably assumed to be) a representative group of MJ dancers makes this anything other than a foregone conclusion.

The author also feels that amongst the 1320 or so currently registered Forum members are a significant number of less experienced but equally comitted dancers whose opinions on the wide range of topics on the Forum are equally valuable (or given their relative scarcity, more valuable) to maintaining a healthy debate than those of the 'old hands'. The author hopes that by seeing to what extent they are currently absent they will be encouraged to become "as mouthy as the rest of us."

Acknowlegements and thanks

It has been pointed out by ChrisA and others that entering in MJ competition is also a 'non-representative' activity and that this should have been included as a relevant selective criterion.

The author would like to thank all those individuals who have taken the time to critique the methodology and motivation behind this study, most of which can be found in the thread Forum Research, and particularly Gadget, whose trenchant criticism considerably strengthened this work by demanding consideration and where necessary, clear-headed rebuttal. Thanks also to all those people who took the trouble to reply to the PM asking for their help, and my apologies to those for whom the receipt of a PM was an unwarranted intrusion. Errors and omissions are of course, entirely mine.

[Chicklet]

Anyone out there think this is telling them something they didn't already know?

[El Salsero Gringo]

Anyone out there think this is telling them something they didn't already know?

Yes; me.

But I'm obviously not as smart as you or else I wouldn't have needed to see the results before making up my mind.

( )

[Simon r]

Yes; me.

But I'm obviously not as smart as you or else I wouldn't have needed to see the results before making up my mind.

( )

For some reason i think you need to get out more ......

thanks for such an in depth report, but as previously replied i think as a group we were already aware thart those that new about the forum, posted on the forum and spent most evenings dancing were prehaps not the avarage MJ dancer.

[Andy McGregor]

The single most prolific poster had written 315 messages, or 7.5% of the total, more than double the next highest contributor. 54 posters each contributed only one message.

I so often find myself on the cutting edge of these graphs - or one edge or the other of a bell shaped curve.

I still remember my surprise at this when I was subjected to my first bank of psychometric and IQ tests. I wasn't surprised at myself, I know how I am - it was how different everyone else is that surprised me

Of course the people who post on the Forum aren't normal/average. My guess it that the regular Forum posters are people with a lot to say about MJ. That can't be average: most people let their dancing speak for itself


And then there are some dancers who are too abnormal to even come here ...

.. did he have time to do her zip back up before the knee landed?

[Magic Hans]

nice little analysis!!!

I can't say that I entirely know what to make of it .... but nice anyway!!

Kind of reminds me of my physics experiment write ups at school ..... or how they ought to have been!!!



Ian

[MartinHarper]

Anyone out there think this is telling them something they didn't already know?

I had guessed this, though I'm a little surprised at exactly how unrepresentative we are. I believe that a few folks have expressed an opinion that the forum is representative of the MJ world as a whole. For example, regarding clapping:

I think we are very representative

[Stuart M]

<all that statistician stuff>

One other factor to bear in mind is the number of Modern Jivers with no (or limited) internet access, of course. Still a significant portion I'll bet

Oh, and Graham, good luck with the job-hunting...

[El Salsero Gringo]

i think as a group we were already aware thart those that new about the forum, posted on the forum and spent most evenings dancing were prehaps not the avarage MJ dancer.

Up until this point, anyone who 'knew' that there was a link between postings and dance experience was making an assumption.

You might be happy with that; for the little difficulty involved I prefer to check. And now that I've done so, you don't need to assume any more. You know.

Anyone who claims to 'know' about a connection between how many evenings per week Forum members dance and how often they post is still making an assumption. Or else they've done their own research and not shared the results.

[Simon r]

Up until this point, anyone who 'knew' that there was a link between postings and dance experience was making an assumption.

You might be happy with that; for the little difficulty involved I prefer to check. And now that I've done so, you don't need to assume any more. You know.

Anyone who claims to 'know' about a connection between how many evenings per week Forum members dance and how often they post is still making an assumption. Or else they've done their own research and not shared the results.

As i have been on the MJ scene for quite a while i know most of those that do post so not assumption but knowledge of my fellow forumites. Well done but not sure how it helps us

[Aleks]

How many hours did this survey take to compile and interpret?

I would most certainly rather have spent my time dancing and enjoying the company of both representative and unrepresentative MJ'ers.....

[ChrisA]

I had guessed this, though I'm a little surprised at exactly how unrepresentative we are. I believe that a few folks have expressed an opinion that the forum is representative of the MJ world as a whole.

Be careful how you interpret this survey.

ESG's survey shows the relationship between level of postings and dance experience amongst those on the forum. It shows (given the definitions he's given us) that those that post most are relatively unrepresentative of the population of forum members at large.

It tells us nothing about how representative forum posters (or non-posters for that matter) are of the MJ world at large, because it gives no direct information about that - for that, we have only the sort of assumptions that ESG is talking about, or the sort of direct first hand knowledge that Simon is talking about.

[Simon r]

Be careful how you interpret this survey.

ESG's survey shows the relationship between level of postings and dance experience amongst those on the forum. It shows (given the definitions he's given us) that those that post most are relatively unrepresentative of the population of forum members at large.

It tells us nothing about how representative forum posters (or non-posters for that matter) are of the MJ world at large, because it gives no direct information about that - for that, we have only the sort of assumptions that ESG is talking about, or the sort of direct first hand knowledge that Simon is talking about.

[Lory]

From my own point of view, it was interesting reading and no harm done but will have no impact or change the way I look at the forum or interpret any threads.

As Simon Says, out of the top 50 posters I probably know most of them in one way or another and as for the faceless, postless 'guests' I'm still none the wiser as to who they are or what they make up it/us all! (although I sometimes have my suspicions )!

[El Salsero Gringo]

Simon, Chris,

It shows (given the definitions he's given us) that those that post most are relatively unrepresentative of the population of forum members at large.

With respect to you both, this is incorrect. There's no data included about the distribution of Forum members at large, and so few conclusions can be drawn about the sample of posters relative to Forum members as a whole.

It tells us nothing about how representative forum posters ... are of the MJ world at large

It might not - but it was intended to. If you think it doesn't then it's up to you to be specific about which assumptions I've made are wrong, or how the conclusion that I've drawn is false.

we have only the sort of assumptions that ESG is talking about, or the sort of direct first hand knowledge that Simon is talking about.

These are two quite different things.

To judge the experience-level of who is is posting on the Forum, Simon favours the use of his own (hearsay and/or experiential) experience, which I prefer to replace with the results of a brief survey. At least then I can quantify the size of the gaps in my knowledge, whereas Simon won't know what he doesn't know - if you see what I mean.


To judge the 'average' MJ dancer, Simon and I both rely on assumptions of who that dancer is. I have at least identified my assumptions for others to criticize.

It may be that Simon, as an experienced teacher with more contact with punters, has an idea for a better benchmark of the 'average' dancer than the one I used - but if he has, he hasn't put it forward yet.

[djtrev]

For what its worth, regardless of ESG's findings I still think the members on this forum are representative because thats what I want to think!!!
Apart from that the whole thing passed over my head at a great height and at great speed which is where it will stay.

As somebody has already said - get out more

[Simon r]

So from the minority of replies recieved you have based your entire theory on the structure and make up of the forum, lets be honest your representation of the forum and the manner you have collected your data is completely flawed.
I do not feel you have any basis on the data that you have collected to give a true or unbiased representation of the forum.
My question still stands what are you trying to achieve, show, as i still see no real conclusion.
I am not being argumentative but the tone of your mail seems aggresive.
As a fairly new member to the forum and to the MJ scene i am sure you have only the best intrest's at heart, but the tone you are setting will put peoples backs up.

[Lynn]

Hmm, could you look at this the other way - not just the contribution to the forum of 'non-representative' dancers - but the contribution of the forum and association with it to making dancers more 'non-representative'??

(OK, probably only applies in my case I know!)

[David Franklin]

It tells us nothing about how representative forum posters ... are of the MJ world at large

It might not - but it was intended to. If you think it doesn't then it's up to you to be specific about which assumptions I've made are wrong, or how the conclusion that I've drawn is false.

I think the big flaw is weighting the survey by number of posts. That most posts are made by experienced dancers does NOT imply most posters are experienced dancers (your survey says nothing at all about the 80% or so of posters who fell outside the top 50 and were not questioned).

I understand why you feel some kind of weighting is important, but I think what you've done goes too far the other way. Consider 2 people arguing over something for 200 posts, and then a 3rd party steps in with a single post that makes everything clear, and discussion stops. Your analysis would have the 2 bickerers being far more influential on "forum consensus" than the 3rd party, when that clearly isn't the case. That's an extreme example, but not entirely without basis IMHO.

Not that I have experience in this area, but for a formal study I think you would also need to be a lot clearer about how you assigned people to one category or another. My impression from what you wrote is that you both occasionally put people in a different category from what they said themselves, and that you also categorised the non-respondants based on other information on the forum. I suspect either action would be hard to justify in a more formal scenario.

On the other hand, in a lot of ways this is nitpicking. I would agree we're not representative in the statistical sense - because the most representative MJer is probably the person who went twice and gave up. But nearly everyone here went twice and thought about giving up, so I'd like to think we can represent even that person's views - at least a little bit...

Dave

[ChrisA]

With respect to you both, this is incorrect. There's no data included about the distribution of Forum members at large, and so few conclusions can be drawn about the sample of posters relative to Forum members as a whole.

Agreed, my apologies, I'd forgotten the PMs you sent were only to a selected few.

It might not [tell us anything about how representative forum posters ... are of the MJ world at large] - but it was intended to. If you think it doesn't then it's up to you to be specific about which assumptions I've made are wrong, or how the conclusion that I've drawn is false.

David F summed it up well, in a PM to me telling me (quite rightly, and very nicely) that I was talking cack ...

The survey doesn't really say anything about "forum members at large", because he didn't send those people a PM. What he's done is arbitrarily define what is an unrepresentative dancer, and found that most of the heavy posters fit that definition.

All you've done is establish that most of the people that post a lot don't meet your definition of an average MJ dancer. You haven't measured anything about the quality of the posts, or whether they actually express opinions, or whether those opinions, where expressed, are shared by the average dancers, whether forum members or not.

So I don't see how the survey shows anything at all about how representative the opinions expressed here are of the MJ world at large.

Speaking for myself, when I wasn't any of the things that fitted your definition of 'non average' I didn't even have opinions on many of the things discussed here, since I didn't know there were opinions there to be had.

To judge the experience-level of who is is posting on the Forum, Simon favours the use of his own (hearsay and/or experiential) experience, which I prefer to replace with the results of a brief survey. At least then I can quantify the size of the gaps in my knowledge, whereas Simon won't know what he doesn't know - if you see what I mean.

I don't think this is true. You analysed the forum to find out who the top posters are, and asked them which category they were in. Simon (as do many of the top posters, of course) knows personally many of the top posters, and enough of their dance background to know which of your categories they fit into.

His experiential knowledge is at least as accurate as the survey results, I would surmise.

On a related note, I just hope this thread isn't going to be used as ammunition for what seems to me to be an increasing tendency for people that take things like technique, music, competitions etc, seriously, to be labelled as elitist.