Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Describing and predicting behavior: The Big Five

By roffe in Culture
Mon Mar 28, 2005 at 04:29:02 AM EST
Tags: Science (all tags)

The answer to the question "What is personality?" depends very much on why the question is asked. There are many ways to deal with this question. Here, I will focus on the trait-based approach, which is one attempt at describing individual differences and predicting individual behavior.

This knowledge is useful. For example, research shows that companies that use knowledge of personality as part of their selection procedures have higher productivity, less employee turnover, and higher employee satisfaction than those that don't.

The last twenty years, personality research has been centering around a model known as the "Big Five," and sometimes referred to as the "Five-Factor Model of Personality". This is the culmination of nearly a hundred years of research. I will sketch the history of the Big 5 here, provide a brief presentation of the factors, and mention the MBTI in passing.

The Lexical Hypothesis

The story begins with Gordon Allport and H.S. Odbert, who hypothesized that

Those individual differences that are most salient and socially relevant in people's lives will eventually become encoded into their language; the more important such a difference, the more likely is it to become expressed as a single word.

This has become known as the "Lexical Hypothesis."

What Allport and Odbert did, though, was to go through two of the most comprehensive dictionaries of the English language available at the time, and extract 18,000 personality-describing words. From this gigantic list, they extracted 4500 personality-describing adjectives that they found describing observable and relatively permanent traits. And then, in 1936, they rested their case.

The case was picked up again in 1946 by Raymond Cattell, who used new technology, i.e. computers, to do data reduction on the Allport-Odbert list. He organized the list into 181 clusters and asked people to rate people they knew by the adjectives on the list. Using factor analysis, he ended up with twelve factors. He added four factors that he thought ought to have been there, and ended up with the hypothesis that individuals describe themselves and each other according to sixteen different, independent factors.

With these sixteen factors as a basis, he went on and constructed the 16PF Personality Questionnaire, which is still being used in universities and businesses for research, personnel selection and the like. Later research has failed to replicate the results, however, and it has been shown that Cattell retained too many factors. The current 16PF, however, takes these findings into account and is in fact a very good test which is still being developed. In 1963, W.T. Norman redid Cattell's work and suggested that five factors would be sufficient.

The Dark Ages

And then for seventeen years, nothing happened: It was proven that personality is not stable, but varies wildly with situations, so that prediction of behavior by personality test is impossible: Social Psychologists demonstrated that character, or personality, is something humans impose on people in order to maintain an illusion of consistency in the world. The final nail in the coffin was Walter Mischel's 1968 book Psychological Assessment which demonstrated that personality tests cannot predict behavior with a correlation of more than 0.3.

Around 1980, three things happened that brought the present strain of personality research out of the Dark Ages. These were Personal Computers, Statistical Aggregation, and The Big Five.

Personal Computers

Traditionally, Psychologists who needed computers had to rent access to a mainframe. Suddenly, they could do their statistical analysis on their desktops. This meant that anybody could, for instance, re-examine the Allport-Odbert list. But why would they, if personality is an illusion?

Statistical Aggregation

It was argued that personality psychologists had looked at behavior from the wrong level. Instead of trying to predict single instances of behavior, which just didn't work, researchers should try to predict patterns of behavior. Correlations soared from .3 to .8. Hey presto, we've got personalities anyway! Social Psychologists still argue that we impose consistency on the world, but that there is in fact more consistency to begin with than they once thought.

The Big Five

At a symposium held in Honolulu in 1981, the four prominent researchers Lewis Goldberg, Naomi Takamoto-Chock, Andrew Comrey, and John M. Digman reviewed the personality tests available at the day, and decided that most of the tests that held any promise seemed to measure a subset of five common factors - in fact the same as Norman had discovered in 1963.

We've got triangulation!

Both the lexical hypothesis, then, and more theory-laden research converge on a model that states that personality can be described in terms of five aggregate-level trait descriptors. The Big Five model is still in a state of transition, and researchers tend to develop their own variants, however when researchers talk to each other, they usually translate their model into the one proposed by Norman in 1963. These are

The Factors

  • Surgency or Extraversion

    How energetic one is. People who score high on this factor like to work in cooperation with others, are talkative, enthusiastic and excitement seeking. People who score low on this factor prefer to work alone, and can be perceived as cold, difficult to read, even a bit eccentric.

  • Agreeableness

    One's level of orientation towards other people. Those who score high on this factor are usually co-operative, can be submissive, and concerned with the well-being of others. People who score low on this factor can be challenging, competitive, sometimes even argumentative.

These are the two "social" factors. People who score more than middle on both plus more than middle on IQ tend to have high "emotional intelligence."

  • Conscientiousness

    How "structured" one is. People who score high on this factor are usually productive and disciplined and single tasking. People who score low on this factor are often less structured, less productive, but can be more flexible and inventive, and can be multitasking.

  • Neuroticism or (inversely) Emotional Stability

    Tendency to worry. People who score low on this factor are usually calm, relaxed and rational and may sometimes be perceived as lazy and incapable of taking things seriously. People who score high on this factor are alert, anxious, sometimes worried.

  • Culture or Openness to Experience or Openness to Ideas

    Tendency to be speculative and imaginative. People who score high on this factor are neophilic and curious and sometimes unrealistic. People who score low on this factor are down-to-earth and practical and sometimes obstructive of change.

There is a lot of research available on the Big Five. The problem is that very little of it is published in books with pictures - most of it is relatively uncompiled in research journals. In order to make use of the Big Five, then, one needs to be up to date on the literature.

When conflicts are due to differences in personality, it is usually due to differences in Openness to Experience. The factors that predict job performance are Conscientiousness (which should be high) and Neuroticism (which should be low). Leaders should be high on all factors except Neuroticism and Agreeableness which should be low - and salespeople should be similar but with even higher scores on Surgency. Incidentally, this is culture-dependent - in most countries outside the USA (including Canada), leaders and salespeople should be higher on Agreeableness than in the USA


What about the MBTI (Myers-Briggs Type Indicator), then, which I assume most of you have heard of? Well, it seems that it measures four of the factors - Neuroticism is left out, but has been proposed as added in newer revisions, against much protesting. The MBTI is not based on the lexical hypothesis but on a theory which is, to Jung's credit, fairly head-on with current research. There is one issue with the MBTI, though, and that is the notion that all the 16 types it covers are equally good but different - this, the Big Five research has shown, is not true. Most ENTJ's are better leaders than most INTP's, not just different types of leaders. This one issue does of course not invalidate all of the MBTI, which is a very good test.

Further research

Current research concentrates on three areas. The first is: Are the five factors the right ones? Attempts to replicate the Big Five in other countries with local dictionaries have succeeded in some countries but not in others. For instance, Hungarians, apparently, don't have Openness to Experience. Of course they do, others say, the problem is that the language does not provide enough variance of the related terms for proper statistical analysis. Some have found seven factors, some only three.

The second area is: Which factors predict what? Job outcome as leaders and salespeople has already been measured, and lots of research is currently being done in expanding the list. You need to search the literature to find just what you're looking for, unfortunately. Barrick and Mount's research are good places to start.

The third area is to make a model of personality. Costa and McCreae have built what they call the Five Factor Model of Personality which is an attempt to use the Big Five to provide a model of personality that can explain issues in personality from the cradle to the grave. They don't follow the lexical hypothesis, though, but favor a theory-driven approach.


Everybody is interested in personality, inasmuch as everybody interacts with themselves and other people. The Big Five is a convenient way to reduce personality into manageable bits. I hope this little piece has been entertaining and informative, and would like to receive lots of questions for a follow-up article.

Some literature and a link

About the history of the Big Five

Digman, John M. (1996) The curious history of the Five-Factor Model. In: Jerry S. Wiggins (ed.), The Five-Factor Model of Personality:Theoretical Perspectives chapter 1, pp. 1-20. New York: Guilford

John, Oliver P. (1990) The "big five" factor taxonomy: Dimension of personality in the natural language and in questionnaires. In: Lawrence A. Pervin (ed.), Handbook of Personality chapter 3. New York: Guilford

About the content of the Big Five

Barrick, M.R. and Mount, M.K. (1991) The Big Five Personality Dimensions and Job Performance: A meta-analysis Personnel Psychology 44, 1-26

Barrick, M.R., Mount, M.K. and Judge, T.A. (2001) The FFM Personality Dimensions and job performance: A meta-analysis of meta-analyses. International Journal of Selection and Assessment 9, 9-30.

Books with pictures

Howard, P.J. and Howard, J.M. (2001) Owner's Manual for Personality at Work, The: How the Big Five Personality Traits Affect Performance, Communication, Teamwork, Leadership, and Sales Austin, Texas: Bard Press

A link and a test

Being the most computer-savvy of Psychologists, prominent Big Five researchers have created a collaborative web site called The Personality Project.


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


Related Links
o heard
o of
o The Personality Project
o Also by roffe

Display: Sort:
Describing and predicting behavior: The Big Five | 143 comments (101 topical, 42 editorial, 0 hidden)
I like personality tests (3.00 / 8) (#34)
by Cat Huggles on Sat Mar 26, 2005 at 08:14:18 PM EST

They ask you questions and then after you've answered them, tell your answers back to you!

urk (none / 0) (#35)
by Cat Huggles on Sat Mar 26, 2005 at 08:32:46 PM EST

Maybe not!

It says I'm intuitive, emotionally stable and open to experiences.

But apparently I'm extremely disagreeable and impulsive. I suppose I am a bit misanthropic and procrastinatory (ie. I'm on k5).

[ Parent ]

Would you like to take this personality test? (none / 0) (#117)
by zephc on Wed Mar 30, 2005 at 12:01:07 PM EST

Here you go, friend.

Also, a 'study guide' to help you out.

[ Parent ]

I CHANGE MY MIND EVERY DAY! (1.00 / 9) (#39)
by communistpoet on Sat Mar 26, 2005 at 10:20:32 PM EST


We must become better men to make a better world.
Well-Designed Tests (2.50 / 2) (#41)
by cronian on Sat Mar 26, 2005 at 10:25:41 PM EST

Unfortunately, saying that or refusing to finish the test puts in the rebellous category, or something like that. If the test writer is smart, they make a special category just for you. If used for important decisions, your results may not figure how you want them to.

It is better to understand the test's methodology first. Then, you can decide what personality they will think that you have.

We perfect it; Congress kills it; They make it; We Import it; It must be anti-Americanism
[ Parent ]
Practical Applications (3.00 / 8) (#40)
by cronian on Sat Mar 26, 2005 at 10:22:39 PM EST

Ok, so who uses these things? Do you know of any companies that hire based on these tests? How can they be gamed?

Pyschological tests might actually predict something, but where is the fun in that. I firmly believe the best reason for studying psychology is to learn how to rig the tests in your favor.

According to Wikipedia's entry on Timothy Leary
When he[Timothy Leary] arrived in prison, he was given a standard psychological test that the prison used to assign inmates to appropriate work assignments. Having written the test himself, he was able to give the answers that got him a job working in the prison library.

So, how does this test work. I understand that gaming psychological tests can prove a useful skill.

We perfect it; Congress kills it; They make it; We Import it; It must be anti-Americanism
You shouldn't try to game the tests (none / 0) (#45)
by roffe on Sun Mar 27, 2005 at 03:46:59 AM EST

I would approach this from a different angle.

If Leary's test was any good, he would have been assigned a meaningful job just by answering honestly. Chances are honesty would have put him in the library anyway. Test theory, and consequently, test construction, has improved since Leary's days, so just to beat the inaccuracies of the test, of course it helped to be familiar with it.

Don't try to game tests, answer honestly and learn from them. If you give fake answers, you will be give unsuitable assignments.

Not being an American, I don't know which US companies use testing, but I believe a lot of them do, especially the big ones. I could tell you about Norwegian companies though :)

Rolf Marvin Bøe Lindgren

[ Parent ]
Leary wrote the test (none / 1) (#66)
by godzillion on Mon Mar 28, 2005 at 09:04:18 AM EST

I believe in this case that Leary wrote the test (the Leary Interpersonal Grid?) himself, so he was very familiar with what the test was designed to measure. Prisons are most interested in people who are docile, agreeable, and optimistic, as such prisoners seem much less likely to: hurt other prisoners, kill themselves, escape, etc. While Leary does seem to have been an optimistic fellow, he also seems strongly opinionated with a streak of the maverick. Had he answered honestly, it seems doubtful that he would have been trusted by the prison administration as much as he had. Keep in mind his subsequent prison break.

[ Parent ]
You're assuming... (3.00 / 2) (#84)
by abulafia on Mon Mar 28, 2005 at 09:40:50 PM EST

...that the test is both honest and a good measure of what it is supposed to be testing, the giver has your interests at heart, and that personal volition in an impersonal world is to be avoided; that somehow everything will work out for the best.

I've frequently found that at least one of the above is not true for many situations.

[ Parent ]

one company I worked for (none / 1) (#51)
by speek on Sun Mar 27, 2005 at 09:32:26 AM EST

They gave me a personality and IQ test before hiring me. The test indicated I was the most impatient person they'd ever known, but they hired me anyway.

al queda is kicking themsleves for not knowing about the levees
[ Parent ]

Huh? (none / 0) (#97)
by Surial on Tue Mar 29, 2005 at 09:30:22 AM EST

That's not (anymore) in the wikipedia entry.

In fact, the wikipedia entry infers that he gamed the tests so he could be a gardener, possibly with the underlying intent to be in a better position to escape (which he did).

"is a signature" is a signature.

[ Parent ]

Leaders? We don't need no stinkin'... (2.20 / 5) (#42)
by Kasreyn on Sat Mar 26, 2005 at 10:45:09 PM EST

the notion that all the 16 types it covers are equally good but different - this, the Big Five research has shown, is not true. Most ENTJ's are better leaders than most INTP's, not just different types of leaders. This one issue does of course not invalidate all of the MBTI, which is a very good test.

*scratches head* Yes, and an INTP is likely to be a much better blue-sky researcher. Why are you suddenly measuring by leadership? What does it have to do with anything?

How would you refute the claims that have repeatedly been made, that the MBTI is nothing but a zodiac-style generalization? Ie., that it categorizes meaninglessly with positive labels for everyone. For instance, have there been any double-blind studies where multiple examiners applied the MBTI to the same person/people and came up with the same index?

"Extenuating circumstance to be mentioned on Judgement Day:
We never asked to be born in the first place."

R.I.P. Kurt. You will be missed.
It's just an example (none / 1) (#44)
by roffe on Sun Mar 27, 2005 at 03:25:48 AM EST

I chose leaders and salespeople as examples because they're both very well researched, wheras, to pick another example, blue-sky researchers are not.

The present text is about the Big 5, not the MBTI, which is why I tried to keep the MBTI references to a minimum, and just use them as a useful comparison since many of K5's readers are familiar with it.

But yes, the MBTI has much better inter-rater reliability than the signs of the zodiac - that is, observers tend to agree on what personality type a given individual has. The MBTI is not my favorite test, still it is one of the best available.

Rolf Marvin Bøe Lindgren

[ Parent ]
Hey! (none / 0) (#69)
by Bryan Bytehead on Mon Mar 28, 2005 at 09:39:45 AM EST

As an INTP, I resemble that remark! Or is it I dissemble that remark? I've probably missplet it anyways.
Visit my blog.
[ Parent ]
Personality testing is a pseudoscience (2.83 / 6) (#47)
by Polyxena on Sun Mar 27, 2005 at 05:18:29 AM EST

There's a good entry on the Myer-Briggs here. The criticisms apply equally well here.

Thanks for the link! (none / 0) (#48)
by roffe on Sun Mar 27, 2005 at 05:26:49 AM EST

Very interesting!

Rolf Marvin Bøe Lindgren

[ Parent ]
Maybe (3.00 / 2) (#90)
by bob6 on Tue Mar 29, 2005 at 04:57:54 AM EST

But I don't think the goal is to be scientific, it's rather a framing tool to give credibility to human resources departments.

[ Parent ]
I have some probleme with the research (2.66 / 6) (#50)
by mcgrew on Sun Mar 27, 2005 at 07:51:25 AM EST

...or rather, the application of what has been found. Even your article takes an anti-human approach, "this is where this little cog of the machine fits best." And what's worse, nobody argues against it.

I would outlaw this kind of test were it up to me. It is as offensive as discrimination by color or sex. Now, were one to show these data to job seekers as guides to job satisfaction, I might encourage that.

"The entire neocon movement is dedicated to revoking mcgrew's posting priviliges. This is why we went to war with Iraq." -LilDebbie

we should enforce random hiring? (none / 1) (#53)
by speek on Sun Mar 27, 2005 at 09:39:00 AM EST

Or only allow intuitive "winging-it" interviews? Not that I'm really disagreeing with your point - a lot of people have feared the application of scientific methods to the study and then control of people, and with good reason. But telling people not to use proven methods* of selecting people seems bad too.

* - I'm not saying these personality tests are proven

al queda is kicking themsleves for not knowing about the levees
[ Parent ]

This could be the topic for an entire new article (none / 1) (#54)
by roffe on Sun Mar 27, 2005 at 01:51:34 PM EST

My original intent was to describe a model of personality and the reasoning behind it, and not focus all that much on testing - which is difficult since much of personality theory deals with testing. Personality researchers construct tests in order to test their own theories of personality. Some of these tests make it to business and industry.

Testing should only be done by competent testers, who know how to select the proper test(s) for the occasion and how to administer and interpret it, and so on (there is more). When the entire test process is carried out properly, the result is a better evaliation than what a job interview can possibly give, to the benefit of both involved parties. Testing is not inherently anti-humanistic - it's very humanistic when it is used properly to discover people's aptitudes and abilities. Testing doesn't have to be pigeonholing, either.

I have seen the results of incompetent testing - it is possible to destroy a person by incompetent test use. Testers have told peope that they cannot possibly become leaders, for example, or that they are possibly alcoholic. Being sceptical to testing is quite laudable, but I don't, as should be obvious, agree with those who reject testing altogether.

Rolf Marvin Bøe Lindgren

[ Parent ]
Meritocratic hiring is inherently unfair! (2.50 / 2) (#76)
by tantadruj on Mon Mar 28, 2005 at 01:56:28 PM EST

There is no way you can make hiring completely fair and unoffensive to everybody. Unless you introduced completely random hiring, which could arguably be the most fair way to select between candidates that match formal criteria for the job. Such a method would consist of precise and independently verifyable list of job requirements, followed by a draw.

As soon as you decide to take the best candidate, you've entered the realm of psychology with all its consequences. You personally might dislike personality tests as tools that can help you determine who is the best, but how is that any worse than not liking one candidate's hesitation with some answers, someone else's creapy smile and the third person's taste in parfume? Personality tests are, in my indeed very humble experience, a useful tool of seeing past such first in-person impressions that might deceive you into making a decision that will turn out to be both wrong and preventable.

[ Parent ]

it depends (none / 1) (#79)
by emmons on Mon Mar 28, 2005 at 05:14:21 PM EST

It depends on what factors you consider in determining fairness. Personally, I consider merit and demerit to be inherent components of one's value. Thus, if a person is rewarded proportional to his or her level of merit, I consider it perfectly fair.

Is it obvious that I'm not a socialist?

In the beginning the universe was created. This has made a lot of people angry and been widely regarded as a bad move.
-Douglas Adams

[ Parent ]

Objections (2.71 / 7) (#55)
by trhurler on Sun Mar 27, 2005 at 03:01:29 PM EST

(This is a repost of an editorial comment thread, collapsed into one big post. I have adapted my argument somewhat, but the general thrust is as it was. I doubt anyone can come close to answering it.)

The problem with personality testing is not that people impose consistency on reality. It is that they impose imagination on reality. Almost all people have almost no capacity for honest introspection. Therefore, self reporting is basically useless unless you are already quite certain you can trust the subject and that the subject can trust his own judgement, both of which are among the things you're trying to determine.

(Original poster replied at this point that for some "important" tests, others observe the subject and report their opinions in addition to the test subject's replies.)

Other people tend to see you either as you see yourself, which is uninteresting, or as the exact opposite of what you see yourself as(which is no more interesting.) In fact, I have found that I can trivially convince anyone ranging from psychologists to random people on the street that I'm anything they want me to be(and I can make them want me to be almost anything if they're trying to find out what I am.) Of course, this may simply be a result of me being substantially more self aware and intelligent than they are, but the point remains: personality tests will give you an answer because they're designed to give you an answer.

Also, most people's insight into themselves is not just limited - it is wrong. They convince themselves that they are what other people tell them they are. They convince themselves that they are something they admire in other people. They convince themselves that they are what they fear. They convince themselves that they are whatever they think will lead to success. The point is, what they see as introspection is actually more akin to rationalization. Asking these people to describe themselves is useless, and asking other people to describe them is only going to lead to a description one of two things: either it will be the mask the person is wearing, or if that mask is rather poorly constructed, it will be whatever mask the person thinks he sees under it(which will be at best a guess anyway.)

The reason most companies don't bother testing to see who would make a good leader is twofold: first, because they know that people act vastly differently depending on their circumstances(see the Milgram prison experiment for one example, but there are other less dramatic cases also.) Second, because they know that most leaders in a hierarchy spend most of their time following anyway, and that characteristic is generally the more important of the two.

If you want to know why the social sciences are basically a failure, answer this: What would be the observable difference between these two scenarios:

1) Companies that do personality screening receive the benefits you describe because personality screening works really well.

2) Companies that do personality screening receive the benefits you describe because the mere fact of doing personality screening indicates a more rigorous overall selection process.

Cliff's Notes on the answer: There is no observable difference. The overall reason the social sciences are a failure is that it is indescribably difficult(perhaps impossible,) to design truly controlled experiments to test hypotheses of limited scope within these sciences. This has always been true, and apparently always will be.

'God dammit, your posts make me hard.' --LilDebbie

Your premises leave a little to be desired (2.00 / 3) (#56)
by roffe on Sun Mar 27, 2005 at 03:29:03 PM EST

Paul E. Kline, one of the most distinguished researchers of the field, noted that psychometrics - i.e. the science of measuring personality - is a pseudoscience. He gave lots of convincing reasons for this.

The only consolidating factor is that psychometrics, in fact, works. For many purposes, it is possible to use this pseudoscience to predict human behavior to a satisfactory extent. The correlations are often low, and ability and intelligence tests predict job performance better than personality tests do.

Humans have a capability of introspection sufficient for the construction of useful personality theories and useful personality tests. But your point, in fact, is a good one: even a poor personality test, including astrology, can create personality profiles that people recognize themselves in. This is the fundamental difference between good and bad personality tests: both provide profiles that the test taker agrees with, but the good tests provide profiles that are much more correct and therefore much more useful.

Rolf Marvin Bøe Lindgren

[ Parent ]
Er... (3.00 / 4) (#61)
by trhurler on Mon Mar 28, 2005 at 12:24:04 AM EST

I don't think you get it. There is no way to measure "correlation" of a personality test's results to the actual personality except by the observations of people who are demonstrably unreliable judges(either the subject, who we know to be horribly unreliable, or observers, who we know can be trivially deceived.) Whatever you are describing as "correlation" is in fact guesswork.

If you mean that personality tests correlate to job performance, that might only mean that a person who knows how to act a certain way is better at a certain job, and might have no connection to any underlying personality traits, and again, if this is true, there would be no observable difference.

Furthermore, the Milgram experiment is all the proof anyone needs that personality depends on circumstances and can change wildly multiple times in a very short period of time.

The social sciences are a failure. I can keep hammering you with this until you either admit it or give up. There is NO WAY to create a falsifiable hypothesis of well defined scope that allows for a legitimate control in social science experiments that have interesting consequences.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
Interesting perspective (none / 1) (#62)
by roffe on Mon Mar 28, 2005 at 03:32:32 AM EST

It may be that the inter-rater reliabilities obtained are due to some sort of consensus-creating mass delusion. The fact that it's possible to make useful predictions based on them makes the notion questionable.

Rolf Marvin Bøe Lindgren

[ Parent ]
No (3.00 / 3) (#86)
by trhurler on Tue Mar 29, 2005 at 01:39:03 AM EST

It isn't a mass delusion. It is much simpler than that. What you don't want to admit is that what you call personality is not only a set of learned behaviors, but that people can pick up and drop them not only over time, but several times in one day. Among a set of friends who all look up to you, you are one person. Among a crowd of people to whom you are a novice at something, you are another person. You haven't really changed in any meaningful way, but the way you interact with others, deal with adversity, feel about yourself, and so on HAS.

To make matters worse, anyone who makes a reasonable study of the subject can learn to manipulate the image others have of him. I can trivially convince you(in person, that is, and if you don't already know me,) that I'm introverted, bookish, nervous, and easily upset, but that's not true. I can just as easily convince you that I'm the King of New York, A number 1! That's not true either. If you're not the type to pay too much conscious attention to how you react to others, I can probably pull off both of those within a week without you even noticing.

I would guess that most people don't really consciously realize the differences in their behavior in different situations. However, think about the difference between what a cop would think of your behavior having pulled you over(you're probably quiet and deferential,) and what a child would think of your behavior if you were his legal guardian, for two extreme examples. You are NOT the same person all the time, by choice or otherwise.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
People are not the same all the time (none / 0) (#92)
by roffe on Tue Mar 29, 2005 at 07:12:16 AM EST

That's why personality tests don't predict perfectly. We do, however, have a lot of knowledge now about the relative situational robustness of the factors.

Rolf Marvin Bøe Lindgren

[ Parent ]
Patterns (none / 0) (#141)
by daigu on Fri Apr 08, 2005 at 06:54:29 PM EST

I can trivially convince you(in person, that is, and if you don't already know me,) that I'm introverted, bookish, nervous, and easily upset, but that's not true.

Could you do it over the course of a year in a wide variety of circumstances? Yes, you can pretend to be anyone you want. You can play a certain role - like you surely do on Kuro5hin - that may not be representative of how you interact with people elsewhere.

Personality is the aggregate. You can mask it. It may to some degree mask itself as it varies in certain circumstances. However, these things do not establish that it does not exist.

[ Parent ]
Too strict... (2.80 / 5) (#75)
by curril on Mon Mar 28, 2005 at 01:47:30 PM EST

By your criteria, even intelligence tests are worthless because you can't differentiate between someone who deliberately scored lower, someone who tested at their best level, and someone who scored higher because of prior exposure to the test questions or interaction with the experimenter.

Yes, people's personalities are fluid and likely to change with time and circumstances, but so is water. It still has interesting properties despite the fact that it isn't meaningful to talk about it's shape. Just because people can alter their behavior doesn't mean that you can't examine the characteristics of that behavior at certain point in time, or that you can't look at aggregate behaviors and how they might change.

And yes, it is difficult to create a reproducible, falsifiable experiments in the social sciences, but that doesn't mean that there is no useful information to be gained or that it is pointless to try.

I find this study on inattentional blindness to be scientific and interesting. A personality test isn't significantly different. They main problem with personality profiling is the logical error of inferring the specific from the general. A good personality profile is a general set of behaviors that are statistically more common in people who possess that profile. To assume that a given individual is going to behave in a given way under a specific circumstance because of their profile is an error in judgment.

[ Parent ]

But... (2.50 / 2) (#85)
by trhurler on Tue Mar 29, 2005 at 01:27:09 AM EST

Have you noticed that if you can't infer specifics from generalities, then the people who say they have a use for personality profiles are wrong? That's exactly what they want them for.

Furthermore, while IQ tests are by no means perfect, one can at least assume that a test taker who has not seen the test before and whose interaction with the person giving the test is monitored in some way will not score too highly, and I've never felt that I was treated unfairly by such a test myself in any case. On the other hand, the underlying premises of a personality test just don't hold up to any real scrutiny. People don't have just one personality, and knowing someone for 50 years is barely enough to really know his personality well, let alone observing him for an hour or having him describe himself in a predefined way.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
Personality tests can still have merit (2.50 / 2) (#106)
by curril on Tue Mar 29, 2005 at 01:21:49 PM EST

you can't infer specifics from generalities, [but] That's exactly what they want them for.

True. People often prefer that others behave in predictable, identifiable ways. Such people will use personality profiles as a discriminatory tool. As you have noted, this is inappropriate since people can choose to behave as they wish. A "good" personality profile is no guarantee that a person will behave as you want, nor will a "bad" profile prevent them from acting appropriately. If standardized behavior is truly necessary in a particular situation, it better to test to see if the candidate is capable of such behavior and then put in place various rewards and penalties to encourage compliance.

the underlying premises of a personality test just don't hold up to any real scrutiny.

I suppose it depends on what you consider to be the premises. For a given situation, there are a number of possible behaviors. If you look at a lot of different behaviors, it is reasonable to try to group them into certain categories, say "extroverted" or "disagreeable". If you look at a set of individuals over time and notice a preponderance of extroverted behavior, it is reasonable to label that set of people extroverted.

It's not really any different than calling someone a smoker. Smokers don't smoke all of the time, they don't have to smoke (addiction issues aside), and they may choose to stop smoking altogether. Nonetheless, it is a useful label. It can also be used inappropriately, such as in the case of refusing to hire smokers even if they agree not to smoke in the workplace.

Personality types are useful for helping to categorize and understand different modalities of behavior. But, as you say, they are often used as an excuse to stereotype. Perhaps it would be better if social psychologists focused less on personality tests and the labeling of individuals' personalities, and instead focused more on the study of the persistence of various behavior classes as well as their relationships.

[ Parent ]

Hmm (none / 1) (#128)
by trhurler on Fri Apr 01, 2005 at 01:56:46 AM EST

You know, somehow when I say this same thing with different words, people get upset.

"Well, I don't discriminate against black people. But, I do happen to know that young black males are statistically more likely to commit violent crimes than other people, and I try to make good use of that information."

(Please note, this is not my actual position.)

Put simply, if personality tests aren't useful for discriminating against people, then they aren't useful.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
On IQ tests (none / 0) (#115)
by tetsuwan on Wed Mar 30, 2005 at 08:20:22 AM EST

Have you encountered meta-testing? That's when in a higher level IQ-test, the questions get gradually more complex until they're uncomprehensible (for me) after 2/3, after which the questions become easier again. There are in fact several test-specific skills.

Njal's Saga: Just like Romeo & Juliet without the romance
[ Parent ]

Er (none / 1) (#127)
by trhurler on Fri Apr 01, 2005 at 01:46:32 AM EST

Most such tests explicitly state that if the test taker knows how the test works in advance, they don't provide accurate results. If the test taker does not know how they work in advance, it doesn't matter. Besides, if you give someone with a given IQ a dozen different decent quality IQ tests, what you'll find out is that his first couple of scores will be a bit off, but after that, once you normalize the scores to be comparable to each other, people score amazingly consistently. Given that almost all people will do as well as they can on such a test, this is a meaningful result. On the other hand, I can have whatever image of myself I like, and a personality test is not going to tell you whether it is realistic or not.

There's just no comparison here.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
yup (none / 1) (#130)
by tetsuwan on Fri Apr 01, 2005 at 04:37:19 AM EST

This is what I've found, too. But my experience with personality tests is very limited. I just heard this buzz about it being hard to cheat on the new tests if combined with an in depth interview based on the results. But I suspect that with just a little awareness of how the test works you can screw the test any way you want.

The problem with the social sciences, stated simply, is the problem of interacting kinds (see Hacking). There are truths to be found in the social sciences, they're just more limited in scope. To say that all social science is a waste of time is to say "let's not talk about humans, society or culture". Bit strange conclusion, I think, taking into account the extreme antropocentrism of you and almost everybody else.

Njal's Saga: Just like Romeo & Juliet without the romance
[ Parent ]

-1, Blow-hard... (1.66 / 6) (#59)
by sudog on Sun Mar 27, 2005 at 10:12:16 PM EST

... with no discernable, convincing argument based on something other than simple egotism.

[ Parent ]
Ah (none / 0) (#60)
by trhurler on Mon Mar 28, 2005 at 12:18:51 AM EST

So then you have a reply, or you just don't like what I have to say?:)

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
'Ah' blahblahblah (none / 0) (#67)
by Harvey Anderson on Mon Mar 28, 2005 at 09:14:12 AM EST

It's how you say it.

[ Parent ]
No topical reply necessary. (none / 1) (#88)
by sudog on Tue Mar 29, 2005 at 02:57:42 AM EST

Unlike your troll victims, I refuse to participate in your typical war of semantics when you seem to think your mutable diction and reinterpretable intent are a substitute for expertise.

Aim your misanthropic autoeroticism elsewhere, you'll get no further satisfaction from me.

[ Parent ]

Big words... (none / 1) (#131)
by Ramrod on Fri Apr 01, 2005 at 07:16:49 AM EST

Nice 'dic'tion I see you have a flair for extravagant language. I will not flame you because I myself suffer from maledictaphobia.

[ Parent ]
Testing Personality Tests (2.50 / 2) (#71)
by daigu on Mon Mar 28, 2005 at 11:28:24 AM EST

Let me guess...you score low on openness?

I can think of a number of ways to test effectiveness. None of them involve looking at it on the company level. I think the problem in your scenarios is how you are framing them rather than intrinsic to testing for this variable.

For instance, you could do a test across companies for a particular type of job and with a large enough sample size you could look for correlation with personality testing. You could have all applicants take the test as part of the hiring process and have someone look for patterns and make predictions. You could test two different groups of people making hiring decisions - within the same departments or within the same companies - and have different ways of cutting the results (double blind, randomized false results or what have you). There are a wide variety of options.

With enough research, you could eventually get to a reasonable confidence level as to whether personality tests are effective or not - based on different metrics such as longevity, selection criteria or what have you. Devising tests for it do not seem any more difficult than medicine or other areas of scientific inquiry where things aren't as neat as say physics.

As for the reliability of self-reporting, who is in a better position to determine whether I enjoy interacting with people or not? Why would you think that I (or anyone) might be confused on this issue?

Granted, there are some reporting errors that stem from questions like: "I care about others," because no one wants to say that they categorically do NOT care. But, you could also correlate for accuracy by doing tests that involve friends, relatives, acquaintances and complete strangers in a variety of situations. Or, you can test answers to the same question across the population of test takers. You might even want to check to see how personality type of these people impacts their interpretations of others.

To say it is useless is a comment on your view of science. I'd also say that it is more likely that companies do not do personality tests for top jobs because people are selected for these jobs for political or personal reasons or based on their experience. It is not a rational decision that lends itself to a personality test - and even if you were comfortable with the validity of personality tests, it would likely be subverted for political reasons in that context.

I could go on. But, it is probably enough to say that you should give this more thought.

[ Parent ]
Actually, (2.50 / 2) (#87)
by trhurler on Tue Mar 29, 2005 at 01:48:18 AM EST

I score above the 95th percentile in extroversion and openness, and I'm at least average in all the categories as they're typically described. Amusingly, I score around the 75th percentile in agreeableness, which ought to tell you how useful these tests are. (And yes, I'm meticulously honest with the answers.) My lowest score is typically in emotional stability, because while I don't worry much, certain things anger me very easily. Still, most people think I'm calm and confident, and while close friends or k5 readers would say otherwise, an observer who didn't know me and only had a short time would conclude that I may never have been angry in my life. :)

Catch me on a really off day though, and I probably would score below the 10th percentile in extroversion. How useful is that?

I learned something a long time ago. By the standards of the hard sciences, the social sciences are a complete joke. Also, your average social sciences major is not bright enough to have gone into physics, yet not personable enough to have gone into an arts or other "nontechnical" field. Does "stupid and socially inept" sound like a promising start in a field that presumes to understand people and their interactions?

You are too easily fooled by the illusion of legitimacy. Here's the best advice you'll get in your whole life: Most experts are anything but.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
Bogus Answer (none / 0) (#98)
by daigu on Tue Mar 29, 2005 at 09:31:50 AM EST

People do not swing on the extraversion or introversion scale.  This is one of the things you are inclined to or not. The problem with your answers is that you respond how you feel based on a given day or moment - rather that accurately describing your tendencies. This is going to give you the results your are describing.

My experience is that every personality test pretty much agrees with each other. I'm INTJ. While I have extraverted moments on particular days, it is not my behavior in the main.

As for experts, I'm not relying on experts. It corresponds with my own experience and the experience of other people I know. This suggests to me that if you do get the results you describe, it is because you are delibrately answering the questions to achieve them.

You can get bad results in the lab, just as you can in social science. It's called human error - and it looks like it is higher for you than others. Why is that you think? I'd guess it is because you are actively introducing error because you have a bone to pick.

[ Parent ]

This only even makes sense (none / 1) (#126)
by trhurler on Fri Apr 01, 2005 at 01:40:50 AM EST

if you take it as a given that these "tendencies" actually exist, and aren't just manufactured properties people ascribe to themselves as part of their need to "figure out who I am." If you don't take that as a given, the tests are meaningless. The fact that people answer them in different ways does not mean that they're describing things about themselves that are "real." It just means they're answering questions in a way they think describes "who they are." You might as well ask a homophobe if he would enjoy sucking a cock.

'God dammit, your posts make me hard.' --LilDebbie

[ Parent ]
Tendencies (2.50 / 2) (#139)
by daigu on Thu Apr 07, 2005 at 07:26:40 AM EST

While provocative, your comment is also obtuse. You are essentialy saying that there is no such thing as personality, and therefore, it is pointless to test for it.

To refresh your memory, personality is defined as: the complex of characteristics that distinguishes an individual or a nation or group; especially : the totality of an individual's behavioral and emotional characteristics.

The easy reductio ad absurdum argument comes down to two questions: Can you differentiate between people or groups based on their behavior patterns? If so, can you test for these differences?

I can tell the differences and I can also test for them - generally by interacting with a person or group. Most people would agree that they have this capability as well. The fact that there is a word for it is evidence most people can distinguish behavioral and emotional characteristics in others.

So, you need to do a lot of work to explain why you cannot (I submit it perhaps may be a disability on your part) and leaves you with a lot of work to support your argument that these tendencies do not exist.

[ Parent ]
Thanks! (none / 0) (#63)
by roffe on Mon Mar 28, 2005 at 04:17:16 AM EST

This article was in fact written eight years after I left college. Good to learn that I haven't lost my touch.

Rolf Marvin Bøe Lindgren

Personality tests are a tool of Wal-Mart, Manpower (1.53 / 15) (#65)
by freemumia on Mon Mar 28, 2005 at 08:36:00 AM EST

Interesting that you should mention "PERSONALITY". Like I've said, it looks like we can say goodbye to alternative lifestyles and pure rivers!! Goodbye!!! Fun while it lasted!? Don't give up hope. Sanctity of life, indeed! Wal-Mart can wipe out 41,630 BILINGUAL African-American wetlands in Kabul, all in the name of "regime change"? Obviously, that makes it okay, then!! (I am being sarcastic!!!! It is not okay, you understand.) I suppose, I am not one of Pat Robertson's outpost of single-celled henchmen!!? I reject graft and hypocrisy!!! Don't make me laugh!! The truth is at paganafricanamericansforjustice.org! What next!? Will the morons come to repress me for being a Native American!? Will my atheist friends now be electrocuted just because they're woodlands!!!? Really it's open season on nudists!! We demand the U.S. get out of Brooklyn!!!! As a vegan Iraqi, as everybody knows I am humorless!!! Cowboys! SINCE 1952, 38,237 HERBIVOROUS whales have been denied in FRANCE.

WTF? (3.00 / 3) (#70)
by Lacero on Mon Mar 28, 2005 at 09:49:32 AM EST

Is this some kind of spy code? Posting illiterate rants on kuro5hin to communicate with cells all over the world?

If not, can I ask for more information on Pat Robertson's outpost of single-celled henchmen? It sounds fascinating.

[ Parent ]

Jungian Personality Profile (none / 0) (#68)
by JosephK on Mon Mar 28, 2005 at 09:24:33 AM EST

In college I took a test called the Jungian Personality Profile, and the only part I remembered was that I scored 99% logical, 1% emotional on a sliding scale. What am I? Spock?
HTML is Dead.
no, just an asshole. (1.20 / 5) (#82)
by Cloud Cuckoo on Mon Mar 28, 2005 at 07:48:48 PM EST

sorry buddy.

[ Parent ]
Hemorrhoid (none / 1) (#95)
by JosephK on Tue Mar 29, 2005 at 08:47:21 AM EST

Well, that pretty much makes you a hemorrhoid, now doesn't it?
HTML is Dead.
[ Parent ]
touche' (none / 0) (#99)
by Cloud Cuckoo on Tue Mar 29, 2005 at 11:04:55 AM EST

[ Parent ]
WTF? (none / 0) (#72)
by Mr.Surly on Mon Mar 28, 2005 at 11:34:23 AM EST

How did this editorial abortion make it past editing:

The story begins with Gordon Allport and H.S. Odbert, who hypothesized that

    Those individual differences that are most ...

Why does the sentence end in the middle, and a new paragraph start?

I did it on purpose. (none / 0) (#74)
by roffe on Mon Mar 28, 2005 at 12:14:53 PM EST

The "aborted" sentence introduces a quotation.

Rolf Marvin Bøe Lindgren

[ Parent ]
There's no indication of that. (none / 1) (#77)
by Mr.Surly on Mon Mar 28, 2005 at 03:00:23 PM EST

Use quotes, or italics, or indentation, or all 3.

[ Parent ]
There is _too_ indication of that! (none / 0) (#78)
by roffe on Mon Mar 28, 2005 at 03:34:49 PM EST

The quote is set in <blockquote>, which renders into italics on my system.

Rolf Marvin Bøe Lindgren

[ Parent ]
Alrighty, I have an answer. (none / 0) (#105)
by Mr.Surly on Tue Mar 29, 2005 at 12:40:38 PM EST

In Opera and Firefox, the <blockquote> does not show, despite being in the source.

Turns out that the Google ad somehow screws it up.  If I download the source of the page, and only disable the Google JS download, the blockquote works okay.

[ Parent ]

I never see the point of personality tests (2.75 / 4) (#80)
by TheGreenLantern on Mon Mar 28, 2005 at 05:29:21 PM EST

Essentially the message always boils down to: "You are beautiful and unique snowflake, but we're going to find a way to catagorize and label you anyway"

It hurts when I pee.
modern culture = plenty o' wank time. (3.00 / 4) (#81)
by Cloud Cuckoo on Mon Mar 28, 2005 at 07:47:43 PM EST

hence behavioral science. My personal system is pretty simple. Everyone I meet fits into one of the below categories:

1. Asshole.
2. Cool Like me.  

[ Parent ]

You forgot (3.00 / 2) (#104)
by ghjm on Tue Mar 29, 2005 at 12:29:54 PM EST

3. Potential Mate.

Other than that, I think you're on to something.


[ Parent ]

Skeptical (2.66 / 3) (#83)
by bugmaster on Mon Mar 28, 2005 at 09:18:41 PM EST

Sorry, I remain very skeptical about all these personality tests, whether they're lexical, Meyers-Briggs, or whatever.

First of all, what exactly are these tests measuring ? Sure, a test can tell you that person X has a low "agreeableness", but how do you actually test that ? Can you make any concrete predictions of a person's behavior based on the results of the test ? I am talking about specific predictions here, not horoscopes to the extent of "you like good things, and tend to avoid bad things".

Secondly, where is the hard cold data that validates the design of these tests ? For example, consider red/green color blindness. Such color blindness is much common in males. We can posit a hypothesis that the color blindness gene is carried on the X chromosome, of which males only have a single copy. By analyzing the DNA of lots of people (male and female, colorblind and not), it was possible to discover the genes responsible for color blindness.

Where is something like this in relation to personality tests ? All I see are lexical analyses of adjectives, which are fun and all, but don't really predict anything verifyable. It's fine to say, "most ENTJ types are better leaders", but what does this actually mean ? How do you measure leadership with the same precision that you can measure weight, length, conductivity, etc. ?

So, in summary, personality tests make vague predictions that cannot be verified experimentally, because there's nothing definite there for them to verify. They are similar to horoscopes -- scientific-sounding horoscopes, but horoscopes nonetheless. This has been my perception, anyway.

Easy (none / 0) (#89)
by mariube on Tue Mar 29, 2005 at 03:03:23 AM EST

Look at the article title. It's all about describing and predicting behaviour for a group of people. It doesn't matter what you call the variables -- the key is that it's possible to create a test which gives you five numbers. It has then been statistically proven that it's possible to predict behaviour based on those. You can call them whatever you want, but correlations are correlations.

[ Parent ]
Correlation is not causation (none / 0) (#96)
by redelm on Tue Mar 29, 2005 at 09:30:18 AM EST

Statistics is a grossly misunderstood science. It is descriptive, not prescriptive. An r2 of 0.8 is pretty miserable if you look at a dataplot.

[ Parent ]

That's what we have longitudinal studies for (none / 0) (#100)
by roffe on Tue Mar 29, 2005 at 11:09:29 AM EST

There are (expensive and time-consuming) research methods that allow inferences of causation.

Rolf Marvin Bøe Lindgren

[ Parent ]
Post hoc ergo propter hoc (none / 1) (#101)
by redelm on Tue Mar 29, 2005 at 12:02:50 PM EST

Time is not an isolator unless you can rule out all other causative agents (ceteris paribus doesn't often exist). Causation is normally established by double-blind randomized imposed trials. This is not possible with personality, nor ethical with many human conditions.

Of course, people speculate all the time.

[ Parent ]

Yes, but ... (none / 0) (#102)
by roffe on Tue Mar 29, 2005 at 12:11:52 PM EST

If I consistently see that people with a certain profile do better on a task than all other people, I will infer that other people with the same profile will show a similar performance. I don't have to bother with causality until people with other profiles try to change their profiles (e.g. through coaching, psychotherapy, or what have you) into more successful profiles. Then, there would be a question of whether a "contrived" profile is functionally the same as a "natural" one. Again, empirical studies would help decide this. Where is causation actually relevant here?

Rolf Marvin Bøe Lindgren

[ Parent ]
You brought it up ... (none / 0) (#107)
by redelm on Tue Mar 29, 2005 at 02:40:23 PM EST

"infer causation" in your grandparent post. Here, you're talking about predicting likely performance. That of course can be done, with determinate (often large) error margins.

[ Parent ]

Okay (none / 0) (#103)
by ghjm on Tue Mar 29, 2005 at 12:22:48 PM EST

but there's also a demonstrable statistical correlation between being black and under-producing at work. Does that make it okay to make hiring decisions based on race?

Answer: No, because the statistical correlations applicable to large groups are not fair measures of individual capability or suitability. Because of this, we have established that hiring should be (at least) race-blind.

The same argument applies to discriminatory hiring that penalizes, for example, introverts.


[ Parent ]

Operationalizing Leadership (none / 0) (#91)
by roffe on Tue Mar 29, 2005 at 05:24:04 AM EST

Clever people have been thinking long and hard about operationalizing leadership. When leaders are evaluated, they are scored on such variables as finishing on time, finishing on budget, customer satisfaction, subordinate employee turnaround, etc.

In most businesses, leaders (i.e. managers) are given bonuses for performance. These bonuses are based on objective measures of leadership.

See? It's not so difficult.

Rolf Marvin Bøe Lindgren

[ Parent ]
Ability, not personality (none / 1) (#116)
by bugmaster on Wed Mar 30, 2005 at 11:54:47 AM EST

In the business world, leaders are scored on ability, not personality. If you managed to finish on time and on budget (by pulling the wool over the developer's eyes, and keeping them in the office 24/7, but I digress), then you win, otherwise you lose.

In order to win, a manager would require more than just personality; he'd require at least a basic knowledge of the project, math skills to deal with the budget/time constraints, a lot of friends in the right places to deal with his competition... Yes, personality is a part of this all, but it's not the only part, and probably not even the largest part.

In other words, what is meant by "leadership" in the business world is very different from what is meant by "leadership" in the made-up world of personality tests.
[ Parent ]

Personality tests deal with the real world (none / 0) (#121)
by roffe on Thu Mar 31, 2005 at 05:16:04 PM EST

In the real world, leaders are scored on personality tests, but also on ability tests. The present article is on personality tests, hence the emphasis. There are some tough test batteries out there.

Rolf Marvin Bøe Lindgren

[ Parent ]
Would you read MBTI-derived horoscopes? (none / 1) (#109)
by it certainly is on Tue Mar 29, 2005 at 07:32:24 PM EST

Just imagine it, the woman's weekly or newspaper has 16 entries giving arbitrary advice for the week, based on your Myers-Briggs personality type rather than your star-sign.

Wouldn't that be so much more fun? As a Scorpio, I don't believe in horoscopes, but the INTP in me wants sagely advice from complete strangers. Go figure.

kur0shin.org -- it certainly is

Godwin's law [...] is impossible to violate except with an infinitely long thread that doesn't mention nazis.
[ Parent ]

Here's one way to look at it (none / 0) (#124)
by onemorechip on Thu Mar 31, 2005 at 10:44:52 PM EST

Compare psychology to other inexact sciences. Biology, for instance. Living things are classified into kingdoms, phyla, classes, and so on down to species and subspecies. If we know that a certain creature is a sperm whale, can we then state exactly how long it will live, exactly where it can be found, how much it weighs, or anything else? Or can we only predict these things approximately?

Classifying people according to how they answer a bunch of questions on a test is also inexact, but it can provide some general insights.

Of course there is the additional issue of cultural bias. I wonder how a Maori or an Inuit would answer that question about "feeling blue".

I did my essay on mushrooms. It's about cats.
[ Parent ]

team selection|act|aid gut| academic filter (none / 1) (#93)
by jago25 on Tue Mar 29, 2005 at 07:35:41 AM EST

I know you hate this sort of thing but here in the UK we get flung together with random people and this is MUCH worse. My god I've been in some shocking teams; 5 leaders all fighting, 5 imaginatives all dreaming, groups of workers arguing over fundamental beliefs that split the company right down the middle; do we believe in liberty? How many times have you seen these arguments in your company? It really does pay to put together a varied and compatible team and I'd do anything to be in one.

I can see lots of objections to these tests but I don't think these objections came from rational thought.

Nobody likes to be classified but you are not powerless against the test, you are powerless to see what they REALLY want in the candidate.

There have been times in the past when I have actually answered such screening questions truthfully - oh how ignorant I was. For example, I applied for a job at a local DIY store and one of the auto phone screening questions was `Do you like to present yourself smartly?` well, that wasn't easy and not relevent! This is a DIY store - the experts don't dress smartly! And what does smartly mean?; intelligently or formal?
And the test asked me nothing about my experience of DIY terminology; i.e. drill bit sizes, metric and imperial measurements etc

Instead what I needed to do what find out exactly the sort of person they want and act to that goal . This screws the whole thing. The selected candidates are the liers. You get a workforce full of actors & liers. And the few that aren't may be the wrong people for the job because you've spent your time selecting people for the job rather than deciding the sort of person you want.

Personality tests skit the bigger picture of WHO IS THE RIGHT PERSON FOR THE JOB? A used car salesman is different to a wheelchair salesman, yet the used car salesman would win in these tests.

Nevertheless I would use these tests but only as a backup to my gut feeling - to aid the human process rather than weeding out CVs beforehand or similar.

When these tests combine with the academic filter we got a bigger problem:

I'm finding it very hard to get the job that these tests say suit me. The reason why is because I'm not very good at academic study, yet I need to do this to get the job.

Perhaps these tests could be used by universities to encourage those who fit the course so we don't waste our time in a career that we want to do, perhaps are pushed to do but that doesn't fit.

There is a lot of bad test use out there (none / 0) (#94)
by roffe on Tue Mar 29, 2005 at 07:42:10 AM EST

Most of the personality tests used in team building and job selection are made by people with little understanding of proper test construction. They manage to sell their tests to managers with even lesser knowledge.

It's difficult to fool a good test adminstered by a competent person. Good testing, however, is not the rule.

Rolf Marvin Bøe Lindgren

[ Parent ]
I'd love to see this article in 2105 (none / 0) (#108)
by jolly st nick on Tue Mar 29, 2005 at 02:51:07 PM EST

It's interesting to speculate how an article like this would look if it were written a hundred years from now. I suspect that we won't look at personality traits as fundamental, but as results of a combination of brain features and environmental factors. I also suspect that this view of matters will not lead to neurological determiism, but more effective tools for modifying long term behavior patterns.

Take the following cluster personality traits: risk taking/stimulus seeking, carelessness, inconsideration, rudeness. These all can result from ADHD, which will probably turn out to be a fairly well characterizable set of neurlogical features that lie at one end of the normal spectrum. Furthermore, not only can ADHD be managed with medication, people can be given organizational trainign that allows them to function higher on the conscientiousness scale, and "people reading" training to improve their ability to track others' reactions to them.

frankly baffled (none / 1) (#110)
by massivefubar on Tue Mar 29, 2005 at 08:18:24 PM EST

I do not see anything more useful or informative in this set of pigeonholes than in the old descriptions of personality based on the four humors or the 12 signs of the zodiac. The only difference I can see is that this particular method of describing personality is less interesting and hence less likely to be an accurate reflection of the true variety and weirdness of actual human beings.

Offhand, the descriptions sound downright silly. In agreeableness, it is suggested that the agreeable person is more likely to be submissive and concerned with the well-being of other people. That is always possible, I suppose, but I believe the bard had it right when he said that a man can smile and smile and be a villain. Plenty of "agreeable" people who never contradict a word you say are simply passive-aggressive or just have a different definition of what being polite entails. It doesn't mean that they don't go ahead and do as they damn well please. Ask any Southern belle.

There is still a brain (none / 0) (#120)
by paranoid on Thu Mar 31, 2005 at 11:19:56 AM EST

The brains of different people are different. Simply put, some modules are more developed, some connections between modules are stronger, some are more excitable, etc. This means that there are objective reasons for personality differences. And it makes perfect sense that these differences could be used to plot the character in a N-dimensional space. Whether these particular five factors are the best way to describe the differences, I don't know, but assuming they have been tested, they might be a decent approximation.

[ Parent ]
So what? (none / 1) (#132)
by MrHanky on Fri Apr 01, 2005 at 05:48:13 PM EST

Phrenology is more objective than Big 5 or any other sort of personality testing, but it's still more wrong, both ethically and scientifically. Personality isn't an object.

Of course, most (all?) classification is to some degree arbitrary. You just find some differences to order by, or a descriptive language that demands classification by so-and-so, etc. That applies even for geometry: Sometimes, a line is the shortest route between to points, sometimes it's a series of points. A plane, flat surface can be rectangular or circular, but it doesn't always make a difference.

Some sorts of classifications and descriptions are better than other kinds, though. And when it comes to psychology, the usefulness of a theory often seems to be inversly proportional to its precision.

"This was great, because it was a bunch of mature players who were able to express themselves and talk politics." Lettuce B-Free, on being a total fucking moron for Ron Paul.
[ Parent ]

I like those personality tests (2.33 / 3) (#111)
by Armin Hardwood on Tue Mar 29, 2005 at 08:56:29 PM EST

that tell me which video game character i would be. they're far more insightful than any of this big 5 nonsense.

What "research"? (none / 1) (#112)
by badtux on Tue Mar 29, 2005 at 09:30:23 PM EST

Just curious. I am aware of the state of the art in personality test research. Basically, it is rather more limited than what you imply. There is no such thing as a personality test that is reliable for all persons of all ages and all cultures. What a personality test measures is how well your test answers agree with what third parties in your environment would think about you if given an opinion survey. They are normed by giving opinion surveys to the family and friends of those who take the tests (people who typically are college students attracted to the test norming sessions by promises of money or extra class credit). It is very rare for any personality test to be normed against anybody other than middle-class, predominantly white college students, and if you are not a middle-class white college student the results of a personality test have about as much validity as a Ouiji board. Sorry, that's the truth -- statistically speaking, these tests simply are not reliable indicators of personality traits for the majority of the work force.

Given the limitations of the norming processes for these tests, I would state that I would be strongly suspicious of any "research" that says that companies that use personality tests work better than comparable companies that do not unless said research had passed peer review muster and been replicated multiple times by people with no interest in the outcome. "Research" produced by the actual publisher of the test is, basically, utterly useless, typically suffering woefully from selection bias, lack of peer review, inadequate control group, and expectancy effects on the part of the researchers affecting the quality of the data gathered.

- Badtux the Research Penguin
In a time of chimpanzees, I was a penguin

There are far more researchers than tests (none / 0) (#123)
by roffe on Thu Mar 31, 2005 at 05:20:20 PM EST

The objective research you ask for has been performed many times.

Rolf Marvin Bøe Lindgren

[ Parent ]
When? (none / 1) (#133)
by badtux on Mon Apr 04, 2005 at 05:07:07 PM EST

That certainly was not the case ten years ago when I did my last survey of the literature as part of a graduate research course. The only researchers actually doing research in the field were either in the employ of the test makers, or were college professors under "publish or perish" measure who only gave the test to college students recruited from the college campus. I looked at the actual design of the studies and they had numerous flaws which were obvious even to a graduate student.

- Badtux the Educated Penguin
In a time of chimpanzees, I was a penguin
[ Parent ]

Where did you look/what did you look for? (none / 0) (#140)
by roffe on Fri Apr 08, 2005 at 12:08:44 PM EST

If you look at the latest research in particular the meta-analyses by Barrick & Mount et al., you'll find that they cite an awful lot of research - the field has exploded during the last ten years.

Rolf Marvin Bøe Lindgren

[ Parent ]
Meta-analysis is no analysis (none / 1) (#143)
by badtux on Sat Apr 09, 2005 at 01:50:15 PM EST

One thing I wish to warn you about is placing too much credence upon meta-analysis papers. These are generally released under publish-or-perish pressure and are, in my personal opinion, generally garbage because of the following issues that afflict most of them:
  1. They give equal weight to poorly-designed studies and to well-designed studies.
  2. They give equal weight to a study with 5 experimental subjects as to a study with 50,000 experimental studies.
  3. They give equal weight to a study that took place over 2 weeks as to a study that took place over 2 decades.
  4. They give no weight at all to study population, i.e., if the majority of studies studied only college students taking psychology classes they do not factor this in at all but instead imply that the studies surveyed representative samples from the general population. (College students taking psychology classes are NOT a representative sample, being predominantly white and predominantly female and predominantly from middle-class WASP backgrounds).
  5. They typically make sweeping conclusions not supported by the underlying data in the individual subject studies.
  6. In order to reach those sweeping conclusions, they end up massaging the data from the individual studies in ways not intended by the authors of the individual studies, and often in ways that are not mathematically valid once you consider sample size and population issues.
That said, meta-analysis papers do serve a useful purpose -- they serve as an index to what the current state of the research is, and allow you to examine the underlying papers to see if they support the conclusions reached by the mata-analysis author. Sadly, usually they do not support the conclusions of the meta-analysis author, typically engaging in some sneaky slight-of-hand in order to use only those data points that agree with the author's conclusions and discard those data points that disagree.

Meta-analysis papers keep making it through peer review because the peer reviewers want their own meta-analysis papers approved by peer review committees since they are operating under the same kinds of publish-or-perish pressures (i.e., a nudge nudge wink wink kind of deal). But that does not make the genre particularly useful for what it was intended (i.e., as a digest of the current "state of knowledge" in a particular area) unless you do a LOT of reading to see whether the author has taken liberties with the underlying data. (A problem, to be fair, which also afflicts individual studies, but it's easier to spot things like selection bias and etc. there).

- Badtux the Research Penguin
In a time of chimpanzees, I was a penguin
[ Parent ]

meh? (none / 0) (#136)
by creeme horn on Tue Apr 05, 2005 at 09:54:30 AM EST

I just can't force myself to be a kurobot and elaborate it but hey... no they aren't. Normed. Like that. Because if you think for a second, you'll hopefully realize that tests that measure agreement between what you and others think about yourself are something entirely different and -- obviously -- useless in this context.

And you didn't seem to do much research. Personality is a fundamental psychology field, with strong activity from proving the factor stability of particular solutions cross-culturally to advancing the structural model.

And the suspicious "research" that says that companies that use personality tests work better is going on routinely as part of normal procedure of developing a selection mechanism, at the very least. Using them improves all the coefficients we use to describe success in selection. By a little :) .

I'm getting tired of geeks bashing psychology because they choose not to understand the basic assumptions.

Off course, being an industrial/organisational, the poster omited the entire Eysenck strain (the "theory-laden research") that imho had much more influence on the big5 (in terms of particular dimensions, but also because Cattell wasn't trying to find superfactors, ie fundamental differential dimensions of behavior). He also failed to note that general personality tests time and again come out as pretty much inferior methods of selecting people for work, and only add a certain weight in a larger battery of procedures.

[ Parent ]
So how many graduate-level courses (none / 1) (#137)
by badtux on Tue Apr 05, 2005 at 12:18:14 PM EST

in research have you taken? I did a graduate-level survey in this subject as part of a masters-level research course in testing and evaluation technology. Yes, the test *IS* normed, in the case of the MBTI it is normed by testing students (a large pool of grade school students, a large pool of college students) with a large pool of questions and answers, giving surveys to family and close friends of those people, then applying statistical methods to find the sets of questions and answers that best reflect the opinions of family and friends. I.e., it measures what your family and friends think about your personality. As with all normed tests, it is only valid with people who are members of the normative group. If I applied the test to, say, Hispanic youth or middle-aged business executives (which it has NOT been normed against), I could not make any conclusions about its outcome.

Look, it's not my fault that you are a gullible fool with no knowledge of research design and thus allow the test makers to fool you into believing that the test measures more than what it measures. But I did a comprehensive survey of these things ten years ago as part of a graduate course in psychological research, and what I saw didn't reassure me as to how useful these things are with anybody other than the normative group in any setting other than the educational settings that they were normed in. Now, maybe things have changed dramatically in the past ten years since I last examined the literature. But seriously, I doubt it. It simply is too profitable to produce shoddy research and then make unwarranted claims based upon said shoddy research.

- Badtux the Educated Penguin
In a time of chimpanzees, I was a penguin
[ Parent ]

I just started reading.. (none / 0) (#113)
by hangareighteen on Wed Mar 30, 2005 at 01:00:26 AM EST

Gravity's Rainbow,  by Thomas Pynchon.

good (none / 0) (#114)
by Norwegian Blue on Wed Mar 30, 2005 at 03:15:04 AM EST

Courage! In terms of sentence count, it's not such a long book.

[ Parent ]
related story: (none / 0) (#118)
by lostincali on Wed Mar 30, 2005 at 01:53:14 PM EST

http://story.news.yahoo.com/news?tmpl=story&cid=1804&e=1&u=/washpost /a4010_2005mar26

This story talks about the use of personality tests in the hiring process. I didn't know the practice was as common as the article makes it sound.

"The least busy day [at McDonalds] is Monday, and then sales increase throughout the week, I guess as enthusiasm for life dwindles."

Dodgy Experimental Design (none / 1) (#119)
by czolgosz on Wed Mar 30, 2005 at 02:08:19 PM EST

So, let me get this right: they started by picking out words in English that they thought were relevant. That's the first instance of the researchers subjectively cherry-picking the data.

They then did some kind of cluster analysis to group terms by semantic proximity (however they defined that), yielding 181 clusters. If they manually intervened then, that's another layer of data selection.

Clark then did factor analysis on the clusters and found 12 factors. So what did he do? Added four more because he thought that they belonged there. More subjective data-tweaking.

But that's not the only problem here. More fundamentally, what finally falls out of this process, even if the data reduction has much more integrity than it appears to, is nothing but a distillation of the inherent prejudices within English as regards personality.

You can see the absurdity of this if you use a physical rather than psychological phenomenon as the starting point. Such an analysis would show that the sun "rises," that night "falls," that there are "falling stars," that every month there's a "new moon," etc. These are remnants of the ancient magical, geocentric world view. But Newton's laws could never be found through such a process. All you can find is a collection of naive folk explanations. It's likely that such tests will be reproducible, but only because, due to cultural inertia and individual laziness, those are the terms people use when discussing such phenomena. This cannot disguise the fact that this is a deeply unscientific process.

As a means of taking a sort of geological core sample of pop-psychology terms over the centuries, this might actually achieve a degree of success. But reproducibility of such tests is as much an indication of the consistency of prejudice as it is a sign that they have any scientific validity.

I am no more surprised to see such tests used by employers than I am to see UK employers still so often resorting to graphology. Since so many managers are shallow and arbitrary, I'm sure they're easily convinced that both personality testing and graphology are evidence of their own brilliance and compassion, and would respond accordingly to a survey.

But let's take a more operational, Popperian tack: if these bozos are so scientific, what would the means be to falisfy their theory? Any monkey with a statistics package on a univeristy computer can be taught to massage data and drive a curve through any set of data points, but that says nothing about the correctness of their experimental design. And, more colloquially, what is the theory actually telling you, and which otherwise unexplained phenomena does the theory predict? Reproducibility and predictive power are entirely different things. With self-selected data and enough number-crunching, reproducibility isn't even that challenging a hurdle.

Why should I let the toad work squat on my life? --Larkin
Graphology vs. personality testing (none / 0) (#122)
by roffe on Thu Mar 31, 2005 at 05:19:23 PM EST

We know that personality tests predict better than graphology due to research on their relative predictive power. Even if the methods of creating tests (which have improved greatly since 1946) rely on a certain degree of subjective judgements, their results far outperform astrology and graphology. This has been tested empirically.

Rolf Marvin Bøe Lindgren

[ Parent ]
Predictive Power (none / 1) (#129)
by czolgosz on Fri Apr 01, 2005 at 02:40:52 AM EST

We know that personality tests predict better than graphology due to research on their relative predictive power.
I ask this without being in the least facetious: what is it that they predict? Or, to put it differently, what are the criteria of successful prediction?

I rather mischievously chose graphology, since that is not even reproducible. But I am concerned that the technique described will not disclose deep mechanisms, and therefore is a sort of "black box" model that has only limited or no explanatory power.

But my only involvement in psychology has been in artificial intelligence and some modeling of low-level phenomena such as vision. More of my research was in the physical sciences. So I may be harboring unrealistic expectations about how much can be known at this stage about higher-level processes such as personality, and about how much information can be extracted from experiments like this one.

Why should I let the toad work squat on my life? --Larkin
[ Parent ]
Expectations of Psychology (none / 1) (#142)
by Thyrsus on Sat Apr 09, 2005 at 06:48:40 AM EST

Chemistry is to physics as psychology is to neurophysiology. Chemistry characterizes the emergent properties to which protons, neutrons, electrons and photons give rise, but chemical characterizations are less complete in their descriptions that physics are, sometimes because the systems they want to describe don't have emergent properties, or the systems are too complex, or they ignore fundamentals that normally don't apply to the subject matter (how often do you see relativistic effects when studying DNA?).

Our knowledge of elementary particles is far more complete than our knowledge of neurons, and the chemical systems commonly modeled are far less complex (going from elementary particles to molecules) than the central nervous system (going from neurons to brains). Nonetheless, there are undeniable emergent features of combinations of neurons; e.g., language. The point to the studies cited is to try to determine what aspects of the "folk science" of personality can be measured and provide better than chance predictions. For instance, one study shows that an individual's self-evaluation as extroverted, conscientious, open, emotionally stable and competitive predicts that others will also perceive the individual as a good leader. There is no certainty. The best correlation they cite is 0.8, and only when attempting to predict aggregate results as opposed to a particular event. This is an emergent property of an otherwise unmanageably complex system - indeed any attempt to perform the orders-of-magnitude more exhaustive measurements that might take prediction up to an 0.85 correlation would likely constitute serious violation of human dignity. I don't, however, see reason to dispute the validity of that 0.8 correlation.

Graphology, by contrast, is an unmitigated waste.

[ Parent ]
What does it prove/predict? (none / 1) (#125)
by danharan on Thu Mar 31, 2005 at 10:52:57 PM EST

Polyxena links to an interesting article in the skeptic's dictionary.

Ah, yes, the Forer effect. Can you reply to some of these criticisms? I'm now confused as to what personality tests actually reveal: what others in their social group will say (perhaps privately) about an other person, or what a person admits or likes to think about themselves?

I remember taking the MBTI a few times, once by a consultant certified by CPP. Testing as INTP at the time, both my roommate and myself agreed that indeed I did prefer, e.g., chatting with a few people in a party than be the center of attention But has this actually been measured? Testing for each of these propensities they mention must have taken a lot of time and money... or did they take shortcuts? Pointers to further reading in the field that addresses these types of questions would be much appreciated!

What they measure (none / 1) (#134)
by badtux on Mon Apr 04, 2005 at 05:37:25 PM EST

Yes, the MBTI has been normed, and measures... well, measures what other people think about you, if you are a typical member of the normative population. They are normed by giving surveys to the family and friends of the normative population, asking a bunch of questions from a large pool of potential test questions, and then applying statistical methods to the question and answer pool to find which questions and answers most correspond with a particular set of survey answers.

As with all such normative processes, it is only valid for members of the normative population -- in the case of the MBTI, thousands of Pennsylvania schoolchildren and hundreds of primarily white female Pennsylvania nursing students -- and even within that normative population is not 100% accurate. Any normative test of this sort applied to someone from a different normative population (such as, say, inner city black youth, middle-aged business executives, etc.) is about as accurate as throwing darts blindfolded.

Furthermore, the question of how meaningful the results are remains unanswered. For example, the MBTI has been given to people suffering from a variety of mental illnesses. But if you know that the majority of people with, say, depression, have INTJ personality types, what does that mean? Does it mean you should not hire people with INTJ personality types because they're all depressed? Well, NO! That is completely misreading what the survey is measuring, which is prdominance of a particular personality type within a particular normative population. This says NOTHING about any other normative population, such as, say, the population of 20-something programmers looking for jobs.

In short, it appears that the MBTI is a nice party game and conversation starter, but hardly anything that is going to be useful in real life, despite all the millions of dollars that the authors of the test have put into advertising that it'll do everything except butter your toast for you. At best it appears useful as something to give a patient and his therapist something to talk about.

- Badtux the Psychologist Penguin
In a time of chimpanzees, I was a penguin
[ Parent ]

Bad norming (none / 0) (#138)
by danharan on Thu Apr 07, 2005 at 01:15:37 AM EST

So basically this test is calibrated to reveal what the people that know you believe you're like and what you'll do rather than what you are actually like, and how you really behave.

Seems to me like it's a larger scale case of fundamental attribution error. It ignores that people may intentionally deceive, or have different personnas in various contexts. Hmm...

[ Parent ]

Internal vs. external behavior (none / 1) (#144)
by badtux on Sun Apr 10, 2005 at 01:18:31 PM EST

In the end, nobody cares about your internal behavior. You can think whatever you want to think, feel whatever you want to feel, and it affects nobody at all. All that anybody really cares about is what you do or say, i.e., external behavior that affects other people. And there's nobody better qualified to comment on that than the people who know you best and have been observing your behavior for decades. By interviewing and surveying these people, your pattern of preferred behaviors in various situations (i.e., your "personality") can be mapped with quite good accuracy.

The problem with that approach, however, is that it is very time consuming and expensive. Thus tests such as the MBTI. As long as it is normed for your particular socio-economic group, it is fairly predictive of your preferred behavior in a number of settings -- i.e. whether you'll be the life of the party or the sullen guy standing against the wall talking about growing tomatos with an old lady, whether you'll act impulsively or spend much time analyzing situations before acting, etc.

The problem with the MBTI is not that it is inaccurate. It's relatively accurate (though not 100% so!) for the population group that it was normed against, though what it measures is not exactly what most people think it measures. The problem is that the normative group (school children, nursing students, and psychology students) is not representative of populations that it is being put to use measuring. It may be fairly accurate at predicting the behavior (based on "personality type", which is just a shortcut way of describing a particular subgroup of preferred behaviors) of a 20 year old white female psychology student, but that does not mean that it will similarly be predictive of the preferred behaviors of a 40 year old black male business executive or an 18 year old Hispanic male ghetto kid. The test has not been normed against either of those socio-economic groups, thus cannot be used to make reliable predictions about their "personality" (pattern of preferred behavioral responses to situations).

- Badtux the Behavioral Penguin
In a time of chimpanzees, I was a penguin
[ Parent ]

Comment about normative groups... (none / 0) (#135)
by badtux on Mon Apr 04, 2005 at 05:42:08 PM EST

The Canadian government looked at using tests like the MBTI to assist with career planning for "at risk" students ("diversity group" students in Canuck jargon). Their conclusion:

Overall, the report concludes that there is a lack of validity research specific to the use of these tests with diversity groups in a career assessment context.

I.e., the tests are only valid for populations they've been normed against, and there has been no such norming done for minority groups and other groups who are not predominately white college students.

- Badtux the Statistical Penguin
In a time of chimpanzees, I was a penguin
[ Parent ]

Describing and predicting behavior: The Big Five | 143 comments (101 topical, 42 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!