Monday, October 5, 2009

Certifications are Evil.....By John McCash

Note: The following does not represent the opinion of Mark McKinnon. He merely had the good grace to allow me a forum in which to post it after it was respectfully declined (for obvious reasons) by the SANS Institute's Forensic Blog. I wrote it chiefly because I hadn't seen anything recently, or as I recall, ever, that so much as acknowledged any downside to certification. I respect the pro-certification viewpoint, but I do disagree with it. And so, without further ado...

Certifications are Evil

by John McCash

Folks, this is an opinion piece, and it's going to be a controversial one. Some of you started composing a scathing rebuttal to it as soon as you read the title. Normally I restrict myself to what I hope are useful technical tidbits, but like most of you out there, I'm a forensic practitioner, and I have little patience for time sinks which provide no benefit (no I'm not including the training in that category, save your flames for the end). I've always begrudged the time commitment (over and above what's required to actually take the training and learn the included material) required to attain certifications, despite which I'm in possession of five, soon to be six, not counting my master's degree, so I like to think I speak from some degree of experience.

I do understand the arguments used by the proponents of certification. In essence, they allow people who have no understanding of a technical discipline to discriminate between other people who do and don't have that understanding. At least that's what they're supposed to do. Let me list two of the most egregious counterexamples that I have found in my own personal experience (with no disrespect intended to either Microsoft or the International Information Systems Security Certification Consortium). I have met, in my career, an extraordinarily large number of clueless CISSPs and MCSEs. These are people who were apparently able to pass the test, but who were unable to, respectively, secure or administer their way out of wet paper bags. To state it in more general/inflammatory terms, one problem with certifications is the number of idiots who are in possession of them. On the flip side of this, I personally oversaw the hiring of a system administrator back in 1996 who had nothing but a High School Diploma and a clue. I still work with him on occasion, and his hiring was one of the smartest decisions I ever made.

One logical response to this issue is simply to make certifications more difficult to get, but there we run into a second fundamental problem. When a certification raises its difficulty in order to exclude a certain percentage of unqualified people, they also exclude a certain percentage of qualified people. As the difficulty raises more and more, the incremental number of unqualified people being excluded gets smaller, and the incremental number of qualified people being excluded becomes larger. The amount of work required in order to to pass increases substantially as well. Qualified people get excluded for several reasons. For one, the more difficult a certification, the more training is typically required before attempting the exam. One forensic certification I heard about last week, the one which finally prompted me to write this posting, requires six months of training and six exams. That's a tremendous amount of time committed to obtaining a fancy certificate and some alphabet soup to put on your resume. Don't get me wrong, I'm not saying that training is useless. But what do you do if you're already in possession of 75% of the knowledge this training is intended to pass on? It's in the financial interest of the certification providers to make it more difficult to pass the certification if you haven't attended their custom-designed training program. Review guides may be available, but typically cover more material than the certification vendor's training, without the subtle emphasis often provided by that training. The practical upshot of this is that an individual who may know 75% of the material on the exam off the top of his head, substantially better than a graduate of the certification course will be (probably) after six months or so, may still have to complete a long and expensive training course just to get to a point where he can reliably pass the certification exam. For many of us, it's simply not worth it. We resign ourselves to being filtered out because we don't have the requisite alphabet soup, even though we're otherwise qualified.

You'd think that at some point, an exam would filter out all the idiots, but that's much harder than you'd think. That's why IQ tests have fallen out of vogue, and why an actual interview is still the best way to select a new employee. This brings me to the third reason certifications, or more specifically certification exams are bad. Many standardized tests consist of simple regurgitation of facts. They don't require that the subject really be able to think, just memorize. Personally, I believe that any idiot can pass such a test if they put sufficient time into preparation. It's possible to design questions to test problem solving ability, but it's difficult. One tactic that's often resorted to, and this is a personal hot button of mine, is to provide the subject limited information, allow him to assume the rest, and make him pick the 'most reasonable' or 'best' solution from the list. The problem with this occurs when the test subject is smarter or knows more than the individual who designed the question. I personally have run into this several times on various certification exams (I got a couple of the questions changed), and I find it intensely frustrating.

Finally, certifications are bad because they provide lazy people with a tool that can be easily misused. Rather than read 100 resumes to determine the 15 most qualified for a particular position (which he may lack the expertise to do anyway), an HR person can simply filter out all those lacking a specific certification. If this still results in a number of resumes that is too large, he can filter on another certification. This sort of data reduction can easily remove more qualified people than unqualified. In my opinion, it's better to pass all 100 resumes down to the hiring manager.

Certifications are bad for hiring managers, because they reduce their pool of qualified candidates, and they're bad for the candidates, because they enable those candidates' resumes to be filtered out before the manager sees them. In the end, they provide the most benefit to the vendors who provide them and their associated training, and to HR organizations, who are able to get by with fewer and less expert people.

Once a certification is accepted as required in a certain area, this fact can be used by people who lack training in that area to obtain it. The downside of this is that people who are already qualified sometimes must forgo more advanced training to take training just to get the certification. I'm not suggesting they don't learn anything in this training, but typically it will be much less than they could have learned had they been able to attend training of their choice.

So, you might ask, what's the alternative? Isn't there some other low-overhead way to reliably tell if a candidate knows anything about a given specialty without actually reading his resume or interviewing him? Well, I have a suggestion. Maybe somebody out there can make it work. It's based on word of mouth, and the PGP web of trust. Basically, there are a number of people who's word I trust if they say somebody has a clue. If everybody had one or more PGP keys with a comment that said "I am an expert in X", then people could sign that key, and the subject could publish the result. If Rob Lee, Ed Skoudis, & Josh Wright all say I'm an Uber Geek (and I'd like to think they might), I tend to think most people would buy into it. Maybe we could call this the web of cluefulness.

As always, please feel free to leave commentary if you liked this article or want to call me on the carpet for some inaccuracy.

Let the flames commence!

John

222 comments:

«Oldest   ‹Older   201 – 222 of 222
Chang bui thi said...

If you have followed some of my tweets you know this as I have had a few rants about the level of some individuals that hold certifications
---
Signature:
Friv 4 School Games – Friv4School – Games For Kids Free Online friv 4 school is a great store for childrens games. Where you can find the best friv games for school and free FRIV kids games. Play right now.

jogos friv said...

Great post,Thanks for providing us this great knowledge,Keep it up.
---
Signature:
juegosjuegos.com || juegos de matar zombies

Hà Ánh said...

I do understand the arguments used by the proponents of certification. To state it in more general one problem with certifications is the number who are in possession of them
----
I come from Vietnam. we offer camera all kinds. Invite a friend to visit my website:
camera giá tốt good price
camera trọn bộCheap, cheaper when purchase
lắp đặt camera vantech
phụ kiện camera
phụ kiện cameramore than a guarantee

friv2 said...

i think i need it. Thanks for update information. Love it :) Keep on posting!.
---
Signature:
i like play games friv 2 | friv2 | friv

baixar facebook said...

I like the idea of a web of cluefulness
---
apply baixar facebook movel online free | baixar facebook | baixar facebook gratis

Qadit Qadit said...

Thanks I like your blog very much , I come back most days to find new posts like this! Good effort. I learn it.

Data Security Information

WilliamHill said...

Don't really agree with you. Certifications are still relevant and in some domains they are fully needed, eg. in online poker regulations.

lionelmessi10 said...

gonorrhea have symptoms so knowing when to seek treatment can be tricky When symptoms do occur they are often within two to 10 days after exposure Many women with gonorrhea discharge think they have a yeast infection and self-treat with over-the-counter yeast infection drug Because vaginal discharge can be a sign of a number of different problems it is best to always seek the advice of a doctor to ensure proper diagnosis and treatment

wasirdenature said...

cara mengobati kemaluan keluar nanah cara mengobati kemaluan keluar nanah cara mengobati ujung kemaluan keluar nanah ujung kemaluan keluar nanah di sertai perih saat kencing ujung kemaluan keluar nanah di sertai perih saat kencing penyebab penis keluar nanah penyebab penis keluar nanah obat untuk kemaluan yang keluar nanah dan perih saat kencing penis keluar nanah pusat pengobatan herbal obat kemaluan keluar nanah mengapa kemaluan keluar nanah mengapa kemaluan keluar nanah obat penis keluar nanah obat penis keluar nanah cairan nanah keluar dari kemaluan pria cairan nanah keluar dari kemaluan pria obat cairan nanah keluar dari kemaluan obat cairan nanah keluar dari kemaluan obat cairan nanah keluar dari kemaluan cairan nanah keluar dari kemaluan cairan nanah keluar dari kemaluan kemaluan keluar nanah di sertai perih saat kencing kemaluan keluar nanah di sertai perih saat kencing

lionelmessi10 said...

cara mengobati kemaluan keluar nanah cara mengobati kemaluan keluar nanah cara mengobati ujung kemaluan keluar nanah ujung kemaluan keluar nanah di sertai perih saat kencing ujung kemaluan keluar nanah di sertai perih saat kencing penyebab penis keluar nanah penyebab penis keluar nanah obat untuk kemaluan yang keluar nanah dan perih saat kencing penis keluar nanah pusat pengobatan herbal obat kemaluan keluar nanah mengapa kemaluan keluar nanah mengapa kemaluan keluar nanah obat penis keluar nanah obat penis keluar nanah cairan nanah keluar dari kemaluan pria cairan nanah keluar dari kemaluan pria obat cairan nanah keluar dari kemaluan obat cairan nanah keluar dari kemaluan obat cairan nanah keluar dari kemaluan cairan nanah keluar dari kemaluan cairan nanah keluar dari kemaluan kemaluan keluar nanah di sertai perih saat kencing kemaluan keluar nanah di sertai perih saat kencing

on ap lioa said...

There are so many unqualified people out there charging crazy rates and getting them because the demand for "experts" is high. I would also argue that we need certification because people lie I am sory this is my work On ap Lioa | lioa

juegos kizi said...

You have a real ability for writing unique content.
----
play game juegoskizi online and play game jogos friv

Hà Ánh said...

Wonderful blog. Thank you for sharing valuable information
đồng phục hà trang
may đồng phục
đồng phục công sở
may đồng phục công sở
đồng phục khách sạn
đồng phục bệnh viện

Hà Ánh said...

I would like to thank you for the efforts you have made in writing this article. In fact your creative writing abilities attract us.
đồng phục||may đồng phục ||đồng phục công sở||may đồng phục công sở||đồng phục khách sạn||đồng phục bệnh viện ||đồng phục văn phòng|| sơ mi nữ

Hà Trang said...

I would like to thank you for the efforts you have made in writing this article. In fact your creative writing abilities attract us.
đồng phục||may đồng phục ||đồng phục công sở||may đồng phục công sở||đồng phục khách sạn||đồng phục bệnh viện ||đồng phục văn phòng|| sơ mi nữ

Aashi siva said...


Really, these quotes are the holistic approach towards mindfulness. In fact, all of your posts are. Proudly saying I’m getting fruitfulness out of it what you write and share. Thank you so much to both of you.

J2ee Training in Chennai Thiruvanmiyur

Samia Hussain said...

Its great, I mean i agree with you, No need to fear man we are with you, We see and record this all time. Bakersfield Tires & Auto Services

Allie Smith said...

Nice information. Feel free to contact HP helpline number UK and get instant solution of any HP computer problems.

Allie Smith said...

Acer customer care number UK provide the best solution of your Acer computer issues.

MCSE Server Infrastructure Certification said...

I agree with your points. There are lots of schools and institutes where a person can get certification on a certain field but it doesn't mean he becomes very knowledgeable and suitable for the job.

Quickbooks Data said...

Nice...
quickbooks intuit number
quickbooks intuit support number
intuit quickbooks number
quickbooks intuit software
quickbooks payroll service
quickbooks payroll service number
quickbooks payroll support number
quickbooks payroll support phone number
quickbooks bookkeeping services
virtual bookkeeping services
remote bookkeeping serivices
Professional bookkeeping serviecs

johanseena said...

Really great post, Thank you for sharing This knowledge.Excellently written article, if only all bloggers offered the same level of content as you, the internet would be a much better place. Please keep it up!

«Oldest ‹Older   201 – 222 of 222   Newer› Newest»