This Saturday I assemble my raiments and get hooded as a Master of Science. In partial fulfillment of the requirements for the degree, I wrote, re-wrote, heavily edited, and finally collected signatures on my report on information security, compliance, and catalogues of control. I bounced back and forth on the utility of compliance, especially of compliance frameworks based on catalogues of controls. My current belief, based on research and an deep case study, is that frameworks that rely on catalogues of controls can’t get it done, when “it” is establishing trust between organizations.
A reason for my research was my own frustration in hearing infosec folks say “security is not compliance!” I heard the “security is not compliance” again and again as a mantra of frustration, collected with anecdotes about compliant yet breached organizations. Yet, security should be compliance, right? That’s what it says in the all the compliance guidance: the objective is to establish a level of security. So why the grousing?
I explored a couple of approaches to the question, but ended up addressing compliance as a means to establish trust to third parties. The certification, whether PCI-DSS, or a FISMA ATO, is a due diligence short-cut, establishing to outsiders that an organization has it together enough to handle your data. This short cut is useful, as anyone who survived the Great Paper Gramm-Leach Blizzard of the early 00’s.
Nonetheless, the problem is in the implementation. Getting PCI-DSS certified or a FISMA ATO is table stakes for operating in the payments or federal contracting games. Organizations are willing to spend money on getting the certifications to get the revenue opportunities. The certifications can even be used as signifiers of trust even for other entities that don’t require compliance.
So why the disconnect with actual security? Looking at other industries, particularly accounting, I saw some parallels. There is a built-in problem with third party assessments. It is a market for lemons, and some assessors stink at their jobs. Like used cars, it’s hard to tell a good assessment from a bad one, until the thing fails.
This feature of assessment is exacerbated by the structure of compliance frameworks with control catalogues. Control catalogues are seductive to both the assessor and implementor. The implementor gets a straight forward checklist that can mostly be managed within the IT department. The assessor gets a universal, reusable checklist, so an engagement can be sped through without pausing to actually examine an organization’s proprietary processes, assets, or culture. Control catalogues can reduce the cost of the lemons to the organization that ends up selling them.
More in my next installment.
Some bibliography:
Kaplan, S., Roush, P., Thorne, L. (2007) Andersen and the Market for Lemons in Audit Reports. Journal of Business Ethics, 70(4), 363-373
Akerlof, G. A. (1970). The Market for “Lemons”: Quality Uncertainty and the Market Mechanism. The Quarterly Journal of Economics, 84(3), 488–500.
And listen to the Blue Lantern.
.






