I wanted to point over to an article on compliance checklists and security by Bill Sieglein.

Over that 15 year period my attitude about using checklists to ensure the existence of security controls has shifted as well. Early on we were begging for some standards and checklists to compare against. Later on we realized that using checklists can lead to a sort of ‘tunnel vision’. Now that the list of regulatory requirements that most organizations have to comply with is growing unmanageable, I am seeing folks lean back on checklists again just to ensure completeness.

To me, checklists occur for several reasons (Sieglein actually mentions #1 and #3 in the article).

• We don’t yet know enough to make our own decisions (or our own checklists!).
• Stakeholders often tend to live in ignorance of insecurity and feel good that the front door is locked, even though a window in back behind a bush can be easily jimmied (home security is a great analogy to how many business stakeholders treat security). I’ll be curious how this works out with the other companies hacked by the TJX hackers that didn’t know it until the feds informed them…
• There is too much information to digest, so we try to condense it to a checklist.

And there is one other issue with checklists that Sieglein somewhat touched on but I wanted to flesh out.

Checklists are basically a binary measure, check or unchecked. Unfortunately, security is not always a binary practice. Ask any security dood “Is it done?/Is it secure?” and the answer will always be either “No,” or heavily qualified. Overcoming this by sticking to checklists means making huge ones, or filling them out extremely regularly.

I’m not against checklists. They have a necessary place in our security environments. We use them to ensure consistency in our work and our expectations/requriements. We use them to give someone else a quick glance into our world. We use them as easier-to-measure data points over time. We use them to organize our own sea of duties and information… But like good beer, they can be enjoyed but they also need to be tempered a bit to avoid falling down into a bad hole (hole, tunnel vision, same lack of vision).