Security Program

Pragmatic CSO review on Slashdot

Submitted by Mike Rothman on Mon, 2008-07-28 13:35.

Nothing like getting a little present on a summer Monday. I wanted to point out that a review of the Pragmatic CSO was  posted today on Slashdot. You can check it out:

http://slashdot.org/article.pl?sid=08/07/28/1330215

Overall, Ben Rothke provided a balanced and positive review of the book, which really hits on the key points I try to highlight not only in the process, but also in my weekly newsletters and podcasts.

 

Incite Redux: Day 1 - Express Your Inner Bean Counter

Submitted by Mike Rothman on Mon, 2008-07-07 10:17.

Good Morning:

Just to give you a general overview of the Incites Redux process, I revisit my 2008 Incites (or projections for those of you not familiar with my lingo). I do this provide some level of accountability, which still seems to be unique in the technology research business. Folks make ridiculous projections, both on market sizing and industry dynamics with impunity. If they are wrong, so what? They still collect their checks and no one is worse for it. Except those poor saps that actually follow their advice.

So hopefully by now you've realized I'm a different kind of analyst and a different type of guy. I not only welcome the scrutiny of my positions, I search it out. So over the next two weeks, I'm going to revisit each of my 2008 Incites and give myself a "grade." Of course, this is self-analysis - but I'm confident that if you strongly disagree with something, you'll let me know. Bashful folks you are not.

Have a great day.

Incite #1: Express Your Inner Bean Counter

Substantiating the value of security continues to plague practitioners, who still can’t specifically answer the question: “Are we secure?” Structured security programs (ISO 27001/2, COBIT, Pragmatic CSO) help align programmatic activities, and look for significant advances in the area of security metrics – where the industry begins to gain consensus about what can and should be tracked.

Read the original Days of Incite post on this topic.

6-month grade: D

OK, this is not an auspicious beginning to my 2008 Incites. Some of my buddies told me I was being a bit optimistic to think that we'd see "significant advances" in the area of security metrics. And they were right. But let's not get the cart ahead of the horse. The first part of the Incite deals with security programs, and if anything the desire for the industry to get a "cookbook" of sorts that provides a set of playbooks to do security remains very high.

Count von CountI've fielded a lot of inquiries and questions regarding IS 27001/2 and also COBIT. Throw a little of NIST's 800-100 and 800-53 and it remains clear that most practitioners still have no idea where to start when embarking on a security program. I expect these frameworks will accelerate over the short term. I also think that a lot of fairly pragmatic IT professionals will default to following the 12 requirements of the PCI DSS. Notice that I said pragmatic IT professionals, not Pragmatic CSOs.

Most IT professionals facing down the spectre of PCI compliance don't have much of a choice, but to move towards the 12 requirements. Truth be told, that isn't a bad place to start - but it's not a comprehensive security program. For security professionals (yes, Pragmatic CSOs), the program needs to be more holistic and more structured. Remember, security is a journey not a destination and it's not just about passing the assessment and getting the rubber stamp. It's about actually protecting information.

Which brings us to the metrics discussion. Over the past 9 months, I've gotten fairly deep in the metrics community and lent a hand to start up a consensus group to define what can and should be counted. Basically, I'm still looking for my own answer to "Are we secure?" and at this point, the only answer still is a resounding no! One of the books I'm reading during my summer hibernation is called "The Black Swan" and it's really impacting my view on what security really is and how we should be measuring ourselves.

I've got a lot more thinking to do around the topic, but I think the question "Are we secure?" is the wrong question to ask. To me, "how quickly can we recover from a fixed number of attack scenarios?" seems to be a more appropriate question, especially given that we've never been able to predict where the next major attack will come - and I suspect we never will. But we may be able to model the type of damage an attack will cause, and figure out how long it would take to recover. 

I know that the risk counters out there want and like to build big models to assess the risk and quantify it (and hopefully over time reduce the applicable risk that is being measured), but I'm still not sold on that approach.

To their credit the Risk Management Insight guys have been very patient with my constant wingeing about their approach and Jack has put together a number of thoughtful pieces about why actually quantifying your risk is a good thing. Yet, I'm still questioning how that kind of analysis yields any kind of return relative to the time and resources required to build the model.

But even that's not the point. The point is that we need to bifurcate the metrics process into (for lack of a better term), an elementary school track and a PhD track. There are some very very smart people out there that are talking at a PhD level. Their work is impressive, but it's not accessible. The common men and women out there, just trying to get out through the day, do not track (nor could they collect) the metrics the PhD's are talking about.

Yet, the PhD's tend to be very critical of the "good enough" approach of the rest of the world. The folks that count patches and AV updates and spams blocked. And this is the problem with trying to produce a SINGLE set of metrics for the entire industry. Thus, I now realize the idea of gaining true consensus onsecurity metrics is probably a pipe dream.

We should be looking for AT LEAST two different sets of metrics and data sets. The first is really tracking activity and that is for the unsophisticated practitioner that is just trying to get a handle on what they are doing operationally. We have the data and although it's not perfect, at least it's there. 

Then there is the ongoing research that the PhD's are pushing to model and quantify risk and figure out what the knobs are that really impact security outcomes. Optimally, the PhD's find some stuff that everyone else can use over time. But to think we are going to get to a consensus anytime soon is, well, optimistic. And in this business, hope and optimism get a D.  

Photo credit: "censored" originally uploaded by tifotter

2008 DOI: Day 1 - Express Your Inner Bean Counter

Submitted by Mike Rothman on Tue, 2008-02-12 17:30.
2007 Incite: Get with the Program
As security professionals continue to struggle with the number of threats and contradictory goals (protect information, but assist business), they increasingly turn to structured security programs (ISO 27001, COBIT, Pragmatic CSO) to assist in getting things done and communicating progress. Security management tools (predominately SIEM) continue to leave customers wanting for value and assistance in automating programmatic operations.

2008 Incite: Express Your Inner Bean Counter
Substantiating the value of security continues to plague practitioners, who still can’t specifically answer the question: “Are we secure?” Structured security programs (ISO 27001/2, COBIT, Pragmatic CSO) help align programmatic activities, and look for significant advances in the area of security metrics – where the industry begins to gain consensus about what can and should be tracked.


Big AbacusAs you can see from above, the Incite on security programs is slightly evolved from 2007. So what’s changed? First of all, the state of security programs is still nascent. CSOs still have a problem substantiating value. They can’t control their to-do list. They can't keep their customers happy. The attackers never take days off, they don't sleep. If anything, life has gotten even harder for security professionals, though it’s certainly hard to envision that happening.

My continued focus on security programs continues to be self-serving. Although the feedback I've gotten from folks that are using the Pragmatic CSO in practice has been outstanding. But I'm not religious, use whatever you want. Just use something. Put some structure into your operations. Have a plan that is business relevant. Be able to substantiate what you do and why, for both internal (senior management) and external (auditors) parties.

Ultimately, you need to be able to crisply answer the question: “Are we secure?” Increasingly, you are seeing the bean counters asking tougher and tougher questions about why funding for security must be maintained and what return they’ve gotten from the years past, where the security team has spent money like drunken sailors. And the idea that you haven’t had a breach (if you are lucky) isn’t really good enough.

What's new this year is a specific focus on metrics because I think that’s really the sticking point. Andy Jaquith’s seminal work (Security Metrics: Replacing Fear, Uncertainty, and Doubt) hit in April of last year. It’s great stuff and really lays out the problem in more depth than anyone has thus far. You’ve got to walk before you run and Andy’s book has made us all toddlers. Now we need to take it to the next step, so to speak.

The industry is still bickering about what makes sense to track and what is going to resonate with the powers that be. Remember, the powers that be want to know how this is impacting the BUSINESS. Not how cool the technology is. They don’t care that 99% of the applicable servers are patched by 11 AM on Exploit Wednesday (the day after Patch Tuesday). They just don’t care.

So we’ve got to make more progress on coming to consensus relative to what is important to track. The good news is that I do think we’ll be making progress on these fronts in 2008. We need to start establishing some “benchmarks” of what good security performance looks like. You may suck or you may be great. How do you know? Until we can answer that question, we’ll be in our own little version of Groundhog Day.

Big abacus image uploaded by: cowsmanaut

Report Card: 2007 Incite #1 - Get with the Program

Submitted by Mike Rothman on Mon, 2007-12-24 07:32.

Yes, it's that time of year again. It's acountability time. Over the next 5 days (culminating in the New Year's Eve spectacular!), I'll be critically evaluating all of my 2007 Incites (that's my vernacular for predictions) and giving some perspective of what happened, what didn't, and why.

So without further ado, let's jump onto Incite #1.

Incite #1 - Get with the Program

As security professionals continue to struggle with the number of threats and contradictory goals (protect information, but assist business), they increasingly turn to structured security programs (ISO 27001, COBIT, Pragmatic CSO) to assist in getting things done and communicating progress. Security management tools (predominately SIEM) continue to leave customers wanting for value and assistance in automating programmatic operations.

Days of Incite Link:http://securityincite.com/blog/mike-rothman/2007-doi-day-1-get-with-the-program
Incite Redux Link:http://securityincite.com/blog/mike-rothman/incite-redux-july-9-2007

Final grade: B+

It’s tough to be a security professional nowadays. The attack surface continues to expand, the vectors are multiplying, the bad guys are getting more and more innovative, and it’s still not clear what our main objectives are. So is all the news bad?

Actually it isn’t. I’m not going to blow smoke in your backside relative to how much progress security folks made in 2007, but the reality is the folks that have adopted a programmatic approach are in much better shape today then they were 12 months ago. Nothing is going to be a panacea relative to getting more relevant with your senior team besides good, old-fashioned hard work and effective, outbound, proactive communication.

The Pragmatic CSO approach and philosophy works. I’ve gotten enough feedback from both early reviewers, as well as some folks that are using the process in practice to know that it works. But you have to do it. You have to get out from behind your desk and work the program, building relationships with the senior team, monitoring your environment, and taking care of all the steps in the program.

I’m very excited about what Pragmatic CSO – Year 2 will bring. There will be more ways to access the content, more assistance in implementing the program, and ultimately more success stories. But as with everything else, you have a choice. You can certainly continue doing what the vast majority of security folks out there continue to do - which is to continue to react to every situation, pray that your bosses understand what you do, and keep your resume fresh - so you can move onto the next job before the hazards of the present job catch up to you. Remember, you don’t have to do anything different - I hear the status quo is working out well.

Relative to security management tools, most end users remain disappointed at how much time and money it takes to make the existing generation of security tools add value to their environment. But that never stops the entrepreneurial bug. Now there are new “risk management” offerings hitting the market and others positioning into the GRC (Governance, Risk and Compliance) space - whatever that means.

GRC tools promise to “automate” the compliance reporting process and maybe even associate security controls with risk. I’ll remain skeptical until these tools become easier to use for companies below the Fortune 100. So at least some companies are trying to make some progress and help with the onerous reporting requirements of today’s regulations and audits, but 2008 will still be an early adopter year for GRC, as the market figures out what needs to happen and then how to solve the problem.

Check out the other posts in the Report Card series.

2007 DOI: Day 1 - Get with the Program

Submitted by Mike Rothman on Wed, 2007-02-14 14:50.
As security professionals continue to struggle with the number of threats and contradictory goals (protect information, but assist business), they increasingly turn to structured security programs (ISO 27001, COBIT, Pragmatic CSO) to assist in getting things done and communicating progress. Security management tools (predominately SIEM) continue to leave customers wanting for value and assistance in automating programmatic operations.

I'll be the first to admit that this first Incite is pretty self-serving. Obviously having just published a "poor man's security program," the Pragmatic CSO - I'd certainly like this to be a self-fulfilling prophesy. But let's examine why a security program is in great demand in the markets out there.

First, there is to much to do and CSOs and other security professionals are having a hard time figuring out what to do on any given day. Second, even if they know what to do - helping the rest of the organization (especially the business folks) understand the value that security brings has been problematic. Finally, the auditors show up every so often and it's usually a miserable experience for everyone.

Basically, many many CSOs are looking for a better way. I believe taking a programmatic approach to security can provide the structure and perspective needed to be successful in today's environment.

To be clear, I don't much care if it's 27001 or COBIT or any kind of program. But doing security in a hodge-podge way, basically playing whack-a-mole to eliminate the issue de jour just isn't working. So it's time to try something new.

What about that security management stuff? That's the 2nd part of the Incite and remains pretty controversial. Again, to be very clear, I don't have an issue with security management. It's necessary and critical to being a successful and Pragmatic CSO. BUT, security management has to add value. If it's so expensive and ponderous, as to actually detract value - then there is something wrong. That's where we are at today. The biggest enterprises see value, but that's about it.

I continue to be haunted by my past as a networking analyst in the early 90's. I had a front row seat as network management evolved and eventually disappeared. It's pretty operational now and dominated by the vendors that provide the networking equipment. The biggest networks in the world use stand-alone management offerings, but most folks use whatever their networking provider offers.

We've seen this movie, and security will be largely the same. First, there is the bundling thing. If you are doing a big endpoint renewal, you can bet you'll get that security management thing thrown in. Just ask. Same goes for UTM and every other major category. And reading Syslog and getting feeds from other devices just isn't that novel anymore.

That's why many Cisco customers default to MARS, even if it doesn't work as good as other offerings (just ask Bejtlich on that one). It's easy to buy and that overcomes a lot of technology and implementation issues. You know what they say, you don't get fired by buying [name your favorite big ass vendor here].

We will see more activity and more clarity about what log management does relative to SIEM this year. And we'll also see tighter partnerships between network behavior analysis (NBA) vendors and SIEM. Why? You get to look ahead of you (with NBA) and behind you (with SEM), which is actually pretty compelling.

But overall in 2007, expect security management to continue to disappoint. That's all the more reason for you to get with the PROGRAM.