Richard White is a recognized industry expert in the fields of cybersecurity infrastructure, cybersecurity remediation, and cybersecurity program development. With over 25 years of experience in systems design, security technology implementation, and security policy development and enforcement, He has developed innovative and affordable approaches for the rapid deployment of cyberthreat detection and remediation technologies. Prior to his tenure at Flushing Bank as the Chief Information Security Officer, he was the Managing Director for Oxford Solutions. Additionally, he served as the Chief Information Security Officer for the United States Capitol Police where he managed all aspects of advanced malware defense, advanced threat detection, incident response, and cyber remediation.View the profile
About the talk
Richard White, CISO, Flushing Bank This session will cover how to create and report customized cybersecurity metrics as related to risk. The goal is to cover the multitude of sources available and then condense these sources into a codified set of reporting metrics. While the examples will be focused on the financial sector, the literature review, data collection, and customization process will relate to most industry verticals.
My name is Richard white and thank you for attending this presentation on Cypress Security metrics development for board and risk committee reporting. Before I begin. I would like to thank our essay for providing us the opportunity to me and I would also like to thank flushingbank for supporting me throughout this entire effort. Thank you means. We're going to talk about today with regard to cyber security metrics how they're developed why they're important and how they need to evolve over time. It's no misunderstanding that
board. Members are holding the ciso concero. More accountable for not having the right information in front of them, to make a decision. And in that context. Lots of Articles and I both academic and and Commercial are available. And I will have two of them that I'd like to provide you. If you would like to see them one is psychotic, and the other one is forbes.com., Both focus on how boards need to have the right information in front of them to reduce risk and influence risk. Of time.
Cyber security metrics before we get into the development of the movie. Want to learn a little bit about the cyber security metrics, have to have a couple of properties. The first one is utility know, all of these bullets on urine stains that the board wants to accomplish with cyber security metrics. And more importantly, it can help lead to your success. Your cyber program's success in your organization's success, but more importantly Cypress Security, metrics need to be actionable. And without the ability
of a board decision to affect cost maturity or risk your reporting, maybe maybe muted. It may have less of an impact if it's not, but he did write or message, right? With regard to what your expectations are. And the reason for reporting. That metric, when I went to see ISO at the Capitol Police, the prior to my arrival, the Capitol Police had very vague cybermetrics, but they didn't want to change them because they were used to them, the chief of police.
The deputy chief Senate sergeant-at-arms. They aren't they, they lightly not pretended, but the lightly understood. Why the metrics needed to evolve to be more informative and better decision-making? But it wasn't something that they had a high degree of intent to apply resources to do So eventually. And I'm going to use this term and feel free to take it away. I applied gentle pressure over a prolonged period of time. I never ceased. I continue to ask questions and continue to describe the benefit of
refined metrics. And eventually I was successful in going to tell you about the feedback later on in the presentation. So when I was the managing director for Oxford Solutions, one of the things that I did as as an executive in that organization is I went on ciso for higher cause I went on bank site. They went on Commercial sites and I was there to help them fix an audit problem. Fix an engineering problem or just generally improve the reporting and viability of their cyber program.
Now, I noticed that all of the customers that I I went to visit as a ciso For Hire with with few exceptions. I can't say 100%, but most of them. They had very similar goals. They want the board to be informed. They want additional funding. They want the controls to be understood. They want to reduce risk and they want to align themselves repeatedly with the organization. That was it. Everything else was? I would say, haphazard the metrics that I reviewed as the ciso on-site. Mostly were inappropriate for
decision-making. They were interesting and titillating but they didn't do much in the form of informing the board and how the board can make decisions with regard to cost risk and maturity to resolve the problems that were being experienced. So what you see here is a collection of things that I began back in 2013, documentation review, literature survey review, asking questions and determining the quality of what really determining the definition of what really determines equality metric.
Now, one of the properties that cyber security metrics have to have it is Clarity. We've already talked about that have to have the ability to be actioned and utilities but they can have all of the utility in the world and without Clarity in reporting. It's difficult to get a positive decision out of any board or committee bottom-line a board or committee is not going to fund or approve anything that they don't fully and clearly understand that they can discuss amongst themselves with other executives. and
in my, Research with regard to utility and Clarity. I really sounded the board wants to know three things. Now, this may be different for your organization. But for my working is Asian in the banking industry. As a whole. I'm very certain that these things apply and should apply to most organizations outside of this industry that I'm in. They want to know if your cyber program is working. How effective is it at controlling and mitigating threats today.
How is it fully staffed as a fully resource? And how effective will it be tomorrow? And Beyond at protecting from those same price, and more modern types of attacks. So they focus on maturity is this labor program? Adequately funded board wants to know that if they will a dollar figure increased fix this problem, will the application of skill sets or resources fix this problem. And is it outside of our risk appetite? Or, is it something we need to take on immediately Board? Needs to know that for breakfast. And lastly
is the Cyber program reducing risk for the organization for the key stakeholders in the customers today and proven over time. So what what we have there is maturity cost and risk. I wish we going to Clarity, I speak and percents of my board very often monthly as matter. Fact about that in here in just a second. But in the early days of my tenure here at flushing bank, I use the terms mean time to detection and mean time to repair a lower resolution and what that caused when I left the room
was a discussion about the meaning of those esoteric cyber. Metrics, so I've lost the benefit of having a board decision. I lost the time that it took to prepare the presentation at that point. And I caused confusion in the boardroom after right after I left, and we'll talk about what that meant, what that meant later, but in general, I am fortunate to be able to meet with my board on a monthly basis. So taking small course, Corrections in over 30 day cycle is beneficial. And we'll
talk about that here in just a second. Additionally, Clarity comes from not only your messaging, why it's important, what they mean what the benefit will be, but also how they are framed, you know, holiday, logically constructed to present a meaningful and Piper related dialogue. Are they easy to understand or do you use the esoteric data that hardly anybody would understand that kick you to Lily a senior executive responsible at a at 4. Now, Additionally, the clarity of the message should be
such that the board can understand what that psychrometric means with regard to organizational strategy and direction. So understanding utility understanding the need for Quality metrics understanding the need for clarity. Let's talk very briefly about a high-level development process. They showed a slight earlier that was had literature review decision-making board goals. That's was the beginning of kind of putting together. Some blocks for a process. And what the
high-level process is it. We're going to discuss in the next light and really assess where you're at. This is the beginning of evolving, your metrics into quality, cybermetrics for your boy. That's where you're at. Do the results of the board, meet your expectations. Meaning. Do you get the right types of results? Do you get the right results that are meaningful? Are they expected in predicted based on how you report it in, but you report it. A literature review. Look at existing literature. Look at
the governance documents that are charged with regulating your industry. I'm in a very regulated environment banking. So I need to look at ffiec. FDIC Misty is CSF and some others from from threats, from SSI, second in it. In a few more, you may be different. And what Mom, I'm suggesting here. Is that gather that literature have it ready? Have the latest version and, and be ready to to select and Define your effort, and why you're picking or selecting specific cybermetrics, and that's an effort that on the next line.
But also, ask questions throughout the development process. It's not just a research and it's not just a documentation and guess, ask questions. What are your peers doing? What are they doing? It works. Well, what are they doing? That works? Not so well, what can you borrow and how can you link it to what the messaging that you need to convey? And what do you want your board to understand? Ask leadership, ask individuals who are charged with or have been charged with in the past board effort board. Understanding board reporting, find out what
they report. See if there's a cyber security element to it, ask and provide what your examples are, and be prepared to discuss them. And lastly, let's not forget. You can ask your peers and your colleagues and additional leadership, but nothing is more valuable than asking a existing board member asked him. If your messaging is clear, ask them. If they are getting out of your report, what they think you should be providing them and if it's off that's a great opportunity to make a real solid and meeting for course correction.
The sign for security development process in more detail it really. Before I get into this and I will be briefed on this. It's for those of you they're going to regulated industry or not, but you but you are under audit and some occasion. In my opinion and it's been my experience that auditors. Enjoy seeing policy, board approved, governance and evidence that you are following a process and a policy, and supporting a standard. This process right here will go a long way
in satisfying, and Auditors, curiosity for the better. Any, for example, the FDIC asked me a few years ago. Why was I reporting these metrics? Well, I showed them my literature review. I showed them my survey. I showed them all the metrics were, derived from the Publications. And I showed them how they were vetted against our executive management and our board members. And whether they had a question or a true amount of trouble with anything, it went right away because
nothing is more substantial than doing what you say. You're going to do and having evidence to prove it. This process is a way to define and prove to Auditors and others that you have a quality process in place that's approved to Define your quality metrics. So we've already talked about these areas at a high-level assess where you're at. You can't begin at the end. You have to start at the beginning. Where are you at? Look for the types of decisions, the types of discussions, the types of questions that you're getting from the board.
Is it what you expect? If it's not assess, look at white with why? It's not. And and can move that Ford make the course directions. The second stage is discussed with executive or senior, leadership, what you're doing and why you're doing it. Why are you developing new cybermetrics? Why are you putting resources and allocating resources into something that we don't need? It's your job to describe the effort, describe what you're doing, why you're doing it and what the benefit will be. If you can
generate a consensus among your initial stakeholders, then you have something to move forward with if you can look at with their feedback is and understand why there's no movement. See if there are better metrics that are more meaningful or add additional metrics that will provide the depth and understanding that those individuals are looking for Once you have a consensus on your early State folders continue to do your research, apply their feedback C and determine if any additional cybersecurity metrics are required. If not move
forward to find them and move forward to a broader discussion with additional executive leadership. Ask the individuals that you've already spoke with. If they could recommend additional folks for you to to fit this against. And once you have your fully vetted and approved set of metrics begin to report on them. Remember that you the board meeting is not the time to make or to present new cyber security metrics. If you're presenting new metrics at a board meeting.
Your chances of being successful, totally are much less than what they would have been if you would have had a consensus against the fully set of bed. It's like an securitymetrics new. Additionally. Once you get to this fully vetted fiber securitymetrics, you can take a little break, but you just really started the process metrics stagnate metrics. Need to the industry changes organizational Direction and strategic Visions change. The Cyber program needs to be able to flex with those requirements or regulated body has
changed, and you need to report those metrics in a way that shows alignment continues alignment and give support the opportunity to prove and see evidence that there is alignment through cybermetrics. It was a great segue because now I want to talk about the actual rotation or evaluation of existing metrics. It is I'm not claiming that developing cyber security metrics is easy. In fact, it's it's difficult, but it takes only takes is commitment and and pressure applied over a. Of time. Right? And correctness will always float to the top
with the right, meaning and conversation behind it. So I'm going to skip over quickly, the frequency of reporting cuz I want to get back to that. I want it in the slide with that but to evaluate your metric again, you're just looking at the same questions. You already, have to develop them. Are they generating discussion, does the board, use them in a meaningful way to make decisions against maturity, accost maturity and risk? Do the follow-up questions from the board? Are they meaningful? And are they? What you expected? Are they discussing the fundamental
elements of the metric or are they discussing the benefit of understanding that metric and applying a action to it light cost maturity or risk? Additionally. Are the metrics that you're presenting, or do they continue to be actionable metrics today or today? They're fresh. They're new to providing the right type of output and it's a, it's an oiled machine. However, overtime, as I stated organizations change leadership changes, regulating bodies and requirements, change direction
changes in What was once a very actionable metrics? Early down, becomes difficult to make a decision on because it's no longer applicable or applicable as it once was in the past. So keep that in mind as you as you have the score set of metrics and your documenting them in your frame, eat them for for continued discussion. Now, I report to my board monthly. I find that very vintageous. Why? Because of 30 day cycle, if something doesn't get input. I have only 30 days to wait till you have another decision or
understanding its principal. And it's, and it's winds itself. Well, to my success of my cyber program's success. Some of you watching, this may also report monthly. However, I bet it's a bigger and a better bet that most of you report. Semiannual or annual to your bored. And that was determined by your executive leadership of the board and the guidance principles that the board operates under but few of you have the opportunity to report monthly. And I think that when you have a short
turnaround between bored, I eat a 30 day cycle, you can make smaller more meaningful course, corrections as well. As you have. I have access to the management team, the senior management team and the board members much more than somebody who reports annual to their board with. And I don't mean that as as that one is better than the other. But for me at this Bank in this organization reporting monthly to the board and hitting that output, on a monthly basis and understanding what their understanding is key. Do you want to continue
to limit with the organizational vision? So, What did the why I had a a survey? And I did some analysis, I get some coding and I put out a lot of surveys and I asked a lot of questions and I had some help from friends and we put together a fairly comprehensive set of cyber metrics. The group of people that I focused on really that naturally evolved into three categories colleagues Who provided operational experience tactical experience, for example, the type of vulnerability that was exploited. How long was it exploited? Was there any
lateral movement? What did the logs indicate that those types of fairy in the weeds technically? Esoteric type of metric. In contrast, those an academic's really focused on providing feedback and Metric suggestion that was in the areas of Interest or areas that currently taught or half-past one or are currently EG. And they tended to focus on administrative like policy and procedure risk analysis, auditing, and the elements that go with the administrative and administrative safeguards for a healthy fiber program, cable feedback, but big difference between my peers and then
Academia and contrast the board when I reached out to board members in past organizations that I have served on served at and encouragement, as well as the board members be board members to reach out to you and get their feedback. It was nearly not 100% but nearly all is on risk, cacost, maturity and risk. And I thought that was very telling. So if you and if you case, you haven't picked it up yet. The very core theme of other than quality metrics for this presentation is
board members. Want to hear the metrics reported in the categories in context of how does it affect cost? How does it affect elf maturity and how does it impact risk? So what makes a good metric? A good metric? First of all, let me kind of describe them to you. What, my core cyber security metrics are. I will see if I can do it from memory. Some of them were meantime to detection containment resolution as well as the results. Not the number of pain test or vulnerability scans, but the results and the
criticality of the findings, for the vulnerability scans, and the penetration test. In addition to that employee training on how susceptible we were to phishing attacks. They how fish prone was. This organization also risk assessments and overall Cypress health and audit findings were very key to board members overall, but they really focus on those are the quality metrics that I have. But really all of them can be described and categorized within the big buckets of cost maternity and
risk, so, I thought that was very interesting and I saw it. It, it was leading me down a path that I thought, from early on orbit way through that was was correct and meaningful. So what makes a good metric metrics that can be used to understand, organizational, need and wrist understanding if the cybersecurity program aligns with and informs organizational strategy, and it speaks directly through any vehicle to cost maturity and risk. So the key, the key here, is that the
board. It's the general consensus of my effort that board wants to hear certain things. They want to hear it in certain categories and they want to hear how areas caused maturity risk, apply to the organization through the metrics that are that are reported. Again, the considerations for the board or a key, but in order for the board to make any consideration of any cybermetrics, it has to be clear and it has to have a high degree of you till they need to review, company, controls, and Pure Performance. That's through cybermetrics. How
well, and how effective does a cyber program protect against the identify and respond to cyber incidents? How well are they predicting? How well is the program? Protect us today? And how would protect us tomorrow? What is the state of compliance? Do we have any material weaknesses or weaknesses that have been identified by an audit internal or external state or federal? And how do we ensure the Board needs to know that the projects within the Cyber program or information security? The roadmap to the future is the line with what the
organization vision is and a strategic vision of the organization. We want to be able to protect against the threats that we will encounter both today and tomorrow and the flavor program should be able to provide evidence of that. And the Board needs to see that evidence and be able to understand it in a in a alignment, very meaningful decision-making, Manor. This is some of the high-level results. Like I said earlier, I Wrote down. I don't know. I probably 300 or 350 cybermetrics.
This is a result of what the high-level categories were. Most of my metrics fell into one of these buckets. And this was before I'm showing you Ross examples of raw things right now, but if you look closely, you can see, but cost, you can see the maturity and you can see the risks throughout that's not accidental. That is what do definding is reporting, and that's what the metrics that I was gathering as well, and metrics that I had discussions about
with other board members other, she is owes and other Executives. So this demonstrates that that the process is is valid and a collection of These metrics at the time. I think it was 300 or 350 can be boiled down into very meaningful discussions and and be used for Effective decision-making. If they're presented, logically meaningfully and framed correctly with each other prior to and during the board meeting. So what do what, what if we heard and what do I want you to take away? There are a lot of things we talked about
Clarity and utility and development processes and times, and difficulties, and things that may take more commitment than other administrative efforts that have been undertaken, but to get started. It's it's simple. But just takes the staying power in the commitment. So immediate, so after you leave after you conclude this presentation and you think about metrics the first thing and you want to move on a path of quality and improve metrics. Take 2 weeks to understand the effectiveness of the decision-making output. Are they, is the Ford telling you
what you expected or are they way off if they're way off, don't that may be a problem with the metric or the message or the logic in the frame that which with which they're presented. So you need to develop a framework to present metrics. Are you using something that's predefined. Are you using them? Something that a colleague, use, whatever that is that is completely up to you? And there's no one framework. I would recommend over the other but to present your metrics in a meaningful and logical manner framework is required.
I would consider And cost maturity and risk as the biggest mop bucket and pull everything out from there that you feel you need to report on as long as there's a map backup, the one of those categories. No, I also am going to ask that. You look at the existing documentation for your industry for me. It's at the v i c e. F d, i c a s s. I c c. I s. C o f f. A in a few others, it may be different, or maybe the same for you. But while you're going through that Gathering the documentation looking at what's available in the
literature, I would like you to consider in. This is not written down here. The miter attack framework. It is apt at kns attack tools and techniques and knowledge base. And it's a collection is very well organized collection of all of the attacks that have been observed observed out from a cybersecurity perspective for a very long time. So the the chain and the organizational components to what bad guys do. And what tools they use, And what their in their impact could be and what type of
results they're looking for? Clearly, you can see the application to defining cybermetrics. Now, when you first look at it, it's going to appear complex. But if you spend a little bit of time, you'll see that it's a very valuable. It's free their own version 8.2. It is very modern and very updated and I highly recommend that you look at it for your first phase. Now, if you'll look at it and you don't find anything applicable to your board report. You should contact me. So we can discuss it because I, I can't
100% guarantee you, but I would be interested in finding a Ford report as applied to the miter attack surface in and not being able to find a quality metric, that should be derived that is for you and happy to chat about it. Anytime. 4 weeks out. You should continue to review the literature source. Define what metrics you've already selected, why did you select them? What are the benefit of this metric? And what do you think the intended understanding will be off the board? Once you have that written in some
form at reach out to key stakeholders that are considered colleagues or friends, but have the same insight and understand the key elements of what you're trying to accomplish. Tell them. What you doing. Tell them what you think, the benefits are going to be in, get their feedback, listen to it and incorporate it because what they're providing you is their insight into Ford mechanics and what's a for discussions are and how you can get in front of those types of discussions. I recommend reaching out to the CIO and the cro. You may
have other titles in your organization, but select an executive that can provide you the feedback that you need. So, two months and Beyond, two months to three months, you need to refine and validate the metrics that you gather. So, you had these discussions, you've did these categories. You've applied wall cybermetrics into these categories, Incorporated feedback. Now, provide that back to your key stakeholders that you initially spoke with. And then ask for their opinion on who you should reach out to
next. You want a broad, you want to end with a very broad audience that can help bet that can help generate consensus and and and generate dialogue so that you're not presenting new cybermetrics at the next board meeting. And Beyond three, three months and Beyond continue to push, continue to document and continue to go through those steps until you get to where you need to be. It's not, it won't happen overnight. In most cases if it does for you, fantastic and I congratulate you but in most cases, it's going to be difficult to get on
board members calendars. It's going to be difficult, to explain the, the premise, and the benefit of what you're doing in the time frame, maybe that's been allotted but that should not be a reason or rationale to in the continued to apply. Gentle pressure over a. Of time and you will truth and logic will prevail. You will be successful. So in summary, we've we've talked about quality metrics. We've talked about the importance of Netflix. Why does the board need metrics, won't? They have to
Quality metrics? They have to make effective decisions that impact the organization. Holy Clarity, and utility are key items and helping the board members understand your point of view why this area is important and more important than all, what they can do about providing a solution. Also, you want to ensure or at least make a solid attempt to aligning your cybermetrics with the big frame categories of costs, maturity and risk board people, or people board members, understand that and you want to speak their language and if you can Mathis Piper security, element
to cost maturity or risk directly and non esoteric terms, you're going to have a better results for the board. It's been proven to me and to my colleagues that have adopted similar styles. We talked about a process for developing quality cybermetrics assess where you're at. Look at it for what it is or you getting the right type of output from your board. Do your board members have questions for you that are focused on results and not about the metric itself. If so, the messaging or the clarity may be in question. You may need to
revisit that if it happens frequently in the same questions continue to come up. We also talked about the addition of of other elements metrics from outside of your organization that may be applied to from other Industries, into your into not an issue. Be flexible to look at other industry. Documents in and look at what cybermetrics, they are suggesting and see if there's a linkage. Understand if there's something that can be beneficial in your report to your board members. And the last thing is once you
go through the understanding and the development of all of this, the process, the writing the discussions and the evaluations, you're not done. You really just begun because remember, your organization matures overtime, directions change standards bodies and requirements. Change the threats. Make pivots and change your cyber program. And what you report should also change. And if you have an issue with the frequency of change, or the frequency of refinement or or if I uation ask get feedback from the board asked, if you were bored, your new rotated
and cybermetrics are meeting to Mark and they're getting out of it. What you expect them to get out of it. That's all I have. It's been a pleasure to present the cybermetrics presentation to you. And if you have any questions, please feel free to reach out to me to or white at flushing bank.com. I'm happy to provide articles. I'm happy to provide the raw cybermetrics. I'm happy to provide examples of refined metrics in and board. Reports course, they would be redacted, but I would just show the the frame be
Buy this talk
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.