Compliance-in-a-Box Tools Leave the Hardest Work with the Customer

Watch  ·  Listen  ·  Read

The pitch behind the checkbox

A customer, a prospect or a partner asks for proof of compliance, and the deal depends on it. If you are working toward that checkbox, you have probably been pitched ready-made products that promise a faster route. I put it this way in the conversation: some of them earn their place, and others leave more work than the pitch suggests.

That gap between promise and workload is what we explored with Doug Stonier, Director of Compliance at Steel Patriot Partners, in this episode of the Find Your Path series. I told him that most things in security seem to be driven by compliance: "It's probably a compliance" is my nine-times-out-of-ten answer, because the deal requires a certain level of it.

A low-poly night landscape of dark blue hills with a winding path of teal and blue stepping stones crossing a glowing teal stream over a small bridge, a wooden crate in the foreground, and a glowing golden open cube frame on a plinth at the far end, under a dark cloudy sky.
A checkbox waiting at the end of a long road

Does the effort depend on the framework?

Yes. Stonier says the level of effort changes significantly between, for example, a SOC 2 report and a FedRAMP High authorization, because their requirements differ widely.

He describes SOC 2 as built on open-ended controls, so a client can adjust the controls to what it currently does and report on how it does things. A FedRAMP assessment or certification, he says, requires the cloud service provider to meet goals established by the federal government. He adds that FedRAMP 20x has eased some of those controls, but "there are still high bars to meet."

Why the highest framework is the wrong starting question

Why does Stonier advise against over-compliance?

Stonier advocates against over-compliance, and he changes the starting question from which framework is highest to what the industry is driving you toward. If the industry only asks for a SOC 2, he says, get through the door with a SOC 2 and then mature your processes to meet higher demands.

He says the higher-demand frameworks bring more automation controls, which means more integrations with tooling and with processes a company already has. That is the point where he says you start looking at a GRC (governance, risk and compliance) tool.

What sits behind the dashboard?

A GRC tool gives you a dashboard of controls and the failing ones to dig into. Stonier's example: "Hey, these are the three hundred and twenty-three controls you have. Uh, you're failing ten percent of your controls." From there, he says, you have actionable items to work on.

The issue, in his account, is the work behind the scenes. A customer buys a tool because its marketing said it would help meet FedRAMP, then finds the knowledge base thin and is told an additional service is needed. Others tried it themselves and hit a roadblock: they lack API (application programming interface) scripting experience, or the know-how to tie evidence to controls and set parameters so the system does not false-positive a vulnerability. Stonier says that is where Steel Patriot Partners helps "bridge that gap of tool knowledge versus compliance knowledge."

The work the dashboard does not do for you

What happens after you click the framework button?

Stonier says clicking a framework button, such as the FedRAMP Moderate framework, imports template text describing how to meet the controls. It takes time and effort to curate that text to the business, he says, or real data produces many failures because the controls are dummy text. His answer is a human element to bridge what the information system says and what has been put into the GRC tool.

The second gap is getting data into the system. Stonier describes a false expectation that you click a button and the system knows where to pull data from. With only an API document and a "go forth" from the vendor, he says, the data often has to be curated and massaged before it reaches the GRC tool. His team identifies which tools a client uses, how to interact with them and how to make the data make sense.

What the button promises and what the customer then faces

What happens when an assessor cannot validate the data?

I raised the point that if the auditor cannot make sense of the data and give the green light, all of the effort is for naught. Stonier agrees on the principle: you can pull data in all day, but if an independent assessor cannot feel confident that what is pulled matches what the system security plan says, you run into issues during independent verification.

He adds two cautions on the auditor side. Many tools offer an auditor view that scopes down what the assessor sees, so work in draft, such as a privacy section not yet finished, stays out of view. And boilerplate template language can be too broad and open to interpretation, or irrelevant, so a company ends up promising something that does not apply to its system. He recommends curating controls to the environment and describing what an auditor is looking at and why it matters to each control.

Fixing the leaks, not just finding them

Stonier points to clients who thought they had a mature compliance process. Scans were running, he says, but the results never reached the people who needed to see them, and some hosts were not being updated. People were checking the box, in his words: "I just run the scan and I put this report over in this box."

He frames the firm's approach as improvement rather than fault-finding: "We're not here just to point out where all the, the leaks are. We wanna figure out how to fix those leaks."

I closed the conversation on the detail, because it is in the details that things get missed: one checkbox, one part of the network, one configuration left behind. For anyone buying a compliance tool, the questions to ask first are who curates the control text, who connects the data and who confirms an assessor can validate it.

Watch and listen

Watch and listen to the full conversation with Doug Stonier, and connect with him on LinkedIn. To find your own starting point, answer a few questions at steelpatriotpartners.com/findyourpath.

This is a Brand Spotlight. A Brand Spotlight is a ~15 minute conversation designed to explore the guest, their company, and what makes their approach unique. Learn more: https://www.studioc60.com/creation#spotlight

Questions answered in this article

How does compliance effort differ between SOC 2 and FedRAMP?

Doug Stonier of Steel Patriot Partners says effort changes significantly. SOC 2 has open-ended controls a client can adjust to what it does, while FedRAMP has goals set by the federal government. Read more: Does the effort depend on the framework?

Should my company pursue the highest compliance framework available?

Stonier advocates against over-compliance. He says to look at what the industry is driving you toward, and if it only asks for a SOC 2, get through the door with that and then mature. Read more: Why does Stonier advise against over-compliance?

How do I decide which compliance framework to start with for a customer contract?

Stonier says to look at what your industry is driving you toward. If the industry only looks for a SOC 2, he suggests getting through the door with it and then maturing your processes to meet higher demands. Read more: Why does Stonier advise against over-compliance?

What does a GRC tool give you?

Stonier says a GRC (governance, risk and compliance) tool gives a dashboard of controls and shows the failing ones, which become actionable items to dig into. Read more: What sits behind the dashboard?

I bought a compliance tool and the knowledge base is thin. What now?

Stonier says customers who buy on a marketing promise often find a thin knowledge base and are told they need an additional service. He says Steel Patriot Partners helps bridge the gap between tool knowledge and compliance knowledge. Read more: What sits behind the dashboard?

What happens after clicking a framework button in a compliance tool?

Stonier says it imports template text that must be curated to the business, or real data produces many failures because the controls are dummy text. Read more: What happens after you click the framework button?

Does an API document mean compliance data flows into my GRC tool automatically?

Stonier describes a false expectation that data arrives at the click of a button. He says it often has to be curated and massaged before it reaches the GRC tool. Read more: What happens after you click the framework button?

Why does it matter whether an independent assessor can validate my compliance data?

Stonier says you can pull data in all day, but if an independent assessor cannot feel confident in it, you run into issues during independent verification. Read more: What happens when an assessor cannot validate the data?

Why can boilerplate control language cause problems with an auditor?

Stonier says boilerplate can be too broad and open to interpretation, or irrelevant to your space, so you end up promising something that does not apply to your system. Read more: What happens when an assessor cannot validate the data?

Read the full transcript · 3,443 words

SPP - Doug Stonier - FYP edited

Sean: Hello, everybody. Thanks for joining us for a new brand story. organization starts somewhere different and is headed somewhere different, and the journey between A and Z is rarely a straight line. But thankfully, uh, the folks at Steel Patriot Partners has been on nearly every version of the path that, uh, you can imagine. And on their website, they actually help you find your own path in the location called Find Your Path. So I encourage everybody to go take a look at that. one of the paths we're gonna cover today is kinda this idea of compliance in a box. I've been asked to do something, by a customer or a prospect or a partner, and I need to get, uh, the checkbox of compliance. uh, if you're working toward that compliance goal, you've probably been pitched, uh, yeah, one of several, I'm sure, ready-made products that promise to get you there faster and on, on target. And some of them earn their place, but others leave more work than, uh, team might suggest. And, uh, and there, [00:01:00] and there is the reality. And we're gonna, we're gonna have a conversation today to kinda get to the bottom of what the truth is there. And I'm thrilled to welcome Doug Stonier on, Director of Compliance from Steel Patriot Partners. Welcome, Doug.

Doug Stonier: All right. Thanks for having me.

Sean: So, so good to have you on. I, I love these conversations and, uh, I think this one is an important one. Most, most things in security, it seems, are driven by compliance, right? You

Doug Stonier: Correct

Sean: a possible risk in there. You might have an, an attack. You might have some threats that, that get the board's attention. But nine times out of ten, I'll just go out there and say, "It's probably a compliance. We need to do this deal. In order to do the deal, we have to have this com- level of compliance." So to kick things off, maybe a few words about your role at Steel Patriot Partners and, uh, you see Steel Patriot Partners kinda tackling this problem from the, from the big picture.

Doug Stonier: Yeah. Uh, again, I'm Doug Stonier. [00:02:00] I'm the director of compliance here at Steel Patriot Partners. Uh, my role is I help, uh, drive our compliance framework reviews, and I help, uh, navigate the ever-changing industry. Uh, I am trying to keep up to date on whether it's in the federal space or commercial space, uh, updates to frameworks and also the tools that, uh, promise the ability to meet those frameworks in a short-shorter period than your traditional path of do your own research or, uh, your manual build-outs. Um, so currently I am assisting our clients and future clients on when they are met with a, "Hey, we have a contract, um, and we need to meet this compliance framework by the end of the year." Uh, how do we get you there from whether you already have an existing, uh, knowledge base and you're just trying to meet new frameworks or you're starting from scratch. Um, so we come in to help with those scenarios.

Sean: And so let's, let's start with the starting from scratch. Um, there are no, no-- there's no lack of do it yourself, uh, frameworks and guides and, and perhaps even, uh, uh, yeah, publicly available tools to kinda help you maybe get an idea of what, what to do. Um, what's your view on organizations that, that start from scratch and want to tackle a particular regulation? I don't know, does it change by the type of regulation, the, the depth and breadth of things? Maybe kinda run through that first.

Doug Stonier: Yeah, absolutely. Um, it does, it does significantly change your level of effort if you're going from, say, a SOC 2, uh, to a fully FedRAMP high authorization, right? Those are gonna have very [00:04:00] wi-widely different, uh, regulation requirements or prescriptive controls versus, uh, open-ended controls. Um, so in that case, uh, let's say if we just go the SOC 2 route, uh, that way a, a client is more able to adjust the controls to meet what they currently do and try to move to industry best practices and, uh, the report reports on how they're doing things, right? Um, whereas a FedRAMP, uh, assessment or certification, uh, is going to require the, uh, the cloud service provider to meet certain goals that are established by, uh, the federal government, what they expect you to do. Um, and while they've definitely alleviated some of those controls with the latest FedRAMP 20x, uh, migration, uh, there are still high bars to meet, uh, in that space

Sean: So what, what's the [00:05:00] range of starting from scratch, um, I'll say self-serve and self-attestation all the way to large programs that maybe tie multiple things together, FedRAMP and CMMC together, for example?

Doug Stonier: Yeah absolutely. Uh, I, I always advocate that you should never do over-compliance, right? You shouldn't just seek a FedRAMP assessment or, or certification just so you have it on your, your resume. Um, definitely look at what your industry is driving you towards. So, um, if, if the industry's only looking for a SOC 2, by all means, let's, let's get you through the door with a SOC 2, and then you can look to mature your processes to, to meet the higher demands. Um, and, uh, th- with those higher demand, demanding frameworks are coming a lot more, uh, automation controls, right? So it's gonna require a lot more of integrations with the tooling, um, and the, uh, the already existing processes that you may already have. Uh, and with that, you're going to run into the scenarios of, okay, I have all this data, but how do I get this into a compliant manner? How do I, how do I know I'm still meeting compliance? Uh, what, what data is actually, uh, needed, uh, versus a, a just a pull from your, your cloud service, right? Um, so when you start getting into those more complicated scenarios, that's where you're gonna look to a GRC tool, right? A GRC tool is going to be something that can take all that information in and put it into a, a nice fancy dashboard, and it gives you, uh, all of, "Hey, these are the three hundred and twenty-three controls you have. Uh, you're failing ten percent of your controls." Okay, can we dig into those? And now you have actionable items that you can, to move along with. But the issue with those accelerators or, uh, o-off-the-shelf, uh, compliance in a box, as we said at the b- beginning, the issue with those is there is a, there's a lot more behind the scenes that you have to do to get those to get you that information, right? And that thinks-- I think that's what a lot of customers run into, is a case of, I-- "Hey, I bought this tool, um, because it, their marketing team said this will help me meet FedRAMP, um, but I can't find the, uh, the, the knowledge board or the, the FAQ section doesn't really provide me information on how to do this. Um, they're saying I need to have an additional service to have someone come in and fix all these things for me." Um, unfortunately, that is the scenario that we've seen a lot is, is companies either relying on a tool, but they don't have it set up correctly, or they, uh, are-- they tried it themselves, and then they're, they ran into a roadblock where they don't have that knowledge or experience with API scripting or, uh, control, uh, tick and tying the evidence correctly, making sure the parameters are set right so that it's not gonna false, uh- false positive, a, a vulnerability or something like that on the system. Um, so that's, that's where we can come in and we, we help bridge that gap of tool knowledge versus compliance knowledge

Sean: And, and how much, uh, or how involved do you get in, and hopefully they come to you early and you

Doug Stonier: Mm-hmm.

Sean: and, and have these conversations, but much of the we understand our own environment and how it operates and where our data is and how our teams function and, and our tools work before you even start to build out a GRC program, let's say, which then leads to the

Doug Stonier: Yeah, absolutely. Uh, I mean, if you have [00:09:00] a mature program, uh, like you've already been through, you're authorized on the systems, you, you've been on the marketplace for a while, you, you, you understand the iteration of common activities and stuff like that, um, your team probably has a good idea of like what they need. It's just can they get it to, uh, show that visually in the GRC tool? And that's where we could come in and, uh, typically in those kind of scenarios, we'll review the documentation that already exists. We'll also interview with the, the teams, provide, uh, feedback on, "Hey, you responded to these controls this way. Is this still even accurate? It seems like you guys are-- you've moved on to these new tools." 'Cause sometimes, uh, those-- the old school SSP was two, three hundred pages of just straight Word document. And sometimes, you know, a Control + F doesn't always fix everything. So, um, there may be outdated statements saying that you're still using an old tool, or you might be, had been doing interviews or, or reviews of system on a certain cadence, but now it's like we do it on a different one. All those things come out through our process of, of doing a full documentation review, doing those, uh, interviews with SMEs, providing essentially a gap assessment on where you are and where you want to be. And then with that gap assessment, we also provide how is a GRC tool going to integrate into that scenario, um, right? Uh, we can tell you where your problem is, but that's not gonna help you, right? We wanna help elevate you from your current status to that, that compliance of the future.

Sean: And where are some of the gaps that you typically see companies, uh, sitting at, sitting with when they

Doug Stonier: Yeah. Yeah. Um, I mean, honestly, it can, it can range. Uh, it can even-- if, if they're in a GRC tool or a, a compliance-in-a-box [00:11:00] space, and they've gotten as far as we know we want the moderate FedRAMP framework. They click that button, and they get a bunch of, uh, of template text that says, "This is how you meet these controls." Unfortunately, sometimes it, it's a lot of time-- it takes a lot of time and effort to go back in and curate those to meet your business, right? Or otherwise, you're gonna get a lot of failures when, uh, you import your, your real data and the controls, the controls are just dummy text, right? They don't know what's going on. So you need to have that human element to, uh, to, to bridge that gap between what our information system is telling us versus what we've put into our GRC tool. Um, and that's the other major gap is, uh, people get these compliance tools, they put them in, and they're able to get-- say they do get through, and they've curated and made the, the text and controls match their current environment. The next part is, great, how do we get that data into our system? Because you only get an API document, and a lot of these companies are like, feel-- go forth. Um, you, you have an engineer on your team, right, who knows how to do API scripting. And, uh, we've run into a lot of cases where that is not what, uh, is happening. They-- There are some false expectations that it's just gonna be you click a button, and the system knows where to pull data from. Where it does take a lot of time to either curate and, and, and massage the data before it gets even into your GRC tool, um, because there's only so much you can do on one side of the, uh, the house. So, um, we've also been doing a lot of that, is, is going in, identifying what tools someone is using, how does-- how can we interact with that tool, and what's the most efficient way, and then how, uh, how can we make that data make sense and be, uh, able to be validated by a independent assessor? Um, that's the other part of this is, right? Is like you can pull data in all day, but if there's no way for an independent assessor to feel confident that what you're pulling is what you're saying is in your SSP, um, you're gonna run into issues during your independent verification.

Sean: So I'm glad you went there. And maybe we'll, we'll wrap with that and, and then we'll close with a, maybe a use case or something or a customer story. But so ultimately, the goal is to get authorized, certified, get the, get the checkbox

Doug Stonier: Mm-hmm.

Sean: of whatever compliance you're trying to achieve. self-attestation doesn't always work. You have to have an independent assessor and, and perhaps even somebody else that, that, uh, does the certification knowing what that looks like. Um, like you said, you can pull all the data,

Doug Stonier: Mm-hmm.

Sean: but if the audit assessor and auditor [00:14:00] is not going to be able to use it or make sense of it and give you the, give you the green light, then all that's for naught. The other thing is scoping, So understanding not just what you're pulling together, but the scope of it. So maybe, maybe touch on those two things and what you see there

Doug Stonier: Yeah, absolutely. Uh, let's just-- we'll touch base on the first one. Uh, auditor comes into your system, and they're gonna start knocking on doors, and they're gonna start trying to figure out what to make sense in this, uh, in the, the accelerator tool, right? Um, thankfully, a lot of these accelerator tools have a, uh, auditor view, so you can kind of pare down what you need to do for the assessment. Um, so if you're doing a continuous monitoring assessment and you only have one-third of the controls in scope or you, um... Or maybe you're, you're, you, you're building your SOC 2 and you've gotten security and availability down, but you're looking at, um, the privacy section, but that's not-- that's maybe it's in draft, right? So you don't want the auditor to look, start digging into that because you haven't fully completed those controls. So thankfully, these tools have the ability to scope down controls so that their view only sees what you want them to see. But you also have to keep that in mind, so if, if you haven't had, had experience with these tools, and then you start exposing information that you don't want exposed to the auditor yet, um, they can start digging, uh, in s-in spaces that you just don't, that are either irrelevant or they're a test or whatever, whatever it is, right? But that's gonna start causing problems, uh, from an auditor to client perspective. Um, also in that case, the boilerplate template language, uh, sometimes is either gonna be too broad and open to interpretation by the auditor or, uh, it's irrelevant to your space, and you've now promised that you're doing something that does not actually impact or affect your system. So, um, it's very important to make sure you curate your controls to your environment, one, and two, uh, when you are bringing in data and validators, it's, it's good to have the descriptors of what an auditor is looking at and why it's important to this specific control, um, and doesn't impact, uh, anything else within the system, right? So, uh, that's the auditor side.

Sean: Yeah, I think everybody's journey is different and, uh, you have a ton of conversations and see a bunch of different scenarios. Is there one you, you can pull out that's, that kind of demonstrates the, uh, the, the value of Steel Patriot Partners working with an organization?

Doug Stonier: Yeah. Um, absolutely. We've had, uh, various, uh, clients come through at different, uh, parts of their life cycle, right? Um, but I think ones that stand out to me the most are, uh, ones that are in continuous monitoring or, um, have what they thought is a [00:17:00] mature process around their, their compliance. And what happens is we come in, um, we've, we've done our interviews, uh, we've analyzed what we currently see in the, their policy procedures and then also see what's in the environment. And it-it-- sometimes it's just people, you know, they're doing their nine to five and the, it's set to how they've always seen it, right? When, when we come in and start answer-asking questions, we've, we've uncovered like, there's a gap here. Like you guys, you've, you've been scanning your system, but the scans never fully made it over to the people that really needed to see this. So, um, that-- we've had that in the case, uh, where, uh, they were, they were hitting, they were checking the boxes, scans were being done, and then, um, thankfully there was automation for updates and patching, so sometimes those numbers just went away on their own. But we started catching trends where some hosts were just not being updated because, um, they were checking the box. They were like, "I, I just run the scan and I put this report over in this box." No one's checking that box. So, um, I think those are the things that I enjoy is it, it-- One, it helps me feel like I'm, I'm doing a, I'm doing something good for the company, right? That, that's what I want to... I want to have a value add. I'm not detracting by saying like, "Yeah, you're not doing your job." I'm like, "Hey, here's, here's a somewhere we can make a, a difference and, and help your systems improve." Um, and that's what I, I find great about Steel Patriot Partners is that, that's what we're trying to do, is we just want-- We're not here just to point out where all the, the leaks are. We wanna figure out how to fix those leaks and also, you know, improve what you already are using to, to accelerate and, and make your compliance as mature as possible.

Sean: Well, there's no, no question the journey to compliance can be, uh, complex, uh, to say the least. [00:19:00] And I, I love how you described it because it's, it's in the details, right? It's that one checkbox. It's that one part of the network. It's that one configuration that gets left behind, and, and you don't know what you don't know. If you don't see it, you think... And you, what you do see looks good, you think things are good. And so having, having set eyes, knowing what the scope in, scope out, knowing what the end objective is and what the auditors will be looking for, and having the experience in connecting things and bringing the data together and presenting it in a way that, that achieves the objective for the business is, uh, what really matters. And, and it's a different path for everybody. And thankfully, uh, as people hopefully hear in, in chatting with you today, you have the, you have the experience. You've seen things and can help them get from A to Z. And, uh, that's exactly what steelpatriotpartners.com/findyourpath is all about. Answer a few questions on that page, uh, the team will be able to kinda help guide you on where you are, what you're gonna probably go through, on your way to achieving what you, uh, want to, want to achieve. Doug, thanks so much for this chat, and, uh, thanks everybody for listening. Stay tuned. There's many more of these, uh, conversations about finding your path with Steel Patriot Partners team. Thanks, Doug.

Doug Stonier: Awesome. Thank you, Sean.