Software Security Gurus Episode #16: Leif Dreizler

Matias Madou:
Welcome to today's Software Security Gurus Webcasts with Leif Dreizler. Welcome, Leif.

Leif Dreizler:
Thanks for having me.

Matias Madou:
Absolutely.

Matias Madou:
Hey Leif, do you mind sharing a few words about yourself?

Leif Dreizler:
Sure.

Leif Dreizler:
I'm currently a security engineer at Segment, but I got my start in the security industry while I was still in university as a pentester. I was working as a pentester, mostly doing web apps for the first few years of my career. Then I actually joined Bugcrowd as their first sales engineer, and was on the go-to-market side of things for Bugcrowd for a couple of years. Then just over three years ago, I joined Segment as an early member of their security engineering team. Then in addition to that, I also helped run the AppSec California Conference, the LocoMocoSec Conference in the Bay Area, the Los Chapter.

Matias Madou:
Nice. Lots of involvement into the community. Fantastic.

Matias Madou:
Thanks for accepting to be the 16th guru on the webcast. For today's chat, I actually have two topics in mind, and I hope they are near and dear to your heart. Let's start with team and structure.

Matias Madou:
One thing I've seen is that at Segment, you essentially have one security engineering team, which contains of three sub teams, the cloud security, the application security team, which contains tooling and training, and the product security group, which is essentially building features like [O-Waff 00:01:42] and servers like 2-FA. Developers can actually use that. Correct me if I'm wrong, but you moved from one group to another one within Segment. As far as I remember, it was from AppSec to product security where you are right now, but can you tell a little bit more around this particular structure?

Leif Dreizler:
Sure. Yeah, definitely.

Leif Dreizler:
Security engineering is just one of the pillars at Segment in our security org. As you mentioned, it has three different sub-teams: cloud security, which is what you would expect based on the name. They're mostly helping people secure our data plane. They do a lot of work in AWS and GCP. We have our application security team, which is the traditional set of application security roles. They're the ones developing training, building tooling to help engineers do things safely, evaluating tooling. Then they are also doing the internal consulting piece of meeting with developers to talk about design reviews, threat models, that kind of thing, as well as running external consulting. If we're contracting with a pen test vendor or something like that, they're typically the liaison between our engineering teams and the external vendors. Then product security, which I feel like a lot of companies have a slightly different definition of. At Segment, we specifically define product security as the security features of our product. We're a combination of a security architecture and software engineering team. We actually partner with engineering teams to help them deliver product security features. We'll typically work in an embedded model where we'll actually go and work with another team for a quarter or two and actually help them design and deliver a feature. we're also a hands-on software engineering team.

Matias Madou:
Is there a specific group that you always work with? Is there a particular engineering team that you always pick, or how is that chosen?

Leif Dreizler:
Historically, we have worked with a couple of different teams that are in the same part of the engineering organization, and they're the teams that have been responsible for building features within our application. It makes sense that the product security team would partner more closely with these teams. But there's no reason why we couldn't partner with another team outside of the groups that we've partnered with in the past. It really just depends on what features are needing to get built and which engineering team is willing to help build those features.

Matias Madou:
Yeah. Qualifications, tech stack, previous experiences all come into play to choose that team.

Matias Madou:
In your application security team, can you tell a little bit more about that? What's possible? Maybe size, and what they're working on, and how many developers they're steering?

Leif Dreizler:
Yeah.

Leif Dreizler:
Originally the teams were more combined. When I joined the Segment, I was the fourth person hired to the security team. Because the team was a lot smaller, the boundaries of what people worked on were significantly more blurred. It was just whatever needed to get done, somebody did. It's still pretty similar today where if I wanted to go work on something outside of product security, it really would not be an issue. We want people to be able to get a breadth of experience. But the application security team, some of the things that I worked on when I was working more closely aligned to that team's charter was helping design custom developer tooling that was based off of findings from our bug bounty program, pen tests, things that we had found internally. It was customed to our tech stack. I would meet with a lot of teams to help them think through features that were under development. I did a lot of management of our bug bounty program. The number of developers I think, were about a hundred people, somewhere around that. Then the security engineering team as a whole is eight.

Matias Madou:
Okay. That's decent size.

Matias Madou:
I hope that gives, for the listeners of this podcast, a little bit of an idea on how an organization with a hundred developers, what you have to have in place to make it actually work, but because it does seem to work very well at Segment.

Leif Dreizler:
Yeah.

Leif Dreizler:
I think that part of the reason why our security engineering to develop a ratio is pretty favorable is because we have a lot of different things that developers are working on. We need people that have different areas of expertise. I wouldn't be that qualified to go work on something that's really heavy on the data plane side of Segment, because that's more on AWS and goaling side. I'm more on the data plane side using mostly TypeScript. Because we're such a large infrastructure company, I think somebody said last month, we processed a trillion events or something on behalf of our customers.

Matias Madou:
Oh, wow.

Leif Dreizler:
There's a huge number of infrastructure problems that a typical SAS company might not really have to think about. Having people that can tackle those challenges and really be experts in the cloud security domain is really important for us.

Matias Madou:
That's good. That's good that you're able to hire the diverse group which have expertise in all these areas, because it's indeed impossible to find one or two people with all the expertise that you need for a hundred people.

Matias Madou:
Maybe let's talk a little bit more about the second topic, which is technology and tech choices that are relevant. In the previous chat that I've seen online, you said something along the lines of, "Hey, at Segment we have no sequel injection, neither do we have cross-site scripting." That I raised my eyebrows, of course, and I said, "Huh, how is that possible?" Can you tell a little bit more about the approach that you're taking at Segment and why you are so certain about that statement?

Leif Dreizler:
Well, just to clarify, the statement was made looking back, we haven't really ... Today I could make the same statement. I think it's a combination of tooling and also the amount of rigor that we have had in assessment. We've not only looked at our tech stack using things like React, which makes cross-site scripting a lot more difficult, using parameterized queries for accessing things within our control plane. But it's also trying to build some additional tooling that helps prevent these types of common issues, because there's always a way to introduce these kinds of things. An example of where we've done this with cross-site scripting is one of the common ways that you can find cross-site scripting in a React application is with JavaScript hrefs. I'm not totally sure why you can still make JavaScript hrefs in 2020. I think they're probably almost only used for cross-site scripting attacks in this day and age, but they're still there for better or worse.

Leif Dreizler:
We have a React primitive called the UI Box. It's open source on GitHub, if you want to check that out or use it. That's a widely-used component in our design system called Evergreen, which is also open source. What we did in UI Box is there's a link feature, which is the common way that somebody would create a link. There's no reason to make a link yourself, not using the link feature. What we do here is we make sure that the protocol is one of the handful of different protocols. You can use HTTP, HTTPS, mail to, Tell, stuff like that, but you can't use JavaScript.

Leif Dreizler:
Then we also set the Rail values for external links. The no-refer and things like that. By just building this into the default for our link component, somebody would have to specifically go in and override this functionality, which there isn't any reason to do. I think that's a really good example of an instance where we didn't even build this. We just went to some of our friends in the development side and said, "Hey, we've noticed this problem a few times from submissions for our bug bounty program." They came up with a really great solution that's very workable, hasn't caused any problems. Then it was released to the open source community.

Matias Madou:
Nice.

Matias Madou:
These are great for startups, scale-ups, companies of your size, a hundred developers, are there any other basic security features that they need to bake in? Because I would assume if you're that size of a company, you still have a lot of flexibility to build these things in, because you're not at 10,000 people that are using it. You still have the opportunity to do that. Are there any other recommendations of basic security features that you would build in if you start from scratch or almost from scratch?

Leif Dreizler:
Yeah.

Leif Dreizler:
When you talk about security features, are you talking about, "Hey, we're a new startup, what are some security features we should be thinking about for our customers?" Or what are some other things, tooling-wise, that we could build to make ourselves safer?

Matias Madou:
I would say tooling-wise. Bake it in yourself.

Leif Dreizler:
Gotcha. Yeah.

Leif Dreizler:
Another thing that was really cool, and again, this was something that the Developers came up with, was something for preventing missing authorization checks. That's something that has been on the OWASP Top 10 as long as I can remember, at least since the 2010 revision, which was when I got into application security. It's something that's pretty hard, because every application is different. It's not something that's nearly as much of a solved problem as something like cross-site request forgery or cross-site scripting or something like that.

Leif Dreizler:
But a really interesting approach that our engineering organization came up with was actually inspecting the database queries that we were making, and then making sure that there was an appropriate authorization check happening in that same function. They used a spy, which is actually a common thing used in testing, to just check, hey, which database queries were called? They look at the name of the query, and then they also look and see what was called in our authorization library. If you're looking at the table workspaces, you need to see a corresponding call to something that relates to workspace permissions.

Matias Madou:
Okay.

Matias Madou:
I remembered now in one of the previous webcasts, you talked about security basics and security fundamentals. Do you mind explaining that for a second? How you see that and how that relates back to the example that you just gave?

Leif Dreizler:
Sure, yeah.

Leif Dreizler:
I wouldn't say that I hate the term security basics, but I just feel like it's a disingenuous way to portray the difficulty associated with a lot of these things, you'll see the armchair commentators on Twitter saying like, "Oh, this company got hacked. They couldn't even do the basics." It's like, well actually, the basics are really hard and they get harder the larger your company is. "How did they not patch this thing?" It's like, do you understand how big a large enterprise is and how many things there are to patch, and how many things there are to test? Yes, that thing is something that a lot of people consider a very basic activity, but doing it at even a medium enterprise scale gets pretty hard.

Leif Dreizler:
I really think that those types of activities, having SSO enabled everywhere, having a way for people to report phishing, a lot of those things that people think is the basics, I really think that it should be called the fundamentals. Because the fundamentals has a different implication. When you talk to somebody mastering the fundamentals of basketball, for instance, it's going to take somebody years and years and years to master their basketball shot. Yes, that is a very basic part of the game in the sense that if you can't do it, you're probably not going to be very good at basketball. But I do think that it is a more accurate way to describe these activities by saying that these are the fundamentals.

Matias Madou:
Couldn't agree more. Couldn't agree more.

Leif Dreizler:
I think that the way that this ties back to a couple of these examples is people would probably consider the fundamentals of application security, these are things that are part of the OWASP Top 10. These are things that everybody should know about, but how many years has a broken access control or some version of that been in the OWASP Top 10? It's not something like cross-site request forgery that has dropped out as technology and frameworks have risen to the challenge. I really do think that the way that the security industry is moving forward is largely they're the ones that are actually building, in many cases, these fundamental solutions to help solve things, either at the company level, or going back to the cross-site scripting example, at the framework or ecosystem level.

Matias Madou:
Yes. It's about mastering the craft, essentially. Basics are not simple.

Leif Dreizler:
Yep.

Leif Dreizler:
I think that engineers, they solve tough problems all day. Security problems are really not that different than any other. Needs a security partner that help educate them about the issue or point them to some resources, but engineers solve problems related to things they've never heard about all the time. I don't think that coming up with these really elegant security solutions is that different than coming up with an elegant solution for something else in software engineering. That's why I think that making friends with your developers is something that I think people have been talking about for probably the last decade. I think most people agree that it's a good idea at this point, but stuff like this really showcases the power of going to an engineer, saying, "Hey, we have this problem. What are some ways that you think we can solve it that are going to be really workable for your team?" Because the last thing you want to do is come to them, they're not going to want to work with you.

Matias Madou:
Yeah.

Matias Madou:
Last question. I saw on Twitter that he gave some free advice on where to buy the best pizza in town, you even gave advice on where to buy the best pizza in Austin. What's the best pizza place? Are you a big fan of pizza or was that a random tweet?

Leif Dreizler:
I would definitely consider myself a big fan of pizza. My favorite style of pizza is Detroit style, which a lot of people haven't heard of, but Little Caesar's is actually Detroit style. Obviously that's one version of it, but it's that thick, pan-fried style pizza where there isn't really the traditional crust that you think of. The pizza served in a rectangular slice.

Leif Dreizler:
If you're in San Francisco, my favorite Detroit style pizza place is called Pizza Squared. It's actually very close to Austin, Via 313 is probably my favorite Detroit style pizza that I've ever had. If you're in the Austin area, I would definitely recommend checking that out.

Matias Madou:
Sounds good.

Matias Madou:
Leif, thank you very much. I hope that this Corona thing is over very soon that I can come to San Francisco and let's go to that place and let's get some pizza.

Matias Madou:
Thank you very much, Lief.

Leif Dreizler:
Yeah, definitely.

Leif Dreizler:
Thanks for having me.

Matias Madou:
Thanks for being the 16th guru on the Software Security Gurus Webcast.

Matias Madou:
Thank you very much.