In our latest TechTalks episode, Zoe Cunningham is joined by Joe Edwards, Technical Principal, and Daniel Waters, delivery principal at Softwire, share how to use DevOps to make your developers more productive. They discuss what they mean by productivity and DevOps, and why DevOps is particularly relevant for financial services.
Techtalks brings you industry insights, opinions, features and interviews impacting the tech industry. Follow us to never miss an episode on SoundCloud now: Listen to Techtalks online for free on SoundCloud or on your podcast platform.
Transcript
Zoe:
Hello, and welcome to Softwire TechTalks. I’m Zoe Cunningham. Today, I’m delighted to welcome onto the show, Joe and Dan. Hi Joe and Dan. Can I ask you to introduce yourself, tell us what you do at Softwire and also an interesting fact about yourself?
Joe:
So, I’m Joe Edwards. I’m a technical principal at Softwire. I’m currently leading a large team working with a big insurance company, doing a digital evolution project, and there’s all sorts of DevOps involved in that. I guess maybe my interesting fact is that I’m a reasonably experienced recorder player and I was in the National Youth Recorder Orchestra once upon a time when I was a youth.
Zoe:
Wow. Do you play all the different-sized ones?
Joe:
Yes, the whole range at home. I don’t get to bust out, you know, the really big and the really small ones very often.
Zoe:
I remember this, that you’ve got recorders and they look like they just get bigger and bigger. But then the really big ones like really strange, isn’t it?
Joe:
Yeah, there’s sort of different types. You can get ones that look like that come from IKEA that are sort of square and plywood.
Zoe:
That’s the one I had, I think. Dan…
Dan:
Hello, my name is Daniel Waters. I’m a Delivery Principal at Softwire. I’ve also been involved in some of the projects that Joe has been delivering, but my time tends to be spread between multiple projects where I have more of a project governance role, I suppose. So my job tends to be to sort of make sure that projects are set up to run nicely, and then I disappear quietly and hope that they do.
My interesting fact about myself: I’m going to point to the barometer on the wall behind me. I like to predict my own weather.
Zoe:
Wow. How successful are you?
Dan:
Not very good Zoe, I’ll be honest. And it’s a much wider range of prediction than I could get from the BBC, but it’s quite nice to try every now and then.
Joe:
It’s your prediction.
Dan:
It’s my prediction and I make it. I didn’t really see today’s rain coming.
Zoe:
So, a barometer just tells you what the pressure is right now, does it, and then you deduce from that what that means for the weather?
Dan:
So, I haven’t ever read a guide about how to use it. I should qualify that. I’m not a trained barometer reader.
There’s a little like golden needle that you move over the black needle. So the black needle moves by itself, the golden needle you move, and then whether it’s risen or fall will help you make the prediction.
So, if it’s going in one direction at certain pressure, hat normally means might be raining. If it’s going up at that pressure, it might actually mean that it’s clearing. So it’s the direction that’s as important as the pressure itself.
I’m sure there are other more detailed ways to read it but I haven’t figured them out yet. To be honest, mine is very pessimistic and tells me it’s raining all the time. It’s not often wrong though.
Zoe:
That is absolutely fascinating. So today, we’re not going to be talking about how to predict the weather sadly.
We’re going to be talking about how to use DevOps to make your developers more productive. And I’d like to start by saying developer productivity is this real goal that everyone’s chasing right, but what do we mean when we say productivity, and can we mean different things by it?
Dan:
Yeah, I think we can mean different things by it. And I think today, I think we should maybe focus on how to make the time that your developers are spent developing, better, and how to make sure that the things that they are doing get out into the real world faster.
I don’t think DevOps is necessarily the tool to help like the performance management side of productivity or that kind of thing. I think it’s definitely more about making sure that your processes are streamlined, and that you are able to get the best use out of your time spent developing.
Zoe:
Okay, so isn’t one development team going to be roughly as productive as another development team?
Joe:
I think that’s sort of definitely not the case. I think there are obviously a few reasons that that could be the case. And the one that we’re talking about at the moment is really the time that’s spent not doing development. It doesn’t matter how clever your developers are, it doesn’t matter how experienced they are, if they have to spend most of their time messing around forming deployments, diagnosing issues with very little information, they’re not going to be able to get very much actual feature work done.
Zoe:
Okay. So that’s super helpful. With that as our framework, what is DevOps?
Dan:
Yeah, I think that’s a good question.
I think a lot of people mean slightly different things by it. To me, DevOps is a bit like Joe said: It’s a lot of the stuff that sort of sits around the edge of your development, but also sits around the edge of your operations where they sort of collide.
Typically, I suppose there are some standard things that you might count in the DevOps wheel-house: So, continuous integration and deployment pipelines, for example, making sure that you’re running automated tests as part of your build patching of your code in the same way every time, doing repeatable deployments.
These are all classic DevOps tools that you might use, and I think we’ll talk a bit more about that later. But there’s also some of the processes I think, almost fall into DevOps.
Joe:
Yeah, I think that’s right. You can bring in things like approvals processes, code review, testing reports, as well as actually being able to link up all the stages in your SDLC and make sure that everything’s running through smoothly.
Dan:
I think one of the really big things that DevOps can help you with is just provide confidence in your software, like Joe was saying. If you have developers messing around at the edges, then that introduces a lot of human error possibility.
Humans are really important part of software development. But there’s a lot of things that computers are better at than humans. And I think doing things in a repeatable way really helps you to build confidence in your application, and there are certain aspects of the development lifecycle that just really suit themselves to that automation.
Zoe:
So DevOps, really… it’s kind of all the parts of ‘being’ a software product, that aren’t the actual coding – would that be fair way to describe it?
Joe:
I think that’s probably not quite the full picture. There are lots of other things that are involved in a software project, like drawing up requirements, acceptance criteria, and user testing, and all that sort of thing.
I would say it’s really about automating as many of those processes. It’s a kind of software developer mindset. Really, that’s sort of where the DevOps comes from: It’s applying those kind of software development ideas to operations, and to everything else, you know. We’ve got all these powerful tools. Let’s use them to streamline everything that we’re doing.
Zoe:
Fantastic. So you mentioned, in your introduction, that you’re working in the financial services area. So, why is DevOps particularly relevant for financial services?
Joe:
So, I think there are probably a couple of key things about financial services.
So, first is there’s a lot of regulation, you need to be sure in what you’re sending out, you can’t just make changes and hope that that they’re kind of ‘fine’. You can get hit with a pretty hefty fine if you send out the wrong messaging on your website, for example. And that means having a process that you have faith in, having an audit trail, making sure it’s repeatable and safe, is really key.
And I think the other one is that most financial services are large and relied upon by people for fairly fundamental services. And so reliability and uptime, seamless deployment, that kind of thing, is really important. You can’t just have your website go down for a few days, because that has real impact on real people more than a lot of other industries, perhaps.
Dan:
Yeah, I think in my time here, I’ve worked with a couple of large insurers and large building societies and banks. And the thing that they all have in common is a real desire to have confidence in the things that they’re deploying.
I think, obviously, every industry cares about this, and every industry would love to be bug-free. But I think for for some of the big financial institutions, this stuff really matters.
Because your volumes are large, you’re dealing with a very important part of everyday people’s lives, the regulation is big, the potential for headlines is big. You really want as somebody who’s managing the deployment train or the release train of your company to know that you aren’t going to come into work tomorrow with a big fire. And then instead, everyone’s gonna be pleased for your release, because it does what you think it should do.
Zoe:
Yeah, so I can see the scope for mistakes, or size of mistakes, that you can make is much larger, which then makes it more important to get it right.
Dan:
Yeah, and the regulation is obviously almost a symptom of that. Or maybe it’s a bit bi -directional, but definitely the regulations there because as a society, as a government, we don’t want those kinds of mistakes to happen. And so we have to abide by those regulations.
But I think it works both ways. You want to abide by them, right. You also want your type to do what it is meant to do to comply with all this stuff, because it’s good for your business as well.
Zoe:
Okay, well, I’m definitely sold on DevOps.
So, assuming people are listening, and they don’t have a good DevOps set up, and now they’re like ‘Okay, I really need to start working on this’, how do you get started?
Dan:
I think it’s a really interesting question.
I think the way that I would potentially first do this is do an analysis of your processes. Try and tease out what you’re doing repeatedly and often and identify some candidates for automation.
I’d probably try and start small. I don’t think you’re going to suddenly go, from an engineering team that does a lot of this stuff manually (maybe does lots of manual testing, lots of manual deployments), to a really efficient development machine overnight. You have to start small, get some confidence in the tools that you’re using, give your team some exposure to this because part of DevOps is that you’re expecting your development team to do a lot of this work. So, you need to potentially do some upskilling there.
So, I would say identify a few processes that feel right for automation, give your team a little bit of a licence to go and find the right tools to actually implement it (you might have to pay for some of these licences, you might have to support that team in that upskilling) but I think it should pay for itself relatively quickly, because you will hopefully no longer have to do whatever the most painful processes in not that long time.
Joe:
Yeah, that was gonna be almost the first thing I said.
Be prepared to invest time and money into it, right. This is not going to be something that you just give someone a couple of hours in the afternoon to set up all your DevOps. It’s something that you need to care for and maintain over time as well.
It often goes hand in hand with other modernization activities that you might have in flight. So, if you have a large on premise system, setting up efficient DevOps pipelines for that is probably going to be quite a challenge. But perhaps as you migrate things into the cloud, piece by piece, that’s kind of the same process: you want to identify a small piece that you can do, and that might be a good target to automate it at the same time, because the tool is ready for it.
Dan:
Yeah, I think that’s a really good point.
And I think I slightly answered from a position of what if you’re coming from no DevOps. The landscape is changing all the time and I think what might have been considered good 10 years ago, is maybe not quite a fit for purpose now. We live in a slightly different world. In fact, there’s quite a lot difference between now 10 years ago in the tech world.
And I think that investment, it could pay off at any point. I don’t think you’ll ever find yourself with a totally unimprovable DevOps situation. There will always be times when you might want to move stuff or might want to automate the next thing as well.
Zoe:
So, it’s a bit like any kind of software improvement. I find ‘incrementally’ is just the word for software development, right? It’s all about doing little bits whenever you can, making it a little bit better all the time.
So let’s get stuck into that then. What are the kinds of tools you can use? What pieces of software would you be going out and looking at to use to help you with DevOps? And maybe what have you used yourselves on projects that you’ve worked on?
Dan:
Used quite a lot of things to be honest.
I think when I started out a few years ago, the sort of go-to tool at Softwire was always Jenkins. But I think we’ve branched out massively since then. My current project runs everything through Azure DevOps, which is unfortunately, a slightly confusingly named combination of Azure and DevOps.
It’s more than just the DevOps pile. It also does all the sort of project management side as well. But it does have some really quite nice integrated features for DevOps, and in particular, we’re using that for all of our pipeline builds, for all of our deployments and for our environment management.
And I think one of the things that I’ve really quite enjoyed about using that set up (obviously, we’re deploying to Azure so it’s all sort of integrated with it), but one of the things that’s quite nice is just the sort of level of transparency it gives you going from user story all the way through to a deployment and a release date. It’s quite easy to sort of navigate through that so I found that really nice.
And with the sort of infrastructure as code approach that we’ve taken on this project that I’m on at the moment, everything that you see in that repository is what that project is. There’s sort of no external dependencies of it. I think it works really nicely. So I’m quite happy with my current DevOps setup, which is nice, we get good test reports out of it, we get a huge amount of value out of that.
I’m not trying to advocate for Azure DevOps. I think that’s just one system amongst many, but it has worked really nicely.
Joe:
Like you, I’ve used Jenkins extensively in the past. We’re now using TFS, which is the predecessor to Azure DevOps. We’re in the midst of trying to upgrade to Azure DevOps. Although it sounds very different, it’s basically the same thing that got renamed.
And I think the core of it is a platform that allows you to script things essentially, and then obviously, there are a bunch of tools that you can use those scripts to run. So, infrastructure as code is the obvious one. I think we’ve used Cloudformation here. We’ve used Terraform in the past, which are very nice.
You know, there’s all sorts of automated testing frameworks, depending on what languages you’re using. I think we’re using Jest, XUnit and Cypress, which is a nice change from Selenium. I think it’s a little bit more developer friendly.
So yeah, I think the platform is the heart of it. And that’s what you use to run all your scripts and your automation and link everything together.
Zoe:
And how technology dependent are these different platforms? Because obviously, I notice in the name Azure DevOps, the name Azure. So presumably, that’s only going to be applicable for projects that you’re already running in Azure, or is that not right?
Joe:
So, we’re currently using TFS rather than Azure DevOps, but it will be momentarily. And we’re using it hosted on AWS and to deploy to AWS. So I think, fortunately, I would say the days of really hard vendor lock-ins like that are, for the most part much less bad than they used to be.
Dan:
One of my favourite DevOps setups was a big code base that used Kubernetes to deploy to. We effectively use a mono repo locally with a local cube Kubernetes instance running on our own MacBooks to effectively mimic the server. And we would use Azure DevOps to deploy to Kubernetes clusters based on each individual PR.
Our Kubernetes is obviously very cloud agnostic. We don’t particularly care where it’s deployed to, and we use as your DevOps to do that. Admittedly, it was to Azure, but I think that was coincidence, rather than by design.
It didn’t have to be and it definitely doesn’t have to be. And I think I really quite enjoyed the sort of power of some of the tooling that we had on that project to create environments that were quite ephemeral for spin up and tear down, as you create pull requests that you could run your integration tests against.
And that whole sort of setup definitely felt very portable, I would say, but also extremely powerful.
We had a team that was maybe 100 people full of not only developers, but also content editors, dedicated test team, other users who maybe designers or maybe information architects, all of whom could go and see their changes on particular branches deployed out to real environments on Kubernetes.
And so there are some really powerful things you can do with this, whether you choose to use some of the integrated tooling or not. And on that capability, we didn’t.
Zoe:
One of the words that you both used early on was investing.
And I know that whenever you’re thinking about investment and actually, either spending a whole lot bit of money on tooling, or on setup, or on just repurposing your developers away from building your core product, you have to think quite carefully about what the returns are.
So, how are the ways that you will start to see that making these changes is making a difference to your productivity?
Dan:
I think you have to see that a lot of manual processes are already pulling your developers away from their core process, right? So if that isn’t happening, then maybe this isn’t the answer for you. But on most, most places, either you’ve automated it, or you haven’t, it has to happen.
And so if you haven’t done it, I think you should just try and identify it already under your noses, sort of, and I think it should be quite possible to maybe figure out a rough estimate of what that’s costing you. And you could potentially make a business case based on that.
But fundamentally, I think, as with everything, you have to keep up with the current sort of state of play on these things, because software development has gotten more efficient over time. And you want to be part of that you don’t want to sort of be left behind on that, I think.
Zoe:
Thank you. Just finally, because we’ve talked a lot about software development, and obviously, that is our area of specialism. But how can DevOps techniques be used within the rest of the organisation beyond just the development team?
Joe:
Yeah, I think that’s really interesting, because we so far kind of framed it as this is a tool to make your developers more productive. But I think actually automating that whole lifecycle impacts far beyond your development team, or your ops team, for that matter.
So, as an example, we’ve been looking at what they’re calling the operating model for bringing changes all the way through from inception into production. One of the big things that I’ve done is separate content from the code.
So previously, (if) the digital team wanted to make some wording change on the website, they would have to go through the development team, the development team would have to schedule it into a release, the release would happen once a quarter or something like that and it would potentially take months to get that into the website.
We’ve separated that all out. The content is now in a CMS, the content editors can edit it there, they can publish it there. That fires out some web hooks that trigger a build and deployment, which sends out some approvals, the approval emails go to the small list of people that are allowed to approve these changes into production, they just click a button, and then it goes on the website. And you know, that whole process could happen within hours, whereas before it would take literally months.
And crucially, in that whole process, there’s not a developer to see. The developers set the system up but they’re not required to step in every time there’s a small wording change.
Zoe:
Yeah, I just want to say that’s fantastic because there’s so much energy devoted to making consumer tech easy to use. We’ve got this very low tolerance for having to click more buttons than we have to.
And yet on the business side, often we’re all working with systems that are then really labour intensive or frustrating. And actually, this sounds like the kind of making that process for the people working on content and approving content, to be as simple as using Uber to book a cab.
Joe:
Absolutely. We haven’t mentioned it, but I feel obliged to point it out: It’s not just about productivity here.
By introducing these processes, you’re making everyone’s lives easier. And that will also make them happier. Staff retention is really important these days. Making everyone have as nice a time as possible is actually a really important thing to bear in mind.
Dan:
Yeah, I think if you’re frustrated, and you’re using a computer at the moment, then it’s possibly worth seeing if there’s something you can automate out of it. I don’t think you should be in that situation very often.
Zoe:
Well, thank you so much, Dan and Joe.
I’m actually thinking I might retitle the podcast to ‘How to use DevOps to make everyone happy as nice to time as possible’. I think that’s definitely a good pitch.
But that has been super interesting. And anyone listening if you want to learn more about DevOps, or automating any part of your processes, drop us a line and we’re always happy to chat.
And please do remember to subscribe to our podcast wherever you usually listen.
