Understanding and Solving Five DevOps Challenges

You’re ready to change how your organization builds, packages, and deploys code. You want to shift to DevOps. But you’re wary about it, too. As you start (or continue) these initiatives, you know that you’re going to encounter DevOps challenges.

It’s easy to talk about DevOps, and sometimes it seems like it’s just a matter of installing a few new tools, automating some tests, and moving on to the next buzzword. But that’s not what DevOps means, and it’s not that easy.

Let’s go over five of the most common challenges and how to approach them.

A person with idea lines out of head solving some DevOps challenges.

Tackle Resistance to Change

If you or your organization isn’t willing to change, your DevOps initiative will sputter or fail.

DevOps demands a significant shift in how you build, package, test, and deploy code. The traditional boundaries between developers, systems engineers, QA testers, and product managers need to come down—or at least be more porous. DevOps tools and automation aren’t only a way to simplify processes, shorten lead times, and save money: they’re part of a different approach to building and delivering software.

At the same time, many people are resistant to change. It may be because they see the organizational changes as a threat. When a department head sees their reports scattered across a set of projects instead of getting daily work assignments from their manager, it’s understandable when they question their future in the company. When you tell a QA tester to automate their tests, they wonder if they’re eliminating their own job.

There are answers for these objections and others like them. There’s a department head role in establishing best practices, hiring and firing, and resource (instead of task) allocation. A QA tester skilled at automation is more valuable and gets more done than one who runs test scripts by hand.

It’s also true that change makes many people uncomfortable. To them, a bold new DevOps initiative looks a lot like this year’s reorganization plan. They need to see how the shift is about a new methodology that improves their productivity and value to the organization.

Regardless of the reasons, resistance is something that DevOps leaders need to plan for.

Create Effective Cross-Functional Teams

Even after you get past the resistance to change, you still need to integrate your teams. Collaboration and communication between functions are key to success in DevOps. You could argue that it’s important for any company’s success, and you’d be right. But it’s particularly critical that you tear down your development and operations silos and replace them with cross-functional teams.

Resistance is something that DevOps leaders need to plan for.

How you form these teams depends on your specific situation. Many organizations have had success with centering teams on products. Others build and reconstitute teams based on projects. No one size fits all.

But there is a size that doesn’t fit. Installing CI/CD tools, mandating that all tests be automated, and then saying, “We do DevOps now!” is not a recipe for success. Hiring some DevOps engineers and setting them up in contention with your existing development and IT teams will fail, too.

Integrating development, operations, and those new DevOps engineers is not an easy task. There’s a good chance that you already have your development teams separated into different roles based on front end, back end, or platform. Your operations teams probably have operations and engineering functions. Now you need to get them working together. Deciding how they will engage with each other is a critical step toward success.

Unify Workflow and Tools

Integrating teams doesn’t just mean putting them in the same room and inviting them to the same meetings.

DevOps puts development and operations teams together. But if they continue to use different tools and approaches to get the job done, they’re not as effective as they can be. Depending on your organization’s size and history, they may be using different project and issue tracking systems. They may have adopted the same system but might be using drastically different methodologies.

A conflict over something as simple as where and how to store documentation can prevent two or more teams from jelling into one.

Which tool is the best one? Who should win the debate? These are contentious issues, and people will have emotional and practical reasons why their way of doing things is better. Regardless of which path you choose, there will be cultural and training issues to contend with.

But, to be honest, the answer to which tool or process is best is less important than getting past the debate and getting to work.

Modernize Legacy Architectures

Another potential obstacle is the very systems your teams are going to work on together. If you’re still supporting a rigid legacy architecture, then your organization may be shaped around making sure that it doesn’t break. There may be a set of handoffs and checkpoints that mimic the shape of your systems.

You can’t change your architecture overnight, but you can start a DevOps initiative and figure out how to get your teams to work together. The first step is to stop accepting “but we’ve always done it this way!” as a valid response to questions over both organization and architecture.

DevOps came about as a response to dysfunction in the IT industry. As products grow more complex, the systems that support them grow the same way. DevOps is an attempt at reducing that complexity with improved procedures, tools, and organizations. When you adopt DevOps, you’re saying that you want to simplify your architecture parallel with your processes.

So, part of your DevOps mission is to look for ways to take advantage of new platforms and tools. Don’t adopt them because they’re there and they’re new. Figure out how to use them to simplify your systems, decrease your delivery times, and increase your system reliability.

You can’t change your architecture overnight, but you can start a DevOps initiative and figure out how to get your teams to work together.

Monitor High Data Volumes

How can you make your new architecture faster, stronger, and more reliable? Our systems reach more people, run faster, and produce more information than ever before. They produce enormous amounts of information. Are you using that data to monitor your systems? Are you using it to learn more about your customers? If you’re not taking advantage of this capability, you’re at a competitive disadvantage.

As your DevOps teams work together to modernize your systems, they’re in a good position to recognize these opportunities. They’re also more apt to work together to ensure that assets like application logs are in a format that’s easy to use. So, the next step is to take those logs and use them to capture events. That’s where an architecture like Scalyr’s Event Data Cloud comes in.

An event data cloud gives you a way to transform your log information into a tool for observability. An effective system like Scalyr’s provides you with tools to take the stream of data and events your systems produce and turn them into data for alerts and dashboards. You can even use your event data cloud as the single source of truth for what’s happening inside your systems. Finally, it’s also a valuable tool for training AI systems for MLOps and customer service.

Don’t Let DevOps Challenges Stop You

DevOps requires more than one transformation. Before you start changing your technology, you need to build new teams. Before you can build these teams, you have to adjust your organization. After you form them, you need to make sure they’re all marching in the same direction.

But that’s not a reason to stop. It’s not a reason to try to go halfway, either. There’s a reward at the end. When you make the switch to DevOps, you make it easier for your staff to deliver better products. You’re also making it easier for them to keep those products running and up-to-date.

So, get started today. No DevOps challenge is too big for you to face!

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).