Project complexity: What makes software development complex… and how your developer can make it easier
23 May, 18 | illumisoftadmin
Project complexity is difficult to define (so get ready for some analogies)
If software development was easy, you’d do it yourself, right? You wouldn’t be trying to find a vendor for your next project. But what exactly does project complexity look like in development?
The experts can tell you. But leaving it to the experts has its pros and cons. Think of building a house (our favorite software development analogy). Construction is complex, so you have to hire a contractor. And although you can’t build it yourself, you still know how you want your house to look.
Project complexity in software development is similar. You know you want your product to work, even if you have no clue what makes it go. In fact, you may never learn how much your developer will handle on the back end of development.
Yet project complexity is also difficult to define, especially in uniformity across the software industry — which is why this blog is rampant with analogies.
There are two types of project complexity: Intrinsic and extrinsic. Both types can cause hiccups in your budget and delays in your timeline. Seek clear communication with the right developer. That is, a developer who holds themselves accountable. That way, you will get a clean, maintainable product that behaves exactly the way you imagined it.
Intrinsic project complexity comes from the project itself, and how difficult it is to solve your problem
Intrinsic complexity is inherent to the problem you’re trying to address with software.
That means all software development has intrinsic project complexity, simply because it exists, if you will. Any problem you’re trying to solve with software has some level of inherent complexity. That’s why you’re trying to acquire software, to simplify it and help you solve the problem.
You may not know how intrinsically complex your project is going to be. So, try to think of intrinsic complexity on a scale. An easy software development project could be creating a library cataloging system. And an extremely difficult project could be a rocket launch.
Think library catalog (very easy) versus rocket launch (very difficult)
Imagine, that you have a library that has no way of organizing their books. You have to try to track books as they come in and out of the library, or guide somebody to find a book. That could be a complex thing; but, if you compare it to the complexity of getting a rocket ship to actually leave Earth’s orbit, they’re worlds apart.
The real reason? A rocket launch has a much higher number of dependent variables compared to a library cataloging system.
A library catalog has few dependent variables but readily identifiable independent ones. For instance, how many books are in the library and how many constituents it serves. That’s why it’s so simple. Independent variables don’t affect one another when they change.
A rocket launch, on the other hand, is full of dependent variables like environmental pressure, temperature and gravity, to name a few. For example, as the temperature increases, so does the pressure. If one variable changes, the other changes as well. Rocket science is full of mathematical formulas — such as force = mass x acceleration — to calculate systems and their relation to the environment. Those formulas mitigate risks; a single error or miscalculation could be disastrous.
So, the more dependent variables your project has, the more intrinsically complex it is.
Intrinsic project complexity is inevitable to your project… but that’s OK
Intrinsic complexity in software development varies case by case. Good news for you! That’s why you’re hiring a developer, to worry about this. It doesn’t matter what problem you’re trying to solve. The complexity that’s inherent to the technology you’re developing is independent. You wouldn’t tell your contractor which tools to use when building your house. It’s the same thing with software: Let your developer create your product with the best tools for the job, and leave the intrinsic complexity to them.
In case you’re wondering, most business software is not intrinsically complex. That’s simply because most business processes are pretty straightforward. Businesses are man-made structures, which tend to be simpler than natural structures.
Hundreds of years of business best practices have simplified processes. They are simple to the point where people were able to run businesses very easily before computers came around. By the time a business owner needs software, the business has already been simplified.
Libraries have been around long enough that people have thought about it and simplified it (think Dewey Decimal System). You don’t need a specialist to run a library’s day-to-day operations. On the other hand, you have to hire a rocket scientist in order to operate NASA.
A rocket launch involves natural physics because scientists are attempting to model the natural world. So, there’s a lot of complexity in the natural world that we have to model accurately in order for the software to serve its purpose.
“If we had to create software that kept track of people clocking in and clocking out of the rocket laboratory, that’s pretty easy,” said Dustin Ewan, vice president of software development at illumisoft. “If we have to create a software that’s modeling a rocket launch, that’s hard, because one’s manmade and one’s natural.”
Another example might be artificial intelligence. Engineers are trying to model human intelligence with software. That’s an intrinsically hard problem because it’s a natural phenomenon that humans are trying to model.
Extrinsic project complexity comes from your developers, and how good they are at solving your problem
Extrinsic project complexity, on the other hand, is not inherent to the problem that you’re trying to solve with software.
Let’s say you took your problem and tried to solve it on paper. And let’s say your problem is a rocket launch. No matter which way you slice it, trying to do all the math for a rocket launch is just hard on paper. Keeping track of books that come in and out of the library on paper, however, is pretty easy.
Now you’re trying to translate your problem into software. That’s your starting point of project complexity. Whatever you took from paper into the software is the intrinsic complexity that already exists in the software project.
So, no matter what your developer does, they cannot reduce the project complexity past that point because it’s inherent to the problem. It only gets extrinsically complex when your developers become careless about how they write the code or how they model the project.
“It can add extra layers of complexity on top of the complexity that already exists in the problem,” Ewan said. “There’s gonna be some extrinsic complexity that’s also inevitable, but it’s within our control as a software development team. We can utilize Clean Architecture and other organizational tools to simplify the code and remain accountable to your project.
“It’s all about having a system in place.”
The bigger your project, the more important it is to organize code
Let’s revisit our paper analogy.
You can’t fit your whole problem and the solution on just one sheet of paper. You need lots of different sheets of paper. Here’s where it gets extrinsically complex. Let’s say you had a nice stack that was all bundled together, and you had those pieces of papers numbered in order. Then, you can easily go through it and find what you’re looking for. What if you took those pieces of paper and just threw them onto a shelf and you needed to go back and find it later? Then you end up trying to dig through all of the paper to find the lost sheet.
If you have a small project with low intrinsic project complexity, this might not matter too much. What if it grows to where you have filing cabinets full of unorganized paper? You’re simply growing the project organically. Whenever you return for it — perhaps to make a change — then you have to dig through every single piece of paper to find it again.
When your project has dependent variables, coding in one place could change something in another place. You can imagine how that might get really complex.
So, as your project grows and the number of dependent variables increases, then the complexity grows as well.
Think of the butterfly effect. Your developer might repeat their work in multiple places instead of making a calculated function (think copy and paste). That means they’ve produced the same code organically each time. If their code has the same error throughout, that error has been duplicated everywhere, every single time. Not only does each bug have to be corrected individually; but now, that unorganized code must also be sorted through.
Complexity is risky; expect a lot of communication and accountability from your developer
Whether it’s intrinsic or extrinsic, project complexity poses risks for you. A successful project will have clear communication, accountability and dedication toward your budget and schedule. A good understanding of potential risks from your project complexity will help you know what to expect when hiring a developer.
Firstly, developers are generalists. We have a general breadth of knowledge — about law, about science, about business — but that poses a risk with your project’s intrinsic complexity.
Communication alleviates intrinsic project complexity
The more intrinsically complex your project is, the more your developer will rely on you to verify their work. That’s why communication is one of our four pillars to project success. If you and your developer are not communicating well, there’s no way your developer can make software that accurately solves your problem… because it’s so intrinsically complex.
“The more intrinsically complex it is, the more your development team has to be like an extension of yourself,” Ewan said. “There’s got to just be a lot of back and forth to verify that the work that we’re doing is accurate.”
If your project is intrinsically simplistic, that creates buffer space between you and your developer. Communication can be a little bit more loose, just because the project is simpler.
On the other hand, if your project is extrinsically complex, then you will probably notice the impact on your budget and schedule.
Accountability reduces extrinsic complexity
All projects have some extrinsic complexity. Every project has overhead costs of communicating about the project. That’s why it’s so critical for your developer to be accountable to your project.
When things become extrinsically complex, your developers have to communicate to each other about how the code is organized. So, if the code is scattered and disorganized, then they’re no longer talking about problems that are inherent to the project. They’re just talking about how the code is organized. That’s why it’s more costly during development. Developers must spend time on overhead, communicating the complexity to one another.
Extrinsic complexity builds technical debt; make sure you avoid it
Another risk of extrinsic complexity is exposure to technical debt. Technical debt is basically excess, unnecessary complexity in the project. And, ultimately, that technical debt is going to have to be paid off if you ever want to change or grow your product. Technical debt is always extrinsic.
The problem with technical debt is that you never know where danger lurks in it.
Take Knight Capital Group, for example. The investment firm accumulated so much technical debt that it went belly up in a matter of minutes.
Their software project was growing organically. Developers were being really fast and loose with how they were doing it, and not being accountable to it at all. The firm’s system had eight computers that it ran on. They issued an update to the software. One of the technicians responsible for deploying the update across all eight of those servers forgot one of them. That exposed a bug that had been hidden deep within their code. The system suddenly started wildly selling, buying and trading on its own. It lost $440 million in less than an hour.
All because of technical debt, their system was so extrinsically complex. There was no way a developer could have known that without investing tons of time and money into auditing the code. And if they would have been accountable and did some cleaner code upfront, it could have all been avoided.
So, you can either pay your developer upfront — with clean but more expensive code — or you can pay it back later with who knows how much compound interest.
Save time and money down the road by hiring a developer who understands your project complexity
Your budget and schedule are dependent on the intrinsic complexity of the problem. No matter who develops your software, the intrinsic cost of your software application will be the same. And, whether you hire somebody for $30 from overseas or you hire somebody for $300 an hour in the United States, the cost to actually solve your intrinsic complexity will be consistent.
A higher-quality, $300-an-hour developer will reduce your project’s extrinsic complexity upfront. On the other hand, if you go with somebody cheap, they’re going to accumulate extrinsic complexity. So, you’re going to pay for it in technical debt later.
If you’re urgent to develop your product, it might be worthwhile to assume some technical debt with cheaper, quicker labor and get it into the market. That only works as long as you can acquire revenue more quickly than you acquire technical debt. You would need to end up making so much money so quickly so that it’s a net positive for you to pay off that technical debt at a faster pace or at a higher price.
A fair warning: Cheap labor is like any other product. You get what you paid for.
If you hire your developer cheaply, you might’ve thought that you were getting a good deal. They made software for you that’s working for the business; you’re making money off of it. What if there was a bug in the system? You’ll end up with technical debt that you had no idea you had. And that technical debt is a true cost of the system; in that case, unnecessary operating costs because of bugs.
It’s like Knight Capital Group. They should have put that $440 million toward more expensive developers who would have avoided bugs. It would’ve been cheaper in the end, even though the actual hourly cost of writing the software was much greater.
Project complexity is difficult to discuss; luckily, it’s something for your developer to worry about
We know software development isn’t easy to discuss when you don’t have a tech background. That’s why your developer should help you understand by using analogies like building a house and organizing filing cabinets.
You know what you want your product to look like, even if you don’t see how complex your project could get. And that’s OK. It’s your developer’s job, not yours, to handle project complexity.
Ultimately, understanding how complex your project can get is a step in the right direction. It gives you a better idea of what to expect when hiring your developer.
Have some ideas? Give us a call, and let’s re-imagine your next project together.