Test it Before You Send It Out There
In the summer of 2012 Knight Capital Group made a mistake that could have happened to anybody.
On August 1, 2012, Knight Capital deployed untested software to a production environment which contained an obsolete function. The incident happened due to a technician forgetting to copy the new Retail Liquidity Program (RLP) code to one of the eight SMARS computer servers.
This simple, seemingly inconsequential act of negligence triggered a series of events which ultimately caused Knight Capital to self-destruct in spectacular fashion.
Knight Capital sent millions of child orders, resulting in 4 million executions in 154 stocks for more than 397 million shares in approximately 45 minutes. Knight Capital took a pre-tax loss of $440 million.
The $1.5 billion company evaporated overnight with nearly 1500 people losing their jobs.
Admittedly, Knight Capital is an extreme example, but a lackadaisical attitude toward testing is all too common in the software world. While most business will not collapse overnight, failing to test properly can negatively impact a business in many different ways: missed orders, incorrect billing, deletion of crucial business data, or even compromising sensitive personal information.
Critical failures due to testing negligence appear in the news more and more as business’s reliance on software continues to grow, and it is all avoidable.
Learn more about software testing and the whole development process by calling and asking questions. We invite you to call us today 816-564-9595
Shortcomings of Manual Testing
Every software company employs some degree of manual testing. Most commonly, the company will start with nearly 100% of their testing being handled manually and as the application continues to grow the amount of testing required usually grows exponentially.
Every feature implemented in the software requires a suite of tests to guarantee it’s correctness including, but not limited to, positive tests to ensure the feature does what it’s supposed to do, negative tests to ensure the feature does not do what it is not supposed to do, boundary tests to verify that maximum and minimum inputs are supported correctly, and many others.
A company is forced to make tough decisions in order to maintain an acceptable amount of testing as the application grows. They can write off certain tests completely to focus on “critical” areas, hire more and more manual testers, or begin to implement automated tests to replace existing manual tests.
Simply writing tests off seems like such an obvious negligence to testing, but it does happen, often in fact, and usually the tests that are ignored are tests that have been ran and shown to be working so many times that the probability of them failing is extraordinarily low. That being said, there is always a chance that skipping the test may fail to catch a consequential bug.
Hiring more testers may seem like a better solution, but it is a costly solution. Furthermore, humans err. Tests can be run incorrectly when testers don’t understand the application well or even something as trivial as the tester not getting enough sleep the night before.
Automated Testing from the Ground Up
The cheapest and most consistent solution is to transition toward automated testing. In fact, automated tests are best written in tandem with the features they will be testing. The role of the manual tester then becomes verifying that new features satisfy business requirements and verifying the automated tests are testing the features correctly.
Adhering to a methodology in which features are not considered complete until automated tests are written and verified will guarantee that future features and bug fixes will not break existing features. Automated tests are like a contract that the software will perform exactly what the business intends. If any code is written that will break the contract, then the tests will fail and notify you immediately.
Some development teams even go as far as writing testing requirements up front with the product owner and having the owner sign off on the tests. The code is then written to pass the tests; this methodology is called Test Driven Development.
Automated Deployments Guarantee Integrity
After the code is written and has passed all testing successfully it is time to deploy the code to production. A simple definition of deployment is taking any and all steps necessary to update the software and make it available for use. Nearly everybody has encountered the most common form of deployment, installing software on your computer. You have probably installed a web browser, email client, or office suite on your computer before. The process is simple and automatic; simply double click on an installer, maybe change a few preferences, and the rest is done automatically without your concern of what steps are involved.
Believe it or not this process is not all that common in the enterprise world, and was the downfall of Knight Capital. Their deployment methodology involved manually copying updates of their software to multiple machines, but they happened to miss one.
Knight Capital could have avoided self-destruction entirely if their deployment process had been automated. Automated deployment simplifies the entire deployment process down to one step: click go.
The rest of the process is handled automatically and if at any point the process should fail then the previously known working version of the software will automatically be restored. Automated deployment would have guaranteed that Knight Capital had the correct version of their software installed on all of the machines in their system and guaranteed that the code they were running was indeed the code that they tested.
Automation, whether it is the automation of your business processes, testing of your code, or deployment of your software, guarantees that your business’s efforts are performed correctly, cheaply, and consistently.
If you are curious about how automated testing and deployment can help your business please reach out to me or visit our website at www.illumisoft.com.
To Email Us:
7280 N.W. 87th Terrace Suite C-210, Kansas City, Missouri 64153
Silicon Valley Office:
530 Lytton Ave, Palo Alto, CA 94301
Seattle Washington Office:
2211 Elliott Ave, Seattle, WA 98121
New York Office:
405 Lexington Avenue, 26th Floor,Manhattan, New York, 10174
7280 N.W. 87th Terrace Suite C-210, Kansas City, Missouri 64153