This is part 2 of my article titled ‘Got ‘Bias’ in Your Organization? Outsiders Welcome!’ If you would like to read Part 1, click here.
While sitting with Matt and racking our brains out for a resolution, I shared with him that I really feel our programmers were suffering from the same bias my daughter had but possibly in a different way. A term came to mind that I heard before from my MBA studies: ‘systemic bias’. Here is what a quick Internet search showed us about ‘systemic bias’:
Systemic Bias is inherent in the experiences, loyalties, and relationships of people in their daily lives, it cannot be eliminated by education or training, but awareness of biases can be enhanced, allowing for the adoption of compensating correction mechanisms. Soure: Wikipedia.com
‘Correction mechanisms’ caught our attention! What correction mechanism could we implement to resolve our problem? After reading this, another famous political quote came to me from my favorite United States President, Ronald Reagan. President Reagan famously coined the phrase “Trust but verify” during his negotiations with Russia General Secretary Mikhail Gorbach over the reduction of nuclear missiles. (In Russian, Trust but Verify is translated to Doveryai no Proveryai, a Russian Proverb.)
In a meeting with Jarret to review our budget, I felt I had a possible solution to resolving the problem. I asked him for additional resources to hire an outside consultant to re-evaluate our code development process and see if that outsider could help us identify the cause of the re-appearing bugs. Jarret agreed.
Immediately, I hired an engineer that had previously worked for the Department of Education in helping their team develop best practices for source code and had vast experience with our hosting platform Microsoft Azure. Our new engineer began by reviewing our source code management process as well as meeting with each programmer to follow each of their individual processes for authoring, storing, testing and posting source code to production. Overall, we had a good system that he classified as “better than most.” Yet, his research showed that our processes had room for improvement.
Slowly, we started to institute new processes as brought to our attention by our consultant. Many of the processes were actually documented by Microsoft, the developer of many of our tools such as Microsoft Visual Studio, Microsoft Azure, Microsoft Team Foundation and Microsoft SQL Server. One of the main problems he identified was how we took our production code from BETA testing to LIVE production. The process included our programmers manually copying code from our BETA testing website and then pasting it to our LIVE website. He felt this process was too manual and ripe for human error.
To resolve this problem, he introduced our team to the concept called “continuous integration”. As the name implies, the concept is to continuously integrate new code instantly into our testing environment as well as to a staging environment, one step short of going live in our production servers. The system he introduced us to allowed all the programmers to work on code at the same time, all simultaneously and automatically. After a quick test on the code, we literally pushed a button and the code would be instantly moved to production ready for our real users to utilize.
With the use of continuous integration, we reduced our time to go live with new code from 5 hours to less than 5 minutes and also removed the likelihood of human error occuring during the process of taking code live. From start to finish, overhauling our process took several weeks. As an outsider, he helped us improve our systems which in turn improved our user’s experience – the ultimate goal. Trust by verify is no longer just a quote for us, it is a way of life.
Why were our existing programmers not able to resolve this problem on their own? It was a bias to routine and their goal not to disturb that routine. It takes great effort to change the way you do something and if you helped develop those processes, you will have even further bias toward protecting them. By bringing in an outsider to review our processes, we were able to discover more efficient methods that helped us and our team evolve.
So my takeaway for resolving potential ‘bias’ in your organization is to bring in an outside expert to re-evaluate every stage of your product development life-cycle. Outsiders don’t come with systemic bias and bring a fresh perspective. In the business of building a fast and scalable website that users love, “better than most” just won’t cut it any longer. To date, no bug has re-appeared since we implemented continuous integration. We have Ronald Reagan and his ‘Trust but verify’ philosophy to thank for helping us solve the problem and General Patton would be proud that we are no longer paying for the same real estate twice!
