KPIs Can Cause Incidents!!! - Bad metrics produce bad outcomes.
Recently, I was going through old emails and found a reply from a junior colleague to a very serious email I had sent. The colleague wrote that after reading my message, they realized they had been mindlessly following instructions without deeper consideration. They promised to carefully consider the ethical implications and correctness of every task, and to proceed based on their own judgment going forward.
Upon further investigation, I realized this colleague managed our vulnerability tracking system. They had been instructed by their team leader to uniformly downgrade the severity ratings of high-risk vulnerabilities. My email had warned them about the potential ethical problems associated with such actions. (Although much time has passed and things have changed, this colleague was very sincere at that time…)
Several years ago, around the year-end performance evaluation period, a team leader tried to artificially boost KPIs related to vulnerability remediation—metrics difficult to control directly. This unethical action made me curious about the potential negative impacts.
After reviewing past vulnerability assessments and incident records, I discovered actual examples where manipulated KPIs led to cybersecurity incidents. Although specifics can’t be disclosed due to security reasons, news articles such as “[Exclusive] Hacker Redirected Bank SMS Authentication Codes, Bitcoin Accounts Emptied” indirectly reported these issues. (Call-forwarding wasn’t the only possible method used by attackers.)
Without KPI pressures, staff would have operated normally, potentially preventing these incidents. However, in modern organizational structures, KPIs cannot simply be removed.
Was the problem the way KPIs were structured? Evaluators naturally prefer result-oriented metrics—either incidents or vulnerabilities prevented—which limits alternative approaches.
Was the KPI management process too loose? Would tighter controls and more frequent feedback have prevented this issue? Actually, at that time, we had already formed a dedicated task force that regularly provided feedback on vulnerability risk ratings.
Ultimately, over time, I’ve realized KPIs for evaluating leaders have become largely ceremonial. Peter Drucker famously said, “You can’t manage what you can’t measure.” However, in organizations created and managed by humans, purely mechanical evaluation was flawed from the start and susceptible to manipulation by human desires.
Can we truly manage organizations effectively through metrics alone? Can businesses prioritize essence over appearance?
Works Cited
“[Exclusive] Hacker Redirected Bank SMS Authentication Codes, Bitcoin Accounts Emptied.” Yonhap News Agency, December 3, 2017. https://www.yna.co.kr/view/MYH20171203004600038. (Accessed June 16, 2024)
“Bad metrics produce bad outcomes.” JoongAng Ilbo, March 5, 2017. https://www.joongang.co.kr/article/21337981#home.