In the past couple years, I heard China is implementing a “social credit” system that determines how trustworthy people are in the eyes of the government. More recently, New Jersey opted to get rid of cash bail in favor of a system that determines how much of a flight risk the defendant is.
I’ve been conflicted about both of these systems. First, let’s tackle China’s social credit system. In order for this system to exist, mass government surveillance must exist, as well as corporate surveillance which collects data on purchasing behavior, tastes, interests, likes, dislikes…but of course, much of this information can be inferred from social media behavior. Is it really so strange to score citizens based on their social media behavior? I’m undecided – sway me.
As for New Jersey’s bold move to eliminate cash bail (for the most part), there are some interesting implications, the first and most important being that wealthy criminals can’t hide behind their money anymore. The worst of criminals often have large cash reserves from their criminal activity, and they use this to post bail, where the poorer prison population cannot afford such a luxury (even if they’re unlikely to be a flight risk). Another positive element of this system is that repeat offenders and flight risks are more likely to be rejected by the system.
However (and there’s always a “however”), what if you come from a family that is less-than-friendly with the law? What if you associate with some unsavory characters short-term – should that ruin your standing in society? What if you come from a neighborhood that is statistically more likely to engage in criminal activity? Are you to be coldly cast aside because you didn’t meet the minimum heuristics of an unfeeling computer system? Or is a computer better able to make these decisions because it can be programmed to be impartial?
If the latter is true, then we as software developers have to have a higher moral and empathetic standard than usual, because the systems we design can make or break someone’s very life. Is it worth it?