Bad civic hygiene

Bruce Schneier has a great cautionary quote about technology and its tendency to be subverted:

It’s bad civic hygiene to build technologies that could someday be used to facilitate a police state.

He’s used the line in a few essays: one about data reuse and the role of the US census in Japanese internment, one about Chinese pornography filters that snoop on Internet users, and one about a failed Obama administration proposal for backdoors in encryption systems, for example.

It’s typically insightful of Schneier to connect these stories with a common thread. Independent of the primary purpose of personal data collection, which can run the gamut from necessary to useful to invasive, the secondary purposes can be much worse. It’s sometimes difficult even for thoughtful people to consider the primary and secondary purposes of data collection independently, or to consider a technology’s designers responsible for its secondary uses.

That’s why the “hygiene” metaphor is so apt. Good hygiene isn’t mandated by law, but people understand that it’s an important practice. It makes cooperation with society smoother, and reduces the prevalence of parasites as systems get increasingly complex.

One example: companies that track users across the web often defend the practice by explaining the virtues of their primary purpose. They may help monetize free content, or deliver more customized materials to users, or something else that society considers beneficial. But these arguments miss the point. Obviously, if the primary purpose is bad for society, it’s a short conversation that ends there. Even in the case of a compelling primary purpose, though, having a good security mindset means thinking about the ways a system can fail. Allowing a society to inch towards a police state is a pretty serious failure.1

For people making new technologies and businesses, it makes sense to not just consider how profitable or efficient or helpful they’re being — that’s their primary purpose — but how their developments can be abused. Skipping that step can lead to major problems for society. As Supreme Court Justice Louis Brandeis said, “The greatest dangers to liberty lurk in insidious encroachment by men of zeal well-meaning but without understanding.”

  1. “Harmless failures” are bad too. Ed Felten refers to those also as bad hygiene. ↩