Under what circumstances is spying acceptable or essential and why?

[Pholus]

5. When it involves *reasonable* suspicion of wrongdoing.

That pretty much includes both foreign and domestic spying.

The tortured parsing of legal terms that justifies the current programs we actually are being told about is not in the best interests of our democracy. See Krugman's commentary on this week which basically explains there would be two ways to do domestic surveillance: in an open, limited "democratic" fashion in which the data collected is limited and the populace informed and in a closed, secretive "authoritarian" manner which is the opposite. We have so obviously chosen the latter route, with rumor and whistleblowers having to motivate the conversation instead of an open and honest discussion. This is not the way a democracy functions. I did not have a say, even one overwhelmed by popular opinion on the other side, in this matter and I resent that.

And once this data exists, it will certainly find new uses which we will not be told about up front. Yesterday the Washington post ran a story about drivers license photo databases. Originally created solely to prevent fraudulent use of the licenses, they are now being used to conduct "virtual lineups" of suspects and also being used to match surveillance video with facial recognition technology. This means every US citizen in that database, regardless of circumstances, are de facto suspects in police investigations now. So much for presumptions of innocence.

And even now, the facts about what is being done are still being hidden from us "in the name of national security." I see estimates of a trillion phone calls per year in the US. If each call has 80 characters of "metadata" (number called, duration, only a couple other things as we've been told), 80 trillion bytes would be 80 Terabytes/year. This "database" would easily sit 4-5 blade servers in someone's office. Furthermore if we are to trust what we were told just yesterday, only 300 numbers were targeted. My personal research data takes 20 TB so I can say with certainty that the project officially described is easily capable of being run out of a single office room on a single server by a team of 4-6 people.

So why do we need a bunch of new multi-billion buildings in Utah capable of holding 5 zettabytes (10 years of EVERYTHING by a quick pencil and paper calculation) if the program truly is this small? Why are there stories about the U.S. government being interested in all kinds of commercial databases? Why does a quick perusal of the DARPA website talk about opportunities to do "anomaly detection" in massive databases.

Nope, this is so obviously about more than a limited collection. Someone, somewhere made the successful sales pitch to these guys that you could build a system like the ones we see on CSI on teevee. When you know the name of the bad guy, press a button and on those massive floor to ceiling monitors on the "command center" wall up pops their picture and every significant bit of data ever collected on that person like you'd had a gumshoe on their tails for a decade. Worse, there are probably promises that you can take all that data, put it in a pot, wave a magic wand and out pops all the bad guys (people who don't fit the mean behaviors of the population). Once again, we are all suspects who will need to prove our innocence rather than them proving our guilt.

The problem is oversight at that point -- given that of those in "the know" have split opinions about whether oversight is sufficient I am not mollified. It is too easy to remember the massive misuse of the limited resources available in the 60's to ever think that this will not grow into stifling anyone who might become a threat to the power structure in place now. Already, both Snowden and DiFi outline procedures which seem to indicate that an individual analyst has a lot of query power on the data available. The potential for personal and systemic abuse is massive, the stakes and payoffs high so it *will* happen.

I love the quote from Orson Welles' movie: "The job of a policeman is only easy in a police state." By that criteria, what does big data represent?