General Discussion
Related: Editorials & Other Articles, Issue Forums, Alliance Forums, Region ForumsThe capability to store and analyze 100 years worth of worldwide communication.Are you ok with that?
Everytime you see a propaganda post on these forums where people deflect to discussing things like Greenwald's view on Iraq right after 9/11, remember this:
The new Utah Data center which is almost finished has the capability to handle and analyze 5 ZETTABYTES of information or as Will Binney another NSA whistleblower pointed out:
at least something on the order of 100 years worth of the worldwide communications, phones and emails and stuff like that," Binney asserts, "and then have plenty of space left over to do any kind of parallel processing to try to break codes."
http://www.npr.org/2013/06/10/190160772/amid-data-controversy-nsa-builds-its-biggest-data-farm
Given the revelations that have surfaced about the NSA, do you honestly believe they would need 5 Zettabytes if they were not storing information on or snooping on regular American citizens? To put it into perspective, the entire population of the United States consumed 3.2 Zettabytes of data in one year. Every video, every movie, every communication, broadcast, picture, book, or other form of data consumption of every human in the US only amounted to 3.2. And the NSA will be able to store/analyze 5.. in one data center... unaccountable to the public.
Ask yourself if you are OK with that. Ask yourself if you are OK with that misplaced power in the hands of an administration that's not Obama's. Ask yourself if you want to continue to elect individuals who feel that something like this is necessary.
Forget about Snowden, Forget about Greenwald, Forget about Obama. Focus on the issue at hand and think..think about the potential for misuse, blackmail, and corruption. Is this the America YOU want?
cantbeserious
(13,039 posts)eom
Spitfire of ATJ
(32,723 posts)Douglas Carpenter
(20,226 posts)If there is a Republican President in the White House - then I am NOT okay with that.
a2liberal
(1,524 posts)Douglas Carpenter
(20,226 posts)reusrename
(1,716 posts)A yottabyte is 1024 zettabytes.
I know, it's HUGH!!1!!1!!!
marions ghost
(19,841 posts)Yottabyte City out there...
Pholus
(4,062 posts)100,000 square feet of server space can't store that much information yet. And considering how cheap they keep saying this is, they have to be using commodity hardware (as does Google). Brewster Kahle (of the Internet Archive) keeps on top of this, and some newly released blueprints in Forbes shows less efficient use of the space than thought.
Brewster's new estimate is 12 exabytes. That's still pretty good -- 12 exabytes is 12000 petabytes and that is 44 years of all phone calls made in the US (272 petabytes/year by Brewster's estimates). Paul Vixie says lower, more like 3 exabytes. That also seems to reconcile a difference between the announced power capability and Brewster's estimates which were a factor of 3-4 higher. The founder of one of the server companies thinks its more like 1 exabyte. Either way, it is still between 4-10 years of voice communications. And 0.5 - 1 year of all domestic US traffic.
And there are several facilities besides this one of course -- the NSA likes big data these days.
http://www.forbes.com/sites/kashmirhill/2013/07/24/blueprints-of-nsa-data-center-in-utah-suggest-its-storage-capacity-is-less-impressive-than-thought/
reusrename
(1,716 posts)I could be wrong about this, but I don't think I'm very far off base here. Some of these numbers that are getting tossed around represent different ways of counting "digital data."
A huge amount of what travels around in digital format is streaming video and such. This is a very different amount than the quantity of data contained in digitized phone conversations and emails. AFAICT, they are storing all digital transfers of information.
There are many people, including the folks at Wired and Slash-Dot, who are saying the yottabyte figure is very much in line with what is going on at the Utah facility. In any event, all those flat-earthers who are claiming that the internets are just too big to save everything, they're just plain wrong. They are living in a fantasy world.
Pholus
(4,062 posts)So true enough. I don't believe there is technological black magic going on though there may be misdirection. One of the biggest ones I see is that every released dimensions for the data center only talk about floor space. Nobody says anything about heights. That could add a factor of 10 right there given how large those buildings are....
And absolutely -- I think it once it was realized that it was possible we are saving everything or as much as we can get away with. And it doesn't have to be raw traffic from every device -- you'd do a lot of brains disservice not to admit that there are ways to save orders of magnitude on what is taken yet still essentially get "all" of it. Perhaps they add some brains to filter stuff -- you know, only keeping one copy of something repeatedly served from a caching server while saving all mail traffic for example.
I'm hardly clever, but I could set up something to do this. It would suck, be slow and bloated and unreliable, but in principle it would work -- that means the really clever guys who eat, live and breath this stuff know how to keep it lean and fast. Therefore it's being done.
I don't really care about the whining from some here about how tinfoil hat this seems. We're not talking about UFO's or Bigfoot -- we are talking about uses for an existing facility and existing technology. If it were just "metadata" it would sit in a single server room. This is a server building. The government is buying every commercial database they can lay their hands on -- scan some trade publications for that fact being repeatedly mentioned. They have requests for proposals for data mining algorithms describing a broad range of data and how they're searching for "anomalies" in that data. They have their National Security Letters and Snowden's slides show they have some level of cooperation from the large players on the internet. They also live in a culture that sees a lack of data as a hinderance if they have to move quickly on something. The reason that General "Collect it all" Alexander is so popular is because he did EXACTLY that in the Middle East and it paid off for us. Now he's promoted -- I don't think it's so he tries a completely new strategy.
TIA was the event that moved me from being a passive lurker to an active poster here back in 2002. I realized during Bush's "reign" that there was an unseemly interest in trying to "track everything" and saw that old "TOP SECRET" label cover a lot of crap up just to prevent embarassment. It's the same people Bush hired, even if they are operating under a Democratic flag, and they're doing the same bullshit.
reusrename
(1,716 posts)Yep, TIA was Poindexter's piece of the pie. All of the Iran/Contra criminal cabal seem to be involved in some fashion or another.
Most of the whining about tin-foil-hattery comes from folks who deny that this shadow government even exists. Do you think it's possible to reach these people?
Our discussions might be our most effective way of organizing and moving forward. I'm glad you have joined the conversation, no matter what the reason. I sort of came onboard during the shame and horror of the Katrina aftermath. The application of the Schwarzkopf doctrine ("we don't do body counts" to that national tragedy still enrages and sickens me.
struggle4progress
(118,294 posts)reusrename
(1,716 posts)struggle4progress
(118,294 posts)reusrename
(1,716 posts)I doubt it will cost too much more than the wars in Iraq and Afghanistan.
Or the bailout for the banks.
They'll just raid Social Security in order to continue to pay off the criminal cabal.
jmowreader
(50,559 posts)NSA wouldn't do solid state drives - too expensive, too small and not yet reliable enough. So...if they're running 3-terabyte hard drives, they'd need 36 billion drives to hold one yottabyte.
Ignore two salient facts: a data center big enough to hold that many drives would be bigger than the new World Trade Center, and it'd need its own nuclear reactor to power it, and consider the feasibility of actually putting your hands on 36 billion hard drives.
Thirty-six billion hard drives is probably more hard drives than have ever been made in the history of computing, including the 20-megabyte one you had in your first PC, the 300-megabyte CDC 9766s we had with their 14-inch platters*, and all the ones that no longer work.
Considering some of the garbage that's been written about NSA, the claim that the Utah data center is going to have that much capacity is really pretty laughable.
* One of the sweetest sounds in all computing is listening to a CDC 9766 spin up and purge. Sounds like a little jet engine.
reusrename
(1,716 posts)I remember those 20-megabyte hard drives; nowadays you can get a drive that's a million times bigger than that.
All experts queried on this subject cite Moores Law somewhere in their response. If you figure a even a 20-year life cycle for the Utah facility, the estimates by the NSA and DoD are pretty realistic.
And why wouldn't their estimates be accurate? There are a few things the government is very good at, and the military/industrial complex is at the very top of that list. I certainly wouldn't bet any of my money against their expectations.
jmowreader
(50,559 posts)But where do you get that many of them? It's a question of production capacity because diverting the entire hard drive production capacity of the world to this data center for a hundred years wouldn't get them 36 billion hard drives.
reusrename
(1,716 posts)Over a twenty year life cycle, the yottabyte figure is right in the ballpark, without the need for anything close to 36 billion hard drives. Besides, there are more compact storage media currently being developed.
Downwinder
(12,869 posts)I would sort of like to go through them.
Uncle Joe
(58,365 posts)you can bet Joe McCarthy would've loved to go through them.
If there had been e-mails from 1917 until the early 1970s ironically with the possible exception of the year 1963, J Edgar Hoover would've loved to go through them.
McCarthy and Hoover's impact would've been as on steroids if they had everybody's meta data.
You can also safely wager that had there had been e-mails from at least 1969 (if not before) until 1973 Richard Nixon would have loved to check them out.
And Lord help us if another Dick Cheney or George W. Bush strand of politician comes to power.
Downwinder
(12,869 posts)the general population.
Uncle Joe
(58,365 posts)different facets depending the need or want at the time.
Information is power and as the old adage goes "power corrupts and absolute power corrupts absolutely."
I see this as just a means of turning back or curtailing the revolutionary growing power of the people via the Internet.
Motown_Johnny
(22,308 posts)madinmaryland
(64,933 posts)Ford_Prefect
(7,901 posts)Downwinder
(12,869 posts)will be like trying to read an 8" floppy.
ram2008
(1,238 posts)Skynet meets Minority Report type stuff.
intaglio
(8,170 posts)What that means is storing, at current rates of use, the metadata associated with 100 years of US worldwide communications. Just like the actions of all the commercial networks and services that move your data round. These other networks are not limited to the ones with which you have an agreement.
Welcome to the modern world - it's just that you have not been paying attention.
4A does not cover this and current laws or treaties do not cover this. If you want it to change then work to get amendments, laws and treaties changed - just do not moan about unprecedented spying when it is not unprecedented, it has not been a particular secret and is not (in the old meaning of the word) spying.
randome
(34,845 posts)You cannot stop technology. We need updated laws to deal with it but castigating an agency for using technology is pointless.
We could also use more transparency and less secrecy all around. Assuming the worst about the data facility is also pointless. We should find out what its precise use will be.
[hr][font color="blue"][center]You should never stop having childhood dreams.[/center][/font][hr]
nenagh
(1,925 posts)and Wm Binney, who worked 30 years with the NSA, has already expressed some of his reservationa about the facility.
90-percent
(6,829 posts)America has piles and piles of obsolete older laws that are still on the books. If you raise a red flag with your electronic communications that you may be "an enemy of the state" in the form of an unflattering facebook or DU post, they can go after your meta data of you entire electronic lifetime and eventually find something you did that was technically illegal under obsolete laws.
This is yet another of many totalitarian mechanisms by which our government can silence and punish anybody that gets them angry or upset or perhaps even offended.
I could probably be sent up the river for a decade on the basis of 20 years of online cynical snark, of which portion may have been directed at the powers that be.
Hell, they could probably send you to prison for bitching about the 2000 Election if they wanted to. I'm sure anybody bitching about that publicly at the time is guilty of something.
-90% Jimmy
tridim
(45,358 posts)I'll eat all the crow in Mannygoldstein's meat freezer.
Do you really believe what you just wrote? Because honestly, it's delusional.
90-percent
(6,829 posts)Is the last 12 years of building the national security infrastructure delusional? Is the militarization of our police delusional? Is our unprecedented current persecution of whistle blowers and Occupy Protesters delusional?
Our government is aggressively putting in place the infrastructure for a national security police state. Are you saying that it will never ever be used in the future if they're already nibbling around the edges in the present?
Do you have the same Constitutional Rights you enjoyed 13 years ago?
Isn't George Orwell's 1984 happening for real all around us?
America fought wars, not the least of which was the 500,000 death Civil War, to preserve defend and protect THE CONSTITUTION. As American's we should never tolerate the erosion of our Constitutional Rights. Especially in the name of fighting terror, which, mathematically is statistically insignificant compared to the myriad other ways we die all the time.
-90% Jimmy
PS - I'm 58 with no kids. I will probably die before global climate change or a totalitarian American government kills mass qty's of us. The trends for both are crystal clear. Just because NYC is not under water now doesn't mean it won't be by the end of the century. Just because thousands aren't being transported to dissident American prison campus in the present, doesn't mean it won't happen within the next 10 or 20 years.
snooper2
(30,151 posts)tridim
(45,358 posts)marions ghost
(19,841 posts)Lots of evidence.
You know it's true.
woo me with science
(32,139 posts)Dreamer Tatum
(10,926 posts)Ever occur to you that you're not on anyone's agenda? 20 years of not being in prison should have tipped you off.
Orrex
(63,215 posts)No, I'm not thilled about this sort of info storage, but it was inevitable.
woo me with science
(32,139 posts)before it even materializes.
woo me with science
(32,139 posts)What could go wrong with that?
Talk about "insider trading" on steroids.
Coyotl
(15,262 posts)Yeah, real genius at work transferring money to military contractors and creating high security jobs for military types, but no insight into how the place will function!
MineralMan
(146,317 posts)the public firestorm would go out almost as quickly. The NSA is not interested in the doings of anyone except people doing dangerous strategic and tactical stuff. The rest is discarded as useless noise.
snooper2
(30,151 posts)Simple Question for everybody
What does the NSA do with duplicate call legs?
So you think they are "recording" every call in the United States, which would include US-International calls as well-
Lets say for a moment that they DID have say a hundreds of 10G to EVERY single carrier in the United States. Because you would have to in order to record EVERY SINGLE CALL. This includes po-dunk MaPa shithole carriers like- Bismark ND BFE Telecom-
If BFE Telecom has grandma call the local drug store that is also on BFE's network, to get that call the NSA would have to be capturing EVERY packet on their network. In addition, If grandma on BFE telecom calls YOU on your Sprint smartphone, the NSA would have TWO copies of that call, because they are capturing every packet on Sprint's network as well. Let's add a layer, BFE Telecom doesn't have their own LD services (LD = long distance) so they have a wholesale contract with Level 3. So now the NSA has THREE copies of the same phone call since they are capturing every packet on Level 3's network as well. Talk about the most inefficient way to capture content of a phone call, when they could just send over a warrant. (Oh yeah, they do. On average about 10 for every million subscribers)
I swear people who are writing these articles aren't very bright. And I kind of feel sorry for the folks who buy it hook line and sinker. One thing I've learned is Ted Stevens was not alone in thinking the Internet is a series of tubes.
I was talking with somebody at the company we contract with as a trusted third party and we had quite the laugh over the BS being spread about NSA spying. It's comical, this after we spent a couple hours getting ONE target to work correctly---
Enjoy your series of tubes, (phone calls go through a couple tubes as well , it's all IP right )
You can do the TUBE DANCE!
JoePhilly
(27,787 posts)anything you just posted.
tridim
(45,358 posts)They probably also buy everything they see on TV infomercials.
marions ghost
(19,841 posts)...your stuff is so Yesterday.
And forget Grandma on the phone, where are your Fairy Tales about PRISM?
snooper2
(30,151 posts)Now the NSA is tracking you too
marions ghost
(19,841 posts)--you better get off that conference call with "Grandma" and pick up your new talking points. They've been out for awhile.
snooper2
(30,151 posts)marions ghost
(19,841 posts)to tell us some Fairy Tales about PRISM. Looks like you are avoiding that topic amirite?
I would like to be educated about PRISM, as I'm sure a lot of Americans of all types would also. Do you know any more than what Snowden & Greenwald have revealed?
Recursion
(56,582 posts)I'm convinced a Comms 101 class would do the hair on fire brigade a world of good.
cherokeeprogressive
(24,853 posts)In that case, full speed ahead!
marions ghost
(19,841 posts)what the NSA & Associates are doing needs to be severely limited.
We need some *&!&@%^%$@& NEW LAWS and regulations, that are 3rd party verifiable at all times. Oversight all over the place.
REAL oversight.
NEW laws, WITH teeth.
Until we see that, we are pawns in a very ugly game.
PowerToThePeople
(9,610 posts)We have to deal with the fact that this is possible. We have to secure our communications.
moondust
(19,991 posts)God needs more information in order to judge your worth and whether or not you present a threat to him/her. You may not fit today's definition of a terrorist but that definition may change in the future.
woo me with science
(32,139 posts)betrays an agenda.
This is the recipe for totalitarianism.
Recursion
(56,582 posts)Whoever came up with 100 years is being stupid.
ram2008
(1,238 posts)[link:http://|]
Recursion
(56,582 posts)Where is 100 years coming from?
ram2008
(1,238 posts)On two-way communications technology, such as cell phones, humankind shared 65 exabytes of information through telecommunications in 20071000 exabytes in 1 zettabyte. So in roughly 100 years, that would be 6500 exabytes or 6.5 Zettabytes. If we're viewing past data collected then it is less zettabytes, if we're including future data it will be more zettabytes. But roughly 100 years.
http://www.sciencedaily.com/releases/2011/02/110210141219.htm
Sometimeswedrown
(45 posts)Make obscene phone calls
Traffic in meth
Threaten a shy kid in a chat room
Download and steal software
OR
Set up a bombing with a known terrorist
Yea then I will worry about it
marions ghost
(19,841 posts)want to join an environmental activist group. Or a political group like Occupy. Or a group that works for voting rights? Or workers rights? Or women's rights? Or social justice?
What if you are an investigative journalist? Or a whistleblower or an investigator of any kind of corruption? What if you are in academia and you want to do a whole lot of research on the NSA and how it is pushing the loss of privacy rights into the stratosphere? An artist or writer who wants to deal with the surveillance topic?
Do you think any of those things will go on as usual? No, they will be chilled from the outset.
We have absolutely no protections.
spin
(17,493 posts)the way they are running the nation on DU.
But you hesitate because you realize you might find you are targeted for intense monitoring.
Or because the new Republican administration is running the nation into the ground, you decide to run for Congress. Oddly enough your opponent ends up with a file on everything you have done or said since you were in high school. He finds something he can use against you and you lose the election.
Now I will tell you the answer to my question. It is this. The Party seeks power entirely for its own sake. We are not interested in the good of others; we are interested solely in power, pure power. What pure power means you will understand presently. We are different from the oligarchies of the past in that we know what we are doing. All the others, even those who resembled ourselves, were cowards and hypocrites. The German Nazis and the Russian Communists came very close to us in their methods, but they never had the courage to recognize their own motives. They pretended, perhaps they even believed, that they had seized power unwillingly and for a limited time, and that just around the corner there lay a paradise where human beings would be free and equal. We are not like that. We know what no one ever seizes power with the intention of relinquishing it. Power is not a means; it is an end. One does not establish a dictatorship in order to safeguard a revolution; one makes the revolution in order to establish the dictatorship. The object of persecution is persecution. The object of torture is torture. The object of power is power. Now you begin to understand me.
― George Orwell, 1984
Pretzel_Warrior
(8,361 posts)We can nail them on obscenity charges. YES!
LWolf
(46,179 posts)I'll defer to others about the technical viability of this claim, or the likelihood of it happening.
To answer your question: when my life ends, I want to quietly fade into non-existence. I don't want my private communications, or anything else about my private life, left for people to sift through, whether they are likely to do so or not. 100 years after my death, I don't want there to be any record of my existence at all.
felix_numinous
(5,198 posts)pre- AI or sky net, the birth of The Machine--and if any of the hundreds of visionary writers' warnings about advanced electronics have any merit, about the implications of delegating power and supplying them with enough data, networking and programming to be used against us--we should be taking a long pause right about now.
Even if we are not at an advanced stage of development, just the fact that a secret agency is in charge of operating and programming them, we can guess as to what their purpose could include--->They are being used to protect the 1%, and will continue to do so unless enough good people intervene.
I cannot believe the powers that be have demonstrated enough social ethics to be handed yet MORE gadgets and endless cash.
Tools are only as good as their designers and operators.
They got enough. All they really need is a good sample to start guessing about the moon landing.