What’s a Gigabyte Good For?

This is cross-posted from my new site intended for notes about my unfinished novel, along with some excerpts.  This is the first excerpt, which explains Recursive Exhaustion in a single short paragraph, then says just how much could be done with it.

“OK, enough mystery. Tell me”, he insisted.

She sighed.

“You may regret asking. Here’s the short version. In the mid-1970s a small group of grad students at NYU played about with new kinds of technology for collecting and using information about various entities. It wasn’t very sophisticated at the time, more of a mathematical curiosity. But they all thought about using it for collected a lot of information about a lot of people.”

“I assume they have, which explains the spy vs. spy stuff.”

“Well they needed a good implementation first, which took a while. There was another problem, now solved. Computers and mass storage devices at the time were not powerful enough for what they wanted to do, but there has been a drastic expansion capability.”

“Is this technology something I would know about?”

“Probably not. Does the term ‘recursive exhaustion’ sound familiar?”

“No. What is it?”

“In a nutshell, start with some data about you. Combine that with similar data about important people, places and institutions in your life. Do that for everyone, every place, every institution, forming an enlarged data record for each. Boil off useless and conflicting data. Repeat until your supercomputer installation runs out of disk space.”

“Oh. Good idea, remind me to try it sometime.”

“That can be arranged.”

“What did they want to do with all that data?”

“Different things. Some people wanted to exploit the technology for personal gain, while others were idealists and thought it could be used to improve society.”

“What happened?”

“The people wanting to exploit the technology for personal gain had no qualms about, acquiring data illegally.”

“Like what?”

“Raw census data with names and addresses is good, income tax records are better. Everything they could get their hands on.”

“Credit card numbers?”

“Useful for stealing money, but pretty crude. Far too easy to detect. Money is one motive, but they wanted larger amounts obtained in undetectable ways. Manipulating the stock market for example. We have evidence that they may have obtained a trillion dollars that way. A million million. Other motives include political power and sexual domination.”

“Lovely. How successful have they been?”

“Very. They started using blackmail and intimidation to make people give them masses of illicit data. From there it was just a short step to using the same means to get whatever they wanted. And whomever they wanted to use, for any reason.

“What about the other group, the idealists?”

“They decided early on never to use illegally obtained information. They wanted everything completely aboveboard. That attracted some exceptional people who have helped them flourish in unanticipated ways. But the new people were even more idealistic and demanded even more ethical behavior.”

“How has this worked out?”

“Starting with a limited amount of information, the bad guys now have an enormous amount on about everybody. They could tell you the name of the first girl you took to bed, and how well you performed.”

“You are making this up. I simply do not believe you.”

“My rule of thumb is Clarke’s First Law. Do you know it?”

“You mean what the science fiction author Arthur C. Clarke wrote? ‘Any sufficiently advanced technology is indistinguishable from magic’?”

“That’s his third law, you idiot. His first law is ‘When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.'”

“Hey, I’m not elderly.”

“Not so distinguished either, but you ask the right questions.”

“So what you just told me about is possible?”

“My own attitude is this: don’t try to judge whether something is possible until you’ve tried to figure out how you would do it. Suppose you absolutely had to find out when I first seduced a boy, how would you go about doing it?”

“I have no idea.”

“Poor man. Work on it a while, wait for an epiphany, or whatever it takes. If you disappointment me by failing to figure it out, I’ll tell you. I hope it won’t come to that. If we might hook up sometime I need to maintain some respect for your intelligence.”

“You are a horrible young woman.”

“Oh no, I’m sure you’d find me thoroughly satisfying, given a chance. Play along and you might get one.”

“OK. How much information do the good guys have?”

“Not nearly as much. Probably only a million terabytes or so altogether. More data on more people than either of us can fully grasp, but all legally obtained.”

“Oh, that’s reassuring. You had me worried for a minute there.”

“Well, using recursive exhaustion and data mining the public stuff can be turned into things you’d rather keep private. It’s ethical to use it because access is strictly limited and every byte originated in public data. There may be a lot of information about your sex life in the database, but no way anyone can see or download it. Nobody can query for anything about you personally. It can only help you find the best people to associate with and the most suitable jobs you can get.”

“And whether I’m in danger or not.”

“That’s true, but the information is even more restricted.”

“What do you know about me?”

“I was looking for someone compatible with me in some useful way, perhaps a lover, perhaps a friend, maybe a co-worker. An ID code for you came up, along with a red flag indicating danger for one or both of us. I was able to see a summary of similar situations in the past. In most of those, both people had been in danger, but the only way of bringing the new one in safely was to meet in person, like this.”

“So you think I am compatible with you in some useful way. I can imagine at least one.”

“I bet.”

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply