Very large scale social data collection without anyone’s consent is now possible. It could be done for good or evil. The dataset created could be of great value in making society work, or of great danger if used by the wrong people.
Recursive Exhaustion is an algorithm for converting facts into data, to form the largest database that can be obtained from a given factbase.
If the distinction between facts and data isn’t clear, just consider data as a very nice presentation of facts, perfect for machine learning and other technologies derived from the work of data scientists. Please look at the Terminology page for similar distinctions.
Conceptually, recursive exhaustion is applied after information gathering methods have filled a factbase with most of the known facts about every person, place, corporation, institution, and other entities, including ones which no longer exist. Much harder to implement but of more value is its use during an ongoing collection of facts.
In my Simple Example page I have written about the use of genealogical data, as in the header for this website. For ease of understanding I describe it as an iterative process, which might indeed be used if the more obvious implementation is unrolled.
In an unfinished work of fiction I described this as “Repeat until your supercomputer installation runs out of disk space.” Today I might say “until the Cloud runs out of space”.
Among other techniques applicable to society are those for collecting and using personal data by tapping the resources of social media. The most notorious example of this to-date is the misuse of Facebook information by Cambridge Analytica. To me, this was almost trivial. They collected a small amount of information on a mere 87 million people.
Through Recursive Exhaustion it is possible to collect a vast amount of information about almost everybody. Indeed the more people studied, the more information can be obtained about each. The best analogy is that of a large radio-telescope installation. The more individual receivers there are and the larger the geographic area over which they spread, the greater the resolving power.
Using one meaning of the the term, recursive exhaustion works by repeatedly exhausting the space or tree of known individuals and their attributes. The secondary meaning involves growing that tree.
In computer science there is a method known as an exhaustive search, also known as a brute-force search. Sometimes it is referred to by its fundamental technique and is known as generate and test. It is one of the most powerful of the general problem-solving techniques, but is computationally expensive.
An exhaustive search is usually conducted on a tree structure, which is a discrete combinatorial object. One might somehow transform a list of people into a tree structure then perform a search to find a person meeting certain characteristics.
The problem with this is that the human population changes. People are born and die. Living people change all the time. A fixed tree structure for the human race is impossible.
Applying an exhaustive search recursively in a social context means that the attributes of one person are reevaluated regularly by considering all changes in his or her social environment. The individuals in that social environment will also have to be reevaluated, so the search for one person requires a data collection step which can propagate recursively throughout the whole population.
As applied to the whole of human society, this violates the most fundamental requirement of a recursive algorithm: it has no end condition.
Nor should it. There is no end to the changes society goes through.
The only way a query could be answered by an exhaustive search of a tree of human information would be if no changes were made to that data. This could be done in a recursive way having an end condition, but would provide poor answers and be ultimately pointless.
As a powerful tool of social technology, recursive exhaustion would be an unending process. A computer system would operate continuously, accepting new data as it became and providing information about individuals as requested.
For example, a non-governmental organization devoted to helping people could query for the people most in need of its services. On the other hand evil people could query for people susceptible to blackmail or intimidation.
Various implementations of the recursive exhaustion algorithm are discussed on another page. Details of its application to human society will be given elsewhere. An example of its use in given on a page discussing genealogy, since that is easy to explain and could actually be of some use to interested individuals.
If you post a comment and it doesn’t appear soon, please use the Contact Form to contact me. People who abuse it will be blocked. To encourage access to this information, I am using at least one ping service: