The “singular value decomposition” is a powerful concept that underlies both factor analysis in psychology as well as basic image compression. It is a means of finding ‘the important parts’. It does this by taking the variables in which the information is framed and converting them to variables that are more native to the information itself. It lets the data speak its own language.
The best way that I’ve found to think about this is by imagining the data as a many-dimensional ellipsoid, which is to say an elongated sphere. The idea is that the ‘important part’ is the longest axis of the ellipsoid and the least important part is the smallest. To smush together the long axis would take more smushing than to do the same to the smaller axis, and the point of good compression is to get rid of axes with as little smushing as possible.
For example, say I have data about caring for plants. Maybe I have a hundred input variables for each plant (water, food, sunlight, CO2, etc). It may turn out that, so to speak, what the plant really cares about is the sum of sunlight and CO2 (with the proper conversions), and with this number we can create almost any kind of plant we want. This sum is the longest axis, because it creates the largest variation. It is one less dimension than thinking about sun and CO2 separately, and three less than thinking about all four variables. It may be that the water and food input create almost not effect on the plant, these then are like the really short axes of the ellipsoid, and they are eliminated by the compression. The actual process is at least a little more complicated than this because it is operating on the input and the output simultaneously.
Spherical information is therefore the most difficult to compress. In this context, sphere-ness of the information refers to the process that it measures making equal use of all the information available to it. The more information that it ignores, the more predictable and less entropic it becomes. The process becomes harder to “fool” because it is looking in a lot of different places.
The Bene Gesserit from Frank Herbert’s Dune, for example, are people who have trained to use every bit of information in their environment. Their processing matrices are almost perfectly spherical and they are able to quickly see and exploit the warpings and elongations in those of others. The Voice is an example of this where they are applying energy along the principle axes of another person and thus exploiting their lack of entropy/sphericallity. Of course, the temptation to elongate is ever present, but the fanatic has the least entropy and is therefore very vulnerable/predictable. It is the maintenance of the entropy/chaos that will allow triumph in the eventual war with the machines.
“One must have chaos in one to give birth to a dancing star” -Nietzsche
