I'm working through Joseph Weizenbaum's classic work, Computer Power and Human Reason. Some parts of it are a bit dry, but others are quite interesting and relevant to computer science today.
I enjoyed how clearly Weizenbaum explains the universal Turing machine, as well as his low-level account of how computers store and process data. These are essential building blocks in understanding how computers work and why they are so powerful, and Weizenbaum handles them well.
One of the most surprising elements of the book is how relevant Weizenbaum's concerns about artificial intelligence remain today. He is skeptical of the notion that humans are mere input/output devices, and that sufficiently advanced machine intelligence could therefore become equivalent to humans. As he observes, we may be universal Turing machines, but we are also much more than that.
Here are some quotes that stood out to me.
On programs as expressions of problem-solving:
A computer is a merciless critic. Therefore the assertion that one understands a thing sufficiently well to be able to program it is, first of all, an assertion that one understands it in very particular terms. In any case, it can be no more than a boast that may well be falsified by experience.
The other side of the coin is the belief that one cannot program anything unless one thoroughly understands it. This misses the truth that programming is, again like any form of writing, more often than not experimental. One programs, just as one writes, not because one understands, but in order to come to understand. Programming is an act of design. To write a program is to legislate the laws for a world one first has to create in imagination.
Only very rarely does any designer, be he a an architect, a novelist, or whatever, have so coherent a picture of the world emergent in his imagination that he can compose its laws without criticism from that world itself. That is precisely what the computer may provide (108-109).
Computers are maddeningly efficient at stumbling over purely technical, i.e., linguistic, programming errors, but stumbling in a way that disguises the real locus of the trouble, e.g., just which parenthesis was misplaced...
... There is therefore a persistent cry for natural-language, e.g., English, programming systems. Programmers who hold to this belief have probably never tackled a truly difficult problem, and have therefore never felt the need for really deep criticism from the computer (109).
On computer programs as abstract games:
The computer is, of course, a physically embodied machine and, as such, cannot violate natural law. But it is not completely characterized by only its manifest interaction with the real world.
Electrons flow through it, its tapes move, and its lights blink, all in strict obedience to physical law, to be sure and the courses of its internal rivers of electrons are determined by openings and closings of gates, that is, by physical events. But the game the computer plays out is regulated by systems of ideas whose range is bounded only by the limitations of the human imagination. The physically determined bounds on the electronic and mechanical events internal to the computer do not matter for that game--any more than it matters how tightly a chess player grips his bishop or how rapidly he moves it over the board.
A computer running under control of a stored program is thus detached from the real world in the same way that every abstract game is. (112)
The computer programmer, however, is a creator of universes for which he alone is the lawgiver. So, of course, is the designer of any game. But universes of virtually unlimited complexity can be created in the form of computer programs. Moreover, and this is a crucial point, systems so formulated and elaborated act out their programmed scripts. They compliantly obey their laws and vividly exhibit their obedient behavior. No playwright, no stage director, no emperor, however powerful, has ever exercised such absolute authority to arrange a stage or a field of battle and to command such unswervingly dutiful actors or troops.
One would have to be astonished if Lord Acton's observation that power corrupts were not to apply in an environment in which omnipotence is so easily achievable. It does apply. And the corruption evoked by the computer programmer's omnipotence manifests itself in a form that is instructive in a domain far larger that the immediate environment of the computer (115).
On the Compulsive Programmer vs. the Professional Programmer:
The compulsive programmer is convinced that life is nothing but a program running on an enormous computer, and that therefore every aspect of life can ultimately be explained in programming terms. Many scientists (again there are notable exceptions) also believe that every aspect of life and nature can finally be explained in exclusively scientific terms. Indeed, as Polany correctly points out, the stability of scientific beliefs is defended by the same devices that protect magical belief systems:
Any contradiction between a particular scientific notion and the facts of experience will be explained by other scientific notions; there is a ready reserve of possible scientific hypotheses available to explain any conceivable event. . .. within science itself, the stability of theories against experience is maintained by epicyclical reserves which suppress alternative conceptions in the germ.The professional regards programming as a means toward an end, not as an end in itself. His satisfaction comes from having solved a substantive problem, not from having bent a computer to his will.
[The compulsive programmer's] main interest is, in any case, not in small programs, but in very large, very ambitious systems of programs. Usually the systems he undertakes to build, and on which he works feverishly for perhaps a month or two or three, have very grandiose but extremely imprecisely stated goals. Some examples of these ambitions are: new computer languages to facilitate man-machine communication; a general system that can be taught to play any board game; a system to make it easier for computer experts to write super-systems (this last is a favorite). It is characteristic of many such projects that the programmer can long continue in the conviction that they demand knowledge about nothing but computers, programming, etc. And that knowledge he, of course, commands in abundance (117).
Programming systems can, of course, be built without plan and without knowledge, let alone understanding, of the deep structural issues involved, just as houses, cities, systems of dams, and national economic policies can be similarly hacked together. As a System so constructed begins to get large, however, it also becomes increasingly unstable. When one of its subfunctions fails in an unanticipated way, it may be patched until the manifest trouble disappears. But since there is no general theory of the whole system, the system itself can be only a more or less chaotic aggregate of subsystems whose influence on one another's behavior is discoverable only piecemeal and by experiment. The hacker spends part of his time at the console piling new subsystems onto the structure he has already built -- he calls them "new features" -- and the rest of his time in attempts to account for the way in which substructures already in place misbehave. That is what he and the computer converse about.
The psychological situation the compulsive programmer finds himself in while so engaged is strongly determined by two apparently opposing facts: first, he knows that he can make the computer do anything he wants it to do; and second, the computer constantly displays undeniable evidence of his failures to him. It reproaches him. There is no escaping this bind. The engineer can resign himself to the truth that there are some things he doesn't know. But the programmer moves in a world entirely of his own making. The computer challenges his power, not his knowledge.
Indeed, the compulsive programmer's excitement rises to its highest, most feverish pitch when he is on the trail of a most recalcitrant error, when everything ought to work but the computer nevertheless reproaches him by misbehaving in a number of mysterious, apparently unrelated ways. It is then that the system the programmer has himself created gives every evidence of having taken on a life of its own and, certainly, of having slipped from his control. This too is the point at which the idea that the computer can be "made to do anything" becomes most relevant and most soundly based in reality (119).
But the compulsive programmer's pride and elation are very brief. His success consists of his having shown the computer who its master is. And having demonstrated that he can make it do this much, he immediately sets out to make it do even more. Thus the entire cycle begins again. He begins to "improve" his system, say, by making it run faster, or by adding "new features" to it, or by improving the ease with which data can be entered into it and gotten out of it. The act of modifying the then-existing program invariably causes some of its substructures to collapse; they constitute, after all, an amorphous collection of processes whose interactions with one another are virtually fortuitous. His apparently devoted efforts to improve and promote his own creation are really an assault on it, an assault whose only consequence can be to renew his struggle with the computer (120).
