A superintelligent system could have disastrous effects even if it had a neutral goal and lacked self-awareness. “We cannot blithely assume,” Bostrom wrote, “that a superintelligence will necessarily share any of the final values stereotypically associated with wisdom and intellectual development in humans—scientific curiosity, benevolent concern

Claude Shannon, the father of information theory, redefined information so as to exclude the need for a conscious subject. All languages can be broken down into two aspects, syntax (the structure of the language, its form) and semantics (its content, or meaning).

Nobody could reach my true self—my mind—which resided elsewhere. My true self was the brain that consumed books in bed each morning with an absorption so deep I often forgot to eat. And yet this “real” self was so ephemeral. It existed in perfect isolation, without witness, and seemed to change from one day to the next.

There could no longer be a purely “objective” view of the world that took into account the whole picture. Science was always particular to a specific observer and had to acknowledge our subjective outlook as humans. We could not speak of reality without speaking of ourselves.

All perception is metaphor—as Wittgenstein put it, we never merely see, we always “see as.”

Perhaps our limited vantage as humans meant that all we could hope for were metaphors of our own making, that we would continually grasp at the shadow of absolute truths without any hope of attainment.

Likewise, a very intelligent but unconscious machine could conceivably acquire enough information about the human mind

The only way for us to survive as humans is to begin merging our bodies with these technologies, transforming ourselves into a new species—what Kurzweil calls “posthumans,” or spiritual machines.

The pointlessness of my existence would often hit me in the midst of some ordinary task—buying groceries, boarding a train—and I would become paralyzed by confusion and indecision. Any discrete action, detached from a larger context, comes to seem absurd, just as a word considered on its own, removed from the flow of language, quickly becomes