Sublime

Skinner’s goal was to make his pigeons peck the button as many times as possible. From his experiments, he made three discoveries. First, the pigeons pecked most when doing so yielded immediate, rather than delayed, rewards. Second, the pigeons pecked most when it rewarded them randomly, rather than every time. Skinner’s third discovery occurred wh

During the first three trials, the cat escaped in an average of 1.5 minutes. During the last three trials, it escaped in an average of 6.3 seconds. With practice, each cat made fewer errors and their actions became quicker and more automatic. Rather than repeat the same mistakes, the cat began to cut straight to the solution. From his studies, Thor

Think of your default patterns of thinking, feeling, and acting as algorithms you’ve been programmed to run unconsciously in response to inputs from other people or the environment. We don’t think about moving our knee when the doctor hits it with a reflex hammer. It just moves. The same thing happens with your thoughts and actions. We receive some

The key, he said, was that he had “learned the right human psychology.” That psychology was grounded in two basic rules: First, find a simple and obvious cue. Second, clearly define the rewards.

Other social media platforms followed, leveraging Skinner’s three laws to maximize button-pecking. They offered immediate reinforcement in the form of instant responses, conditioned reinforcement in the form of “likes” and “followers”, and unpredictable reinforcement that varied with each post and each refresh of the page. These features turned soc

The work of Bush, Mosteller, Resorla, Wagner and Sutton all turned stacks of data from conditioning experiments into strings of symbols that could describe the algorithm needed to do the learning part of reinforcement learning.

Skinner famously found that the fiercest and most repetitive or persistent behavior tended to come from variable ratio schedules

As observers, we can both see the behavior we are using, and alternative behaviors; we can then choose

They knew from Andrew Ng and Stuart Russell’s work that one of the cardinal rules is to reward states, not actions.