Modern society relies on computers to manage and transmit sensitive data. These computers run our banks, provide our telecommunications services (such as phone, TV, and Internet), and operate critical systems found in automobiles and power grids. The software on these systems is vulnerable to automated attacks and, if attacked successfully, can be used to cause the loss of money, property, and life. While researchers have developed automated, easy-to-use countermeasures to thwart such attacks, it is unclear whether these countermeasures work.