For those of us with smartphones, it’s an unusual week when we’re not notified of a handful of updates to our apps. While some add features, many address vulnerabilities that threaten the security of our data. In the world of online data, malice seems ubiquitous.
Malicious behavior is similarly common in the natural world, where agents tend a means to enter a system and subvert its rules in their favor.
This “cheating,” as most of the world calls it, can damage, disrupt, and destroy other agents or the system itself.
“Malicious behavior tends to arise in almost every complex system that is comprised of self-optimizing agents – if they also have the ability to learn or evolve,” says SFI External Professor Stephanie Forrest. She and fellow University of New Mexico professor Melanie Moses, also an SFI external professor, have been considering this theme over the past few years, in connection with Forrest’s work on cybersecurity and software evolution and Moses’s work on robotic swarms and ants.
As part of their early December working group, “Evolution and Restraint of Malicious Behavior in Complex Systems,” they hope to explore and generalize how cheating evolves and how a system identities and handles it.
Simple but adaptable robots and cyber-physical systems are also of interest, as “they may end ways to subvert rules or produce unintended consequences if cheating lets them meet their specified goal,” Moses says.
Conversations with SFI President David Krakauer and VP for Science Jennifer Dunne sparked the idea of bringing together experts in the evolution of cheating and its restraints in natural systems together with experts knowledgeable and open-minded about malicious behavior in software and robotics.
Among the topics germane to the project are the dynamics of individual and collective behavior, conflict management in social primates, immunity and other biological defenses, stable ecologies that inhibit harmful behavior, cognitive robotics, robot ethics, swarm intelligence, and trends in data-mining for fraud and abuse.