Berger code

Transcription

>>Presenter: I first met Cory Doctorow in 1986 when he was working for Metallica and trying to anticipate the coming Napster wars. People don't know this about him, but he is a fervent defender of the RIAA and the MPAA. This has all been just a big lie, he says, [laughter] to make people feel that he's one of us. Recently, actually, I ran into him in Washington, DC, and he was telling me, "You guys do the legal minimum of compliance with the DMCA. We think that you're just pirates." I said, "You've been working for the RIAA now for 16 years. I don't see how you can possibly say that with any sort of honesty." Then I realized who I was talking to, of course. [laughs] [laughter] Realistically, I met Cory when he was doing sugar water. Right? For the first time. >>Cory Doctorow: [inaudible] >>Presenter: Yeah. And then Zelig-like, I noticed as he rose as an author. I actually don't want to spend much time introducing him. So I won't. Everyone, please welcome Cory Doctorow. [applause] >>Doctorow: Hi folks. I give-- I write science fiction novels and stories. You've got some of my short story collections there in front of you. They all relate, in one way or another, they kind of circle these issues. I feel like science fiction stories put the sinew and the marrow into the argument. Before George Orwell came along, if you wanted to talk about surveillance, you could say, "I kinda feel like it might change my behavior if I were being watched all the time in some abstract way." And someone else might say, "Yeah, but if we knew everything about you, we could provide services to you and we could know when bad things were going to happen" and so on. And now we have this great word we can use to describe what that means. You can say it's Orwellian. So there's now a lot of muscle on the bone when you talk about this stuff. That's what I do in the fiction. But I don't want to stand here and read stories to you, although I have a podcast where you can hear me read stories. What I'm going to do today is the argument that the books that you're holding in your hand are the blood and sinew for, and take it from there. The talk runs 35 minutes. And then there's time for Q&A. The one thing I want to say as a caveat to this: I've given this talk twice now. I gave it at the Long Now Foundation and I gave it at DEFCON, both in the last week or ten days. Both times, there was a little bit of feedback. There's a hypothetical, technical solution I propose, and I'll tell you when I get there. I want to clarify that it is purely hypothetical by way of example, and not a thing that I think we should do. With that said, [laughter] I'm going to get to it. I gave this talk in late 2011-- Ha. Ah! There we go. I gave this talk in late 2011 at the 28C3 in Berlin called "The Coming War on General Purpose Computation." In a nutshell, the hypothesis of that talk was computers and the internet are everywhere, the world is increasingly made of computers and the internet. We used to have these separate categories of devices like washing machines, VCRs, phones, and cars, and now we just have computers in different boxes. Cars are computers we put our bodies into. 747s are badly secured Solaris boxes connected SCADA controllers. [laughter] Hearing aids, pacemakers, other prostheses: computers we put in our body. That means that from now on, all of our socio-political problems in the future are going to have a computer in the middle of them. That will beget a regulator who says, "Can't you just make me a computer that solves the problem? Can't you make me a self-driving car that can't be programmed to drag race? Can't you make me a bioscale 3D printer that doesn't print out an organism that puts the human race at risk or blows Monsanto's quarterly profits?" That is, "Can you make me a general purpose computer that runs all the programs except for the one that pisses me off?" [laughter] Now, we don't know how to make that computer. We don't have a theoretical model for Turing Complete minus one. Our closest approximation to a computer that runs every program except for the one that abets a criminal or evinces a social problem is a computer with spyware on it when it comes out of the box. That is, a computer that watches everything that you do all the time, so that when the moment comes, it can say, "I can't let you do that, Dave." A computer that runs secret programs that the user isn't supposed to even know about. If the user finds out about it, the user can't terminate these processes, even if the user really thinks that they run contrary to their interests, and even if the computer that they're running on belongs to the user. In other words, digital rights management. Now, digital rights management's a bad idea for solving social problems for at least two significant reasons. The first one is that it doesn't actually solve the problem. Breaking DRM isn't hard for bad guys. As the copyright wars have shown us, digital rights management is a solution that ends within 24 hours. As soon as a bored Norwegian teenager encounters the DRM, it goes away. DRM only works if the "I can't let you do that, Dave" program remains a secret. Once the most sophisticated attacker in the world finds out that secret and puts it on the internet, everybody else on the internet has the secret, too. Now, the second reason is that DRM not only has weak security, but it weakens security. In order to be secure, you need to be certain about what software is running on your computer. You can't secure the software on your computer if you don't know what software is running on your computer. When you design the "I can't let you do that, Dave" facility into a computer, you create this enormous security vulnerability. You now have a program running that users aren't even supposed to know about. If they know about it, they can't find details of or terminate or override. When some bad guy hijacks this, they can do things to your computer that, by design, your computer doesn't show you. You probably remember, Sony BMG put root kits on 51-- no, 6 million CDs, 51 audio CD titles, and distributed them to their customers. They stealthily installed malware. The root kit made any process or file that was prepended with dollar sign SYS, invisible to the file manager and process manager. Immediately, malware writers started prepending dollar sign SYS to their program files and their processes because if they ever found themselves on a computer whose immune system had been blown by the Sony root kit, that immune system would no longer even be able to see their process. Now, once governments solve problems with DRM, there's this perverse incentive to make it illegal to tell people things that might override the DRM, things like "This is how the DRM works" or "Here's a flaw in the DRM that might allow an attacker to secretly activate the microphone or turn on the camera or grab your keystrokes." Now after I gave this talk at 28C3, I got a lot of feedback from various civil libertarians and other people, including some very distinguished computer scientists. I got a very thought provoking email from Vint Cerf after I wrote this, which really made my day. It led me to the conclusion that within the fields of civil liberties and technology and policy, there's a kind of good guy consensus that if you own your computer, you should be in charge of what's running on it, at least as between you and corporations, or you and the government. That mandating what software you may or may not run on your computer is just not a good idea if it belongs to you. Now, most computers-- Let's examine, for a minute, what it would mean, as an owner, to be able to absolutely control the software that was running on your computer in an adversarial relationship against, not an advanced persistent threat, but at least against script kiddies or griefers or just your garden variety deputy dog cop who wants to screw with your computer. Most computers today are fitted with these: TPMs, trusted platform modules, a secure code processor mounted on the motherboard. The specification for TPM is published. There's an industry body that certifies that devices that advertise a TPM actually have a real TPM in them and not a fake TPM. To the extent that that spec is good, and to the extent that these people are diligent in doing their jobs and sue people who list a device as having a TPM when it doesn't, it's possible to be reasonably certain that if you think you have a TPM, you do have a TPM, and that it faithfully implements the spec. TPM is secure. One of the ways in which it's secure is that it has some secrets. But it's also secure in that it's designed to be tamper evident. If you try to extract the keys from a TPM, it's supposed to be really obvious that something has been done to your computer. Someone takes your real TPM out and puts a fake TPM that they 3D printed or cooked up in their hack lab or made down in Quantico and sticks it in your computer, it's supposed to be really obvious that it's happened. There's a TPM threat model that crooks or governments or police forces or some other adversary try to compromise your computer, and TPM tamper evidence lets you know when that's happened. But there's another TPM threat model. It's that a piece of malicious software infects your computer. Now, all the censors—. When that happens, all the censors that are attached to your computer, --the mic, the camera, the accelerometer, the fingerprint reader, the GPS, and so on--, can be switched on without your knowledge, and the data can be cached on the or can be sent to a bad guy or both. Not only that, of course, all of the data on your computer-- your sensitive files, your stored passwords, your web history-- can also be harvested and sent to a bad guy, or harvested and cached for a later retrieval, as can all your keystrokes. All the peripherals that are attached to your computer can either be subtly altered, turned off, or turned on to do bad things. Today, those peripherals might be your printer, your scanner, your SCADA controller, your MRI machine, your car, your avionic, your 3D printer. You can understand why that would be a bit freaky, but of course, in the future, those peripherals might also include your optic nerve, your cochlea, and the stumps of your legs. When your computer boots up, the TPM can ask your bootloader for a signed hash of itself and verify that the signature of the hash comes from a trusted party, someone you trust. Once you trust the bootloader to faithfully perform its duties, you can ask it to check the signatures on the operating system, which, once verified, can check the signatures on the programs that run on it. And so on and so on up the stack, ensuring that you know which programs are running on your computer, and that any programs running in secret have gotten there by leveraging a defect in the bootloader or operating system or the other components, and not because this computer was designed to actually hide things from you. Now, this story always reminds me of Descartes: he starts off by saying that he can't tell what's true and what's not true, because he doesn't know if he can trust his senses, he doesn't know if he can trust his reason. He does some mental gymnastics, which I won't get into here, although that's generally the thing people find interesting about him, but what's interesting to me is that once he establishes this tiny little nub of certainty, a kind of mental gymnastic exercise that says, "Well, I can trust my reason, I can trust my senses." Then he is able to erect this stable edifice of a worldview on it. He knows one thing to be true, and everything else can be hung off of that one thing. He can build it up. Now, a TPM is like that. It's a nub of stable certainty: if it's there, it can reliably inform you about your bootloader, and thus, your operating system, and thus, the processes running on your computer. Now, you may find it weird to hear someone like me talking warmly about TPMs. After all, these are the technologies that make it possible to lock phones, tablets, consoles, and even some PCs so that they can't run software of the owner's choosing. Jailbreaking usually means finding a way to subvert a TPM. Why on earth would I want a TPM in my computer? As with everything interesting in tech and policy, the devil is in the details. Imagine for a moment that there's two different ways of implementing a TPM. There may be more, but imagine these two. The first one we'll call lockdown. In the lockdown world, your TPM comes with a set of signing keys it trusts, and unless your bootloader appears in that list-- is signed by one of those signing keys, it won't run it. It won't boot, the operating system won't run. You're just stuck there in whatever it is that people who installed the bootloader on your computer want you to run. You can't change that. There's another mode that I'll call certainty. In the certainty mode, you tell your TPM which signing keys you trust. The first time you turn your computer on, you initialize it with some authentication token-- whoops-- like a key or a password or some other thing. A biometric that it knows so it knows who it belongs to. Then you, the owner, are the only person who gets to say what it trusts. You can say, "I don't trust this person's operating system" or "I do trust that person's operating system." "Only run operating systems that are signed by Cononicle, EFF, ACLU and Wikileaks. [laughter] Approximately speaking, these two modes correspond to, of course, iOS and Android. iOS only lets you run the code that's been approved by Apple. Android lets you tick a box and say, "I'm a grown up. Let me choose who I trust”. Critically, Android lacks an important facility: it lacks the facility to verify that what you think you're running is what you are running. It's freedom without certainty. Now, freedom without certainty is a big deal in a world where the computers we're discussing can see you and hear you, where we put them in your pocket and take them into the toilet, where they sit by your bedside, where they fly airplanes, where we put our bodies into them, and they drive our cars around, which is why I like the idea of a TPM, provided it's implemented in the certainty mode and not in the lockdown mode. Now, if that's not clear, think of it this way: there's the war on general-purpose computation, and that's what happens when control freaks in governments or companies decide that they should have the final say in what you do on your computer. There's also-- And then there's this: there's the counter position, which is that defenders against those people are also control freaks, but they're control freaks like me. We want to be people with the ultimate destiny over what we install on our computers. Both sides want control, they just differ in where the nexus of control should be. Control requires knowledge. If you want to be sure that songs that are moved onto an iPod, stay on the iPod, and don't come off of the iPod, the iPod needs to know that the instructions that it's getting are coming from an Apple-approved version of iTunes, and not one pretending to be iTunes. Otherwise, you don't get the roach motel. If you want to be sure that my PVR won't record a watch-once, video-on-demand program, or if it does record a program, that it won't output it to anything except something that will honor whatever business rules came along with it. You have to be sure that you know what programs I'm running and what they do. But if I want to be sure that you aren't watching me through my webcam, I need to know what firmware is running, and I need to know that the little green light always comes on when my webcam switches itself on. If I want to be sure that you aren't capturing my passwords through my keyboard using a software keylogger, I need to know that the OS isn't lying when it says there aren't any keyloggers resident in the system. Whether you want to be free or whether you want to enslave, you need control and you need knowledge. That's the coming war on general purpose computation. Now I want to investigate what happens if we win it. That's the civil war over general purpose computation. Let's stipulate that we have a victory for the "freedom side." It means that we have computers where owners always know what was running on them, because the computers would faithfully report the hash and the associated signatures for any bootloaders they find, and control over what was running on computer's ghost to you, because the computers would allow their owners to specify who was allowed to sign their bootloaders, operating systems, and so on. There are two arguments that we can make in favor of this victory, why this victory would be a good one. The first one is a human rights argument. If your world is made of computers, then designing computers to override their owner's decisions has significant human rights implications. Today, there are people who worry that the Iranian government might demand import controls, so that all the computers that come in have UFE-style bootlocker that only boots operating systems that have lawful interception back doors built in. You can move the spying right to the edge, to the user of the computer, the owner of the computer. But tomorrow, it may be that I live in the UK, and it may be that our Home Secretary says, "If the NHS gives you a cochlear implant, it has to intercept and report all the extremist speech it hears." The human rights stuff is easy to understand. The second argument comes from property rights. The doctrine of first sale is a very important piece of law. It says once you buy something, you own it. You should have the freedom to do anything you want to it, even if it gores the ox of the person who sold it to you. DRM opponents like me, we love the slogan, "You bought it, you own it." Property rights are an incredibly powerful argument to have on your side, anywhere. But they're especially powerful to have on your side in a nerd fight, because you can't swing a cat in Silicon Valley without hitting someone who thinks that property rights are an important way of solving most social problems. But it's not just nerd fights. Copyfighters get really pissed off about the term "intellectual property," because property is also a really good way to win arguments in policy circles. Before the term "intellectual property" came into prominence, we had other terms like "creators' monopoly". It's very hard to go to a regulator or a lawmaker and say, "My monopoly isn't large enough", but going and saying, "My property rights are being not respected enough or need to be expanded so that I can make sure that they're policed adequately," that's a very powerful argument to have on your hand. That's where the civil war part comes in. Human rights and property rights both demand that computers not be designed for remote control by corporations or governments. Owners be allowed to specify their OS and the programs running on them to freely choose that nub of certainty in the void that allows them to build their whole stable edifice of certainty on. Now, remember that security is relative. You are secured from attacks on your ability to freely use your music if you can control your computing environment. But, if you can control your computing environment, the Recording Industry Association of America is now vulnerable to attacks on their ability to rent you music on a single-use basis. We have this notion of streaming, this consensus hallucination that there's a difference between a stream and a download, as though there's some means of transmitting a stream of bits to someone's computer without actually having them download that stream of bits, like the internet is made of mirrors and speaking tubes. [laughter] We say "Stream", we mean "I think that your receiving software doesn't have a 'save as' button." Now, if you get to choose the nub from which the scaffold dangles, you get control and power to secure yourself against people who attack your interests. If the Recording Industry Association of America, or the government, or Monsanto get to choose the nub, then they get control and the power to secure themselves against you. So we all agree that at the very least, owners should control what runs on their computers, or I'll ask you to stipulate that. Now, what about users of computers? Users of computers don't always have the same interests as the owners of computers. Increasingly, we will be users of computers that we don't own. Where you come down on the conflict between owners and users of computers, I think, is going to end up being one of the most important both technological and moral questions of the coming decades. There's no easy answer I have, no bright line, for when users or owners should trump one another when it comes to computers. Let's start with a position I'll call "property maximalism": "If I own my computer, I should have the absolute right to dictate terms of use to anyone who wants to use it. If you don't like it, find someone else's computer to use. This one's mine. I set the rules." How would that work in practice? Well, you got some combination of an initialization routine where you set the root of trust, tamper evidence, law, and physical control. For example, you turn on your computer for the first time, and you initialize a good secret password, possibly signed by your private key, and without that key, no one is allowed to change the list of trusted parties who are allowed to sign your bootloader. We can make it against the law to subvert this for the purpose of taking control away from the owner. That makes writing malware that hijacks your computer extra special, super duper illegal, but it also makes stealthy DRM installation even more illegal. We can design the TPM so that if you remove it, or tamper with it, it's really obvious. You give it a fragile housing, so that when it's changed out, you can tell at a glance that it's happened. Then, if you still trust physical locks, you can put it under lock and key, too. Now, I can see a lot of benefits to this, but there are unquestionably some downsides to giving owners absolute control over their computers. One wedge issue is probably going to be a self-driving car. There's a lot of these around already. They come out of places like this and other places. It's easy to understand, on the one hand, why self-driving cars would be insanely great. We are terrible drivers. Cars totally kill the shit out of us. [laughter] They are the number 1 cause of death in America for people aged 5-34. I saw my friend, Katherine [indistinct], last night. She pointed out that it's also the number 1 way for humans to kill other humans. If you kill another person in your life, you're almost certainly going to do it with a car. I've been hit by a car. I've also cracked up a car. I'm willing to stipulate that humans have no business driving. It's also easy to understand how we might be nervous about the prospect of people homebrewing their own self-driving car firmware. On the one hand, we do want the sourcecode for these cars to be open, public, and subject to scrutiny so that defects can be discovered, so that hidden features that may act against their owners' or users' interests can be quickly found out. You'd want to know it if there's a kill switch built in. You'd want to know it if your car secretly drives you past more McDonalds when the kids are in the back seat. It's going to be plausible, I think, to say, "Cars are safer if they have a locked bootloader and if that bootloader only runs a firmware that's been signed by the Department of Motor Vehicles or by the FTC”. But now we're back to you, whether you get to decide whether your computer is running the software that you want it to run. Now, there are two problems with this solution, the solution of giving the state a veto over your self-driving car. The first one is that it won't work. As the copyright wars have shown us, firmware locks aren't effective against dedicated attackers. People who want to sow mayhem with custom firmware will be able to. We need a security model that doesn't believe that all the other cars on the road are going to be well-behaved. If that's our security model, then we are all dead meat. Self-driving cars must be conservative in their approach to their own conduct, and liberal in conduct they expect from others, a venerable principle familiar to people who work in computers, and also, the advice that you got on your first day of driver's ed. And it remains good advice today. Now, the second problem with this is that it invites some pretty sticky parallels. Do you remember the information superhighway? Now, if we can justify securing physical roads by demanding that the state or a state-like entity gets to certify the firmware on our cars, how would we articulate a policy explaining why the devices on our equally vital virtual roads, our information superhighways, shouldn't also be locked with comparable firmware locks for PCs, phones, tablets, and other devices? After all, we have a general-purpose network now. That means that MRIs, space-ships, and air-traffic control systems share the information superhighway with Game Boys, Arduino-linked fart machines, and the dodgy voyeur cams sold by spammers from the Pearl River Delta. In addition to that, you're going to have more wedge issues. You'll have things like avionics and power-station automation. These are a lot trickier. If the FAA mandates a certain firmware for a 747, it's probably going to want those 747s designed so that the FAA and the FAA alone gets to choose what runs on it. Just as the Nuclear Regulatory Commission is going to want the final say on the firmware for a reactor pile. This may be a problem for the same reason that a ban on modifying firmware in self-driving cars is. Once you start saying it's the place of government to sign and certify firmware on computers that they don't own, it invites people to find other computers that they should send the firmware for. But on the other hand, cars and nukes exist in a completely different regulatory framework to most of the other computers we use. Or rather, planes and nukes. Remember, a 747 is just a Solaris box. A nuke is just a specialized computer as well, with a particularly exotic housing. It may be that since these things already exist in this regulatory regime where they have no-notice inspection and so on, that adding signed firmware locks is not going to be something that invites comparisons to all the other computers in the world. But there's a bigger problem with owner control. What about people who use computers, but don't own them? This is not a group of people that the IT industry as a whole has a lot of sympathy for. We spent an enormous amount of energy as a group, devoting ourselves to stopping non-users-- or non-owners from harming owners. Users can do things like inadvertently break the computers they're using, they download menu-bars, they type random shit they find on the Internet into terminals, they plug malware-infected USB sticks into their computers, they disable the firewalls, they install plugins or add repositories or add certificates to their machine's root of trust, they punch holes in the network perimeter by accident, and they accidentally cross-connect networks that are absolutely, positively not supposed to be cross-connected. We also try to stop users from doing deliberately bad things, like installing keyloggers and spyware to attack future users, misappropriating secrets, snooping on network traffic, deliberately breaking their machines, deliberately punching holes in the network perimeter, deliberately disabling their firewalls, deliberately interconnecting networks that are supposed to remain secret-- separate, rather. There's a kind of symmetry here. DRM and its cousins are deployed by people who believe that you can't and shouldn't be trusted to run the computer you that you want on your own computer. IT systems are deployed by computer owners who believe that computer users can't and shouldn't be trusted to set policy on the computers that they use. Now, as a former systems administrator and a former CIO, I'm not going to pretend that users aren't a terrible challenge. But I think that there are good reasons to treat users as having rights to set policy on the computers that they don't own. Let's start with the business case, because I think that's the easy one to make. When we demand freedom for owners, we do so for lots of reasons, but one of them is the possibility that programmers won't have anticipated all the contingencies that their code might run up against. There may be a day where the code says no and the owner needs to say yes. Owners sometimes possess local situational awareness that can't be captured in nested "if-then" statements, no matter how deeply you nest them. This is where communism and libertarianism both converge. This guy, Hayek, thought that expertise was very diffuse, and that you were more likely to find the situational awareness necessary for good decision making very close to the decision itself. Devolution gave you better results than centralization. And then there was this guy, Marx, who believed in the legitimacy of workers' claims over their working environment, saying that the contribution of labor was just as important as the contribution of capital, and demanding that workers be treated as the rightful "owners" of their workplace, with the power to set policy. For totally opposite reasons, they both believed that the people at the coal-face should have the first cut at running the operation. The death of mainframes was attended by an awful lot of Sturm und Drang and hand-wringing and concern over users and what they were going to do to the enterprise. In those days, users were even more constrained than they are today. They could only see the screens the mainframe let them see, and only undertake the operations the mainframe was programmed to let them undertake. When the PC and Visicalc and Lotus 1-2-3 appeared, employees risked being fired by bringing these machines into their offices, or bringing home office data to use with these machines. They did this because they had a computing need that couldn't be met within the constraints set by their employer and its IT department, and because they didn't think that the legitimacy of their request would be recognized. The standard response to a request from an employee to do something that the IT department doesn't like is one or more of: "A regulatory compliance prohibits you doing the thing that you think will help you do your job better" or "If you do your job that way, we won't know if you're doing it right" or "You only think you want to do your job that way" or "It's impossible to make a computer that works the way that you think it does" or "Corporate policy prohibits you doing it." Now, these may be true, although sometimes they aren't. And even when they are, they're the kind of "soft truths" that we pay bright young things millions in VC money to try to falsify, while if you're a middle-aged admin assistant, you merely get written up by HR for doing the same thing. The personal computer arrived in the enterprise through the back door, over the objections of the IT department, without the knowledge of management, at the risk of censure and termination. It made the companies that fought it trillions. The reason that giving workers more powerful, more flexible tools was good for firms is that people are generally smart, and they generally want to do their jobs, and because they know stuff that their bosses don't know. As an owner, you don't want the devices you buy locked, because you might want to do something the designer didn't anticipate. And employees don't want the devices that they use all day locked, because they might want to do something that their bosses didn't anticipate. This is the soul of Hayekism: that we're smarter at the edge than we are in the middle. The business world pays a lot of lip service to Hayek's 1940s ideas about free markets. But when it comes to freedom within the companies they run, they're stuck a good 50 years earlier, mired in the ideology of Frederick Winslow Taylor and his notions of "scientific management": The idea that workers are just particularly unreliable kinds of machines whose movements and actions should be scripted and constrained by all-knowing management consultants, who would work with the equally wise company bosses to find the one true way to do their jobs. In other words, the exact same ideology that let Toyota cream all three of Detroit's big automakers during the 1980s. Letting enterprise users do the stuff that they think will allow them to make more money for their employers often results in making more money for their employers. For the record, scientific management is about as scientific as trepanation and Myers-Briggs tests. [laughter] The business case for user rights is a good one, but I really wanted to just get it out of the way so we could dig into the real meat of the argument: the human rights case. This may seem a little weird on its face, but bear with me. This is a guy named Hugh Herr, and I saw him give a talk earlier this year. He's the Director of the Biomechatronics lab at The MIT Media Lab. You may have seen him do a TED talk. There's a bunch of them on YouTube. It's electrifying to see him give these talks. You should go and watch one after this. He starts out with a bunch of slides of cool prostheses his lab has cooked up. There's legs and feet, and hands and arms, and even this awesome thing that if you have untreatable clinical depression, they stick your head in a magnet, and the magnet suppresses activity in the parts of your brains that are overreacting, and people with untreatable clinical depression become treatable. It changes their lives and brings them from the brink of suicide back into a happy place. Then he shows this slide of him, and he's climbing up a mountain. You can see he's clinging to the mountain like a gecko. He's super buff. He clearly knows what he's about. And he doesn't have any legs, he just has these awesome mountain climbing prostheses. Now, he's been standing at a podium like this. In fact, he does it wearing a little lav mic or an ear mic, and he walks up and down while he's giving the talk. Then he stops and he says, "Oh yeah, didn't I mention? I'm robot from the knee down. I lost my legs to frostbite. These are my legs." Then he does this cool thing. He runs up and down the stage, jumping up and down like a mountain goat. It's the coolest thing you've ever seen. When I saw him give this talk, the first person who asked a question stuck their hand up and said, "So what do those cost?" He named a price that would buy you a brownstone in Manhattan or a nice terraced Victorian in Zone One. A pretty penny. The second question that was asked was, "Whose going to be able to afford these?" And he said, "Well, of course, everybody. If it's a choice between owning legs and owning a house, you'll take the 40 year mortgage on your legs." Which is by way of asking you to consider the possibility that there are going to be people, potentially a lot of people, potentially you someday-- remember, we are only temporarily able-bodied-- who are "users" of computers that they don't own, where those computers are going to be parts of their bodies. I think that most of the tech world should be able to understand why you, as the owner of your cochlear implant, should be legally allowed to choose the firmware that runs on it. After all, when you own a device that is surgically implanted in your skull, it makes a lot of sense that you have the freedom to change software vendors. Maybe the company that made your implant had the best algorithm for signal processing at the time that they were stuck in your head, but what if a competitor patents a superior algorithm next year? Should you be doomed to inferior hearing for the rest of your life or the 20 year span of the patent, whichever comes first? This is a problem that can't be overcome merely by escrowing the code of important embedded systems. That might help you if the company goes bust. It also can't be helped by code publication, the thing you would want anyway for your cochlear implant, just to make sure that it was good code. This is a problem that you can only overcome by having the unambiguous right to change the software, even if the company that made your implant requires you not to. So that helps owners. But what about users? Consider the following scenario: you are a minor child and you have deeply religious parents who pay for your cochlear implants. They ask for the software that makes it impossible for you to hear blasphemy. You are broke, and a commercial payday loan company wants to sell you ad-supported implants that listen in on your conversations and insert contextual ads that trigger discussions about the brands you love. Or your government is willing to install cochlear implants, but they want to archive everything you hear and review it without your knowledge or consent. It sounds far-fetched, but remember, the Canadian border agency, just a few months ago, had to be slapped down from its plan to put hidden microphones through the entirety of all of the country's airports, so they could listen in on and record all the conversations taking place in every airport in real time and later. Will the Iranian government, will the Chinese government, will other repressive governments take advantage of this if they get the chance? Speaking of Iran and China, there are plenty of human rights activists who believe that boot-locking will be the start of a human rights disaster. It's no secret that there are high-tech companies who have been happy to build "lawful intercept" back-doors into their equipment to allow for warrantless, secret access to their communications. These backdoors are now standard, so even if your country doesn't want the capability, it's still there. In Greece, for example, there is no lawful interception requirement, but of course, all the telecoms equipment they buy is made for jurisdictions in which there is. They just don't turn on lawful intercept. During the 2004/5 Olympics bid, someone, we don't know who, broke into the Greek telecom switches, turned on the lawful intercept capability, listened in on the conversations at the highest levels in government, turned it off again, and walked away. It's only because they didn't erase the logs that we know about it. Surveillance in the middle of the network is nowhere near as interesting as surveillance at the edge of the network. As the ghosts of Misters Hayek and Marx will tell you, there's a lot of interesting stuff happening at the coal-face that never makes it back to the central office. Even so-called "democratic" governments know this. This is why, for example, last year, the government of Bavaria started illegally installing the "Bundestrojaner", or the state-trojan, on people's computers, when they were of interest, something that allowed them to access cameras, microphones, hard drives, and so on. And of course, it was very badly written, so it allowed anyone else to do that, too. Once you were infected, you were infected for everybody. It's a safe bet that the totalitarian governments will happily take advantage of boot-locking and move surveillance right into the box. You may not import a computer into Iran unless you limit its trust-model so that it only boots up lawful intercept operating systems. Now, assume that we get an owner-controls model, wherein the first person to use the machine gets to initialize its root of trust. You still get the problem, because in Iran, every computer that comes into the country is first opened by the customs authority, who installs a root of trust that's run by the government. Because it's tamper-evident, even if you figure out how to override it, the next time a snitch or a policeman looks at your computer, they can tell that you've been up to something naughty and locking the government out of your computer. Of course, repressive states aren't the only people who like this. There are four major customers for the existing complexive censorware, spyware, and lockware. There's repressive governments, there's large corporations, there's schools, and helicopter parents. That is to say, the technical needs of protective parents, school systems, and enterprises are convergent with the governments of Syria and China. I don't mean that they have the same ideological grounds, but they have awfully similar technological means to attain their ends. We are very forgiving of any institution that pursues those ends, provided that they're doing so in order to protect either shareholders or children. For example, you may remember that there was widespread indignation, from all sides, when it was revealed that employers were asking prospective employees to turn over their Facebook login credentials. Employers argued that they needed to be able to review your list of friends, what you said to them, and what you did with them, in order to make sure that you didn't have any skeletons in your closet that would compromise your ability to work for them. Facebook logins were fast on their way to becoming the workplace urine test of the 21st century. A means of ensuring that your private life didn't have any unsavory secrets lurking in it, secrets that might compromise your work life. Now, the country wasn't buying this. From Senate hearings to op-eds, the country rose up against this practice. But no one seemed to mind that many employers routinely insert their own intermediate keys into their employees' devices-- their phones, their tablets, and their computers-- that allows them to spy on their employees' Internet traffic, even when it's "secure", with a little lock showing in their browser. This gives your employer access to all the sensitive sites you access while you're on the job, from your union's message board to your bank website to your Gmail to your HMO or private repository managed by your doctor's office to Facebook. Now, there's a wide consensus that this is okay because the laptop, the phone, and the tablet that your employer issues to you are not your property. They are company property. And yet, the reason that employers give us these mobile devices is because there is no longer any meaningful distinction between home and work, between personal life and professional life. Corporate sociologists who study the way that we use our devices have found consistently that employees are not capable of maintaining strict boundaries between "work" and "personal" accounts and their devices. And of course, in America, we have the land of the 55+ hour work week, where few professionals take any meaningful vacation time, and when they do get away for a day or two, they bring their Blackberry along. Even in the old, predigital, traditional workplace, we recognized that workers had human rights. We didn't put cameras in the toilets to curtail employee theft. If your spouse came by the office on your lunch break and the two of you went into the parking lot so that she or he could tell you that the doctor said the cancer was terminal, you would be rightfully furious to discover that your employer had been listening in on the conversation with a hidden mic and watching through a hidden camera. But if you take your laptop on your lunch break and access Facebook and discover that your spouse has left you a message saying that the cancer is terminal, you're supposed to be okay with that because the laptop is your employer's property. There are plenty of instances in which not just peons, but important and powerful people, not kids and corporate employees, are going to find themselves users of computers that they don't own. Every car-rental agency would love to be able to lo-jack the car they rent to you. Remember, cars are just computers you put your body into. They'd also like to log all the places you've been for "marketing" purposes and analytics. And there's lots of money to be made in finagling the way your GPS roots you around to make sure that you drive past certain billboards. But in general, the poorer and the younger you are, the more likely you are to be a tenant farmer in some feudal lord's computational land. The more likely it'll be that your legs will cease to walk if you get behind on payments on them. That means that any thug who buys your debts from a payday lender could literally — and legally — threaten to take away your legs (or your eyes, or your ears, or your arms, or your insulin, or your pacemaker) if you don't come up with the next payment. Before, I discussed how an owner override might work. You have some kind of combination of physical access-control and tamper-evidence, designed to give owners of computers the power to know and control what bootloader and OS was running on their machine. How will user-override work? I think an effective user-override has to leave the underlying computer and its programs intact, so that when the owner takes it back, she can be sure that it was in the state she believed it was in when she handed it over. In other words, we need to protect users from owners and owners from users, as well as users from other users. Here's one model for that. This is the hypothetical. I'm not suggesting we do this, I'm suggesting it by way of example. Imagine that there is a bootloader that can reliably and accurately report on the kernels and OSs it finds on your computer. This is a prerequisite for all the scenarios we've discussed: the one in which the state controls your computer, the one in which the owner controls your computer, and the one in which users may be able to control their computers some of the time. Now, give the bootloader the power to suspend any running operating system to disk, encrypting all its thread and parking them, and the power to select another operating system from the network or an external drive. So I walk into an Internet cafe, and there's an OS running that I can verify. It has a lawful interception back-door for the police, it stores all my keystrokes. It stores all my files, all my screens in an encrypted blob that the state can decrypt. Now I'm an attorney, or a doctor, or a corporate executive, or just a human being who doesn't want all of his communications being available to anyone who can bribe a cop. So I do some kind of three-finger salute on my keyboard. It drops into a minimal bootloader shell, and I can give the net-address of an alternative operating system, or insert a thumbdrive. Now the cafe owner's operating system gets parked. I can't see inside it. But the bootloader can assure me that it's dormant and not spying on me as my operating system fires up. When it's done, all my working files are trashed, and the bootloader confirms it. Not just because this keeps the computer's owner from spying on me, but it keeps me from spying on the computer's owner. Now, there will be technological means of subverting this. You could make a thing that looks like the bootloader but isn't the bootloader. But there is a world of difference between starting from a design spec that aims to protect users from owners and vice-versa, and one that says that users should always be subservient to owners. Now, human rights and property rights often come into conflict with one another. For example, landlords aren't allowed to enter your hotel without adequate notice-- or your home without adequate notice. In many places, the hotelier can't throw you out if you keep paying for your room, even if you overstay your reservation. Repo men can't take away your car without serving you a notice and giving you the opportunity to dispute it. When these laws are streamlined, we get all kinds of bad effects. Robo-signers taking away people's houses even though they've paid their mortgage or don't even have a mortgage. The potential for abuse in a world where everything is made of computers is, of course, much greater. Your car might drive itself to the repo yard. Or your high-rise apartment building may switch off its elevators and its climate systems, stranding thousands of people until a disputed license payment is settled. Now this has already happened with a parking garage. Back in 2006, there was a 314-car Robotic Parking model RPS1000 garage in Hoboken, New Jersey, whose owners believed that they were up to date on their software license payments and whose vendor disagreed. So the vendor shut off the garage and took 314 cars hostage. The owner said that they were paid up, but they paid again because what the hell else were they going to do? Now what will you do when your dispute with a vendor means that you can go blind, or deaf, or lose the ability to walk, or become suicidally depressed? The negotiating leverage that accrues to owners over users in this scenario is total and terrifying. Users will be strongly incentivized to settle quickly, rather than face the dreadful penalties that could be visited on them in the event of a dispute. And when the owner of the device is the state or a state-sized corporate actor, the potential for human rights abuses skyrockets. Now, this is not to say that owner override is an unmitigated evil. There are lots of reasons why you might not want users to override their computers. Think of a smart meter. Smart meters need to be able to turn down your building's temperature by a couple of degrees, otherwise we have to keep using dirty coal because it's the power source that we can raise and lower on demand. Now, that works best if users can't override the meter on their wall. But what happens if there's a big freeze, and a griefer or a crook or a government turns off your heat? What happens if the HVAC in your house is cranked to 110 degrees during a heat-wave and you can't override it? Once we create a design norm of devices that users can't override, how far does that end up creeping? Especially risky would be the use of owner override to offer payday loan-style services to vulnerable people. If you can't afford artificial eyes for your blind kid, we'll subsidize them, but you have to let us redirect your kid's visual focus to sponsored toys and sugar-snacks when you go to the grocery store. But foreclosing on owner override probably means that there will be poor people who won't get offers that they would get otherwise. I can lease you something, even if you're a bad credit risk, if I know I can repossess it handily. But if your legs can decide to walk away to the repo depot without your consent, you will be totally screwed the day that muggers, rapists, griefers, and the secret police figure out how to hijack that facility. It gets even more complicated, of course, because you're the user of many systems that you aren't-- in the most transitory of ways: the subway turnstile, the elevator, the blood-pressure cuff at the doctor's office, public buses and airplanes. It's going to be hard to figure out how to create "user overrides" that aren't nonsensical, although we can start by saying that "users" are someone who are the sole user of a device for a meaningful amount of time, although we'd then have to define "meaningful." This is not a problem I know how to solve. Unlike the War on General Purpose Computers, the Civil War over computers seems to present a series of conundra without any obvious solutions, at least, obvious to me. Which is why I'm talking about them to you. These problems are a long way off, and of course, they'll only arise if we win the war on general purpose computers first. But come victory day, when we start planning the constitutional congress for the new world, where regulating computers is acknowledged as the wrong way to solve problems, let's not paper over this division between property rights and human rights. This is the sort of division that, while it festers, puts the most vulnerable people in our society in harm's way. Agreeing to disagree on this one is not good enough. We need to start thinking now about the principles we'll apply when the day comes. Because if we don't start now, it may be too late. Thank you. [applause] So I've got some time for questions now. You don't have to ask me questions about this. You can ask me questions about books and stuff. >>Male #1: I have a question. So one thing usually people don't talk about is why we should not allow people to inspect what we're doing. If you're not doing anything wrong, why do you care? I've only seen an argument to this once, when people said, basically, it violates a fundamental expectation of humans to be individuals rather than part of a collective ant hive or ant colony of some sort. Now I wonder if you could speak to that. >>Doctorow: Yeah. I mean, there's-- I think that that argument starts by presupposing that everything private is secret, and everything secret is private. We say, "Oh, well, it's not a secret what you're doing, so why do you need to keep it private?" But I can make a pretty good guess about what you do when you go to the toilet. I'm pretty sure I knew what your parents did to get you here. But it takes a pretty special kind of person to want to do that in public. There are behaviors, and not nefarious ones-- In fact, some of the most important ones, the ones that, you know, are the origin of all life and the reason you don't explode in a shower of poo, that we habitually do in private, and that aren't the same when you have to do them in public, particularly if you're coerced to doing them in public. The modern concept of privacy is pretty new, but there are elements of our privacy that are quite old: the privacy of thought, the privacy to make mistakes. I mean, remember this notion that if you want to double your success rate, you triple your failure rate. It's very hard if you have to make all your mistakes in public. You may-- Does anyone here work on Blogger? I mean, before Blogger was a really big deal, when it was a little deal, it was running on an NT box that Ev found somewhere. It went down all the time. And no one cared, because he wasn't in the public eye. Now, you guys can't afford to experiment with the Blogger backend the way Ev could. Ev could refactor his code altogether, take it offline for two days, and then put it back on again. He was able to innovate really, really fast, in a way that you guys can't, because you don't have privacy in what you do. What you're doing is public. If you've ever watched a kid play, and play in a way that's sort of pushing at their boundaries, they do this thing where if they don't know you're watching, they make a lot of mistakes, and they just keep pushing through them. But if they catch you secretly watching them while they make mistakes, they put the thing away and they walk away from it. It just kills their play. As a father, it's the thing that breaks my heart when I do it, because it's very tempting to look over at your kids when they're doing something awesome and intense. But then, you humiliate them and you embarrass them. So there's something about us that wants to have vulnerable moments not take place in public, that wants to choose the moment of disclosure. I think that that's-- that doesn't change just because we have Facebook, or just because we can track user behavior with 1x1 pixel gifs. You know? Yeah? >>Male #2: You've published a lot under the Creative Commons license. So I was curious, from the point of view of someone who's incredibly cynical and just wants to make a living writing things, would you advise it? Will it catch on? >>Doctorow: So if you want to make a living writing things, I would advise you to stop trying, because [laughter] that's a bit like saying, "I want to make a living buying lottery tickets." It's like--. That sounds like a great plan if you can find the winning lottery tickets, but if you don't have a plan B for earning a living, you have the wrong career. Writing is a very, very high-risk entrepreneurial venture that almost everybody who tries it fails at. Some fraction of the people who try it succeed using Creative Commons, and some fraction succeed without using Creative Commons, but they're rounding errors against all the people who try to earn a living with writing. So for me, the reason to use Creative Commons is not just commercial, although I think in my case, it enhances my commercial fortunes, because people who get the book for free then go on to buy the book. That may not be true of everyone. There isn't a kind of global theory about this. But there's two other dimensions to it. The first one is moral, and then the second one is artistic. The moral case is that I copy all day long, you copy, everybody copies. If it wasn't for mix-tapes, I would have been a virgin until my mid twenties. You know. Copying is what we do. If I were 17 years old today, I would have a giant hard drive and it would have a copy of everything. So I--. To pretend that when I copy, it's like part of a legitimate, artistic adventure that allows you to assemble your influences and recall them as you need them, but when you copy, you're just a thief, that's just dumb, right? And moreover, it leads to all these crazy consequences, where we're talking about three strikes rules in New Zealand, for example, where they're saying if you-- I know that's an Australia shirt, but very close to Australia, where they're saying-- >>Male #2: We don't think very much of the Kiwis. >>Doctorow: I understand. It's like Canadians and Americans. But they, you know, they're saying if you are accused of three acts of copyright infringement, we take away your internet access and all the stuff that comes with it, and that's partly being driven by people who say, "Well, if you copy my stuff, it hurts my fortune." Being able to give my stuff away means that I'm not part of the rubric for Draconian network policy. But then there's a third dimension, which is the artistic dimension. It's the 21st century, and if you're making art that you don't intend to have copied by people who like it, you're not making contemporary art, because the realpolitik is if someone likes it, they'll just copy it. Right? I mean, we put DRM on ebooks. This is crazy. It's like they've never heard of typists. [laughter] It's like there have-- It's like they don't know that we live in the moment with the largest number of skilled typists in the entire history of the world, you know? This is an amazing--. My grandmother was like a 75 word per minute administrative assistant, and she was like a circus freak. Today, she's not even in the top quintile. Everybody can type. If you're making art that you don't intend to have copied, you're not making contemporary art. That's cool. I mean, if you want to be the blacksmith at Pioneer Village or reenact the Civil War, that's awesome. Go follow your weird. But I'm a science fiction writer, so I'm supposed to make at least contemporary work, if not futuristic work. It gives me great satisfaction to allow it to be copied. >>Male #3: So, a question about the average user. So if somebody is sitting out watching this broadcast on YouTube, and they think, "I want to be a part of the solution", it doesn't feel like there's a great avenue for them to express the need for digital freedom other than disobedience. Do you have any suggestions as to what people might do? >>Doctorow: Well, the first talk, the 28C3 talk which is on YouTube, ends with a pretty good, compact call to action for people of all stripes. If you're a hacker, get involved in things like Free Software Foundation and policy stuff that the Free Software Foundation does, or get involved with the Electronic Frontier Foundation. If you're a lawyer, join the Cooperating Attorneys list for EFF and get involved in other groups like FSF and so on.org, or Nets Politique. Bits of freedom all over the world: we have them. If you're an artist, use Creative Commons. And so on. I actually think that there's a lot of venues. Today, we have, between Defective By Design and Fight for the Future, who led the SOPA/PIPA fight. We have so many different groups that are doing really exciting things that need people to do everything from send an email to their congressman at the right moment to design logos and packaging and write copy and put the word out and blog and give talks to their school. If you're a student, you can join Students for a Free Culture. That's all great. I think we don't have that answer for the civil war thing yet. We don't. And we kind of—. And my point about the civil war is that we'll get to the civil war pretty quickly after we win the war. We get to the thing where as soon as you give owners total dominion over their computers, you immediately get to the moment where users can't trivially change what's on their computers. I would be really nice if, as we sit here advocating for owners having total control over their computers, that we start thinking about when users should be able to change that. >>Male #4: So you gave some examples comparing and contrasting physical devices versus electronic goods, and devices that are used by many users and may be only one user. And you said, "Well, some of these policies, there's analogies between physical and electronic that you should use, and some of them don't really make sense. There's a discord." Are you saying that, overall, we should go by case-by-case in order to examine: does this make sense? >>Doctorow: No. I mean, I hope that's not what we end up having to do. I mean, that was the point at the end and the beginning. I don't know how to solve this one. And it would suck if, basically, you solved this with nested if-then loops. It would be really nice if we had a nice, generalizable case that we could say, you know, "If it's not a nuclear power plant, anything goes"? Or something else. Right? I mean, if every single thing that is Turing Complete has a different set of rules for when users and owners get to control it, that's going to be a really big rule book. So I would love to have a better one. I don't know what that is yet. Maybe, after I've given this talk for another year or two, between all the feedback I get from people like you, I'll be able to propose a solution. >>Male #5: It was interesting that you talked about having a user override mode where you could change what the employer was doing and come back. I don't know if you followed much of it, but [beep] Chrome OS does that now. >>Doctorow: Yeah, I just heard that at the weekend that there's a user override. And it's funny, because the first email that got me thinking about this was Vint Cerf saying, "Why shouldn't Google be able to choose what software runs on my Chromebook if they bought it?" >>Male #5: And we don't, and I work on that. I'll be happy to talk to you about that. And for the rest of you guys, I'm about two months away from having a way that you can put your own keys on it, so you can sign your own images and boot your own stuff. Right now, you have to turn the security off. But we're doing that, and I would love to do more. >>Doctorow: I think that's a really cool model. I'm done. Of course, the really challenging thing is going to be computers that don't have interfaces, like your legs. >>Male #6: So this is a little tangential, but we are reaching the point where a lot of third parties can maintain public databases of public information about you. So, like the American Credit Report is one of the original settlers. So is there any legal theory that would give you rights over the database? >>Doctorow: Is there a legal theory that gives you rights over that database? I don't know, but I like the fact that you said rights instead of property rights, because I think that we have started-- We have the best of intentions sometimes. We created property rights and facts, or property-like rights and facts, about you that don't make any sense. Like, you know, the Well, which is very old, and now endangered. Salon just put it up for sale, this online conferencing system. It's motto is "Yoyow", "You own your own words." And that sounds like a really cool idea, but it has all these weird fraud things, like if we're in a conversation and I quote something you said, do you get to tell me to not quote it? I mean, this is a contract, not fair use. This is what our contract says. And the European data norms are starting to move towards ownership of your personal information. But what does it mean to own your phone number? You know? Does that mean that if your phone number happens to contain the first seven digits of pi that other people can be enjoined from writing pi? And it's funny because we do actually have ways of expressing value about things that aren't property that we may be able to bring in here. We talked about interests a lot. My daughter is not my property, she's pretty important to me. And if you kidnap her, the charge isn't theft. But we can acknowledge that my daughter has an interest in herself, that I have an interest in her, that my wife has an interest in her, that her grandparents have an interest in her, that the state has an interest in her, that her friends have an interest in her. That's what it means to be a person in a society. We need to start, I think, talking about information that way. It's crazy, I think, to talk about things like phone numbers or your address. This is where I think, you know, even though I'm a privacy advocate, I think the Germans were crazy about saying you own the likeness of the front of your house. I mean, that's just dumb to me. 'Cause it means that, like, as you move through time and space with prostheses that record the world, you can't record your neighbor's house. You can't record your kid's first run down the street on her bicycle without the training wheels because she rides past your neighbor's house and they own the likeness of their house. Right? That's just dumb. We need to be able to express--. And property is a bad organizing metaphor for a thing that a million people own. Right? You end up with, like, shareholder corporations or, you know, there's this whole Spider Robinson aphorism when 700 people share an apple, no one benefits, especially the apple. You know. [laughter] That's true of physical, rivalrous property, but non-rivalrous information doesn't have that characteristic. We still may want to give exclusive access or semi-exclusive access to certain parties. Like, the image of your colonoscopy may be something between you and your doctor. But to call it your property is the wrong thing. It doesn't organize well that way. I don't know what does, but I know what doesn't work. >>Male #6: No, but the point I'm making is that, let's say ten years from now, somebody could run a background check on your without your opting into it? >>Doctorow: They can already do that. I mean, they already can. >>Male #6: Okay. >>Doctorow: Yeah, I mean, it would be-- So Lessig talks about four ways of organizing-- of regulation. He talks about law code markets and norms. So we don't have a lot of code to help people protect their privacy. Like, when you fire up your laptop, it doesn't-- So here's an example. If you had a browser that, every time you turned it on, loaded the-- checked to see whether it was being asked to load the Google Analytics JavaScript, and suppressed it, but implemented all the features, all the libraries locally so that pages didn't break. That would be code out of the box that treated-- that defaulted to treating privacy as though it's valuable. And so now, if Google wants to get your private information from you, information about where you are on the internet, they have to offer something of value to you that is inextricably linked, because that is extricably linked, right? You can-- We can, in fact, conceptually understand how you divide it. So, like, if my Android phone, when I installed an app like my daughter's Connect the Dots app, it said, "In order to use this app, you need to tell us where you are all the time." If it let me say, "Tell this programmer where I am all the time, but make it up", then the program would actually have to devise an offer where where I am was actually a piece of using it. So I use another Android app all the time called Hailo, for hailing black cabs in London, which are a pain in the ass to get when it's raining and so on. And Hailo knows where I am all the time, and so Hailo has an offer where if everybody else couldn't get my location trivially just by, like, getting me to download a Connect the Dots app for my kid, Hailo would be sitting on a giant asset. And you'd have real privacy markets. Like right now, we have this idea that we know what your privacy is worth, and it's worth nothing because you trade it for zero. But you don't have the option of not trading it for zero. So you could imagine--. One way that you could stop people from being able to do background checks on you really trivially is if all the devices that you have didn't hemorrhage information about you all the time, as though it had no worth. >>Male #7: Okay, change of gears to a fluffy, lighthearted thought experiment. >>Doctorow: Sure. >>[Male #8]: Since you mention the Norwegian script kiddies, imagine the Marcus Yallow Memorial Pentathlon. Which countries take gold, silver, bronze there? >>Doctorow: Oh, wow. I'd like to think-- I mean, without being ideological about their governments and just thinking about their track records, I would think that you'd get the four brick countries plus Israel, probably. Brazil, Russia, India, China, Israel. Without endorsing or condemning any of those governments. >>Male #8: All right. Thanks for coming. It seems like you're looking for sort of an overall rule for what people can and can't do with their devices. But I'm afraid it's going to end up just a whole pile of special cases, kinda the way it is now. Like, if you look at your car, what hardware and software you can do, some things are legal, some things violate smog laws, some things violate safety laws, some things will be charged with criminal negligence if something goes wrong, some things you'll get sued if something goes wrong. >>Doctorow: Are you saying that if you modify the firmware, or if you modify the firmware and then something bad happens? >>Male #8: Well, you know, with the smog law, just modifying the firmware, I believe, is illegal. >>Doctorow: Is that right? I didn't know that. >>Male #8]: Well, I think in California. You know, there's other things you can do that-- If I decide to reprogram my brake system and I crash into something, I'm likely to get either sued or go to jail. But other parts of the system, you know, it's probably okay for me to redo-- if I want to put in a new engine, but in new shock absorbers. So it's-- I think these issues apply to both the hardware and the software and firmware. >>Doctorow: So, it makes a certain amount of sense to me. I think you're describing after the fact, largely, modulo this question about whether changing the way your car is smogged gets you in trouble or doesn't. I think, mostly, you're describing after the fact stuff. So in the same way that if I program my nuclear power plant so that it melts down, I'm held liable for having written bad code. Or, if I program my software to find radio so it turns into a spark app generator and blows all the RF in my region, again, I have done something bad and I'm punished for that, but it's not against the law to write my own software to find radio code. You can check code in and out of GNU radio on GitHub without breaking the law. If you use that code in a way that ends up breaking something, that may be illegal. And that's kinda what I'm talking about here. It may be that users take control of their legs to run up to someone and kick them in the face. And I don't think that-- I think that writing code that lets you take over your legs is good. I think that, having taking over your legs, to kick someone in the face is bad. And I think that we can punish the one without punishing the other. >>Male #8: Okay, that seems fair. >>Doctorow: Yeah. >>Male #9: You touched earlier on the concept of an illegal number, which is something that I've thought about a lot, because all information can be a number. Which raises the question, which I think is central to all of this, of where you draw the line. >>Doctorow: Sure. >>Male #9: And obviously, to anyone who has studied any kind of mathematics, there is no line in numbers. So the question is: why is there a line in anything? >>Doctorow: Right. >>Male #9: In essence, why do you believe that this problem is at all soluble? >>Doctorow: [inhales] [laughter] Let me find my illegal number here. There we go. Yeah. That's a really good question. You're right. Everything can be encoded as a number. I mean, now we're getting into girdle and incompleteness and whether numbers are special. >>Male #9: That was just an example, though, of why it seems a priori that this is likely, technically, not soluble. So the question is: why? >>Doctorow: So I think it's soluble in time scales. It's not soluble in infinity. So, for example, we may say that--. So today, we have a bunch of rules about locking and unlocking that are largely governed by the copyright office, because the relevant law is the DMCA and anti circumvention rules. And so you may have heard that in the triannual review, it was made legal to unlock phones and tablets, iPhones and tablets, and also to unlock phones so that they can switch carriers. That's a thing that works for now. It won't work against more robust bootlockers and it doesn't help certain classes of users. But, I mean, I don't think we pass technology laws that are supposed to last through the ages. I think we pass technology laws that are supposed to last, we hope, through the half life of the technology. You're right that there will come a time when the rainbow table of all numbers exists, and all possible decodings for them exist. I mean, it may occupy all the hydrogen atoms in several parallel universes to ours. But it's at least within the realm of contemplation. But that doesn't mean that between now and then we should try not to have any rules about how numbers are used. You know. I mean, you might say, "Well, the specifications necessary to get your AR15 to go full auto can be expressed as a series of numbers." That's a bad example, because you don't suppress the rule about the specification. But, like, the 3D meshes. That currently exists as a 3D mesh. I think that was uploaded to Thingiverse. I think they've taken it down. That converts AR15s from semi to full. I may not agree with regulating that, but I wouldn't disagree with it on the grounds that it's impossible. Right? Like, that it's impossible to say-- telling people-- giving people AR15 automatic modification kits can't be made to work, or shouldn't be made to work, because those numbers might also be poetry or something. Like, it just seems like, although that's true, it's true in some sense that the law can safely ignore for a while. Does that sound right? It's hard. >>Male #9: It's good enough that I don't want to take up any more time following up. Thank you. >>Doctorow: Okay. Thanks. [applause] [whistling]