Utilities Act 2000

Transcription

>> GUNTHER: My name is Eric Gunther with EnerNex Corporation. I'm the chairman chief technology officer at EnerNex. We're electric power engineering, research and consulting firm. I happen to be working in the smart grid arena a lot right now. My background is electric power engineer, but I also have heavy background in enterprise architecture, utility communications and the like. Basically, I often times get introduced as someone who speaks geek in mode of languages. So, we'll--which happens to be an important characteristic for smart grid today. Now, what I'm going to talk about today is provide a little bit of overview of the use of Internet technology, specifically TCP/IP and other more advanced protocols in utility automation from four--from five years before Google and earlier. So, I'll occasionally refer to my BG and AG timeframes for--before Google and after Google, so BG and AG. The first thing to note that for utility automation, what we're dealing with here are a convergence of at least two infrastructures. I can argue more, but we at least have two infrastructures we have to deal with. We have the existing electrical power infrastructure. The lines you see along the road and in the fields, you know, between cities that connect central generation stations, substations for long distances, the distribution stations, the small fenced in areas with smaller equipment you see in cities and--but that then emanate out along feeders anywhere from a few miles to a 40 or 50 miles long, you know, to where the loads are located. So a significant physical infrastructure we've had in place for over a hundred years in one form or another very similar to what we still see today. That physical infrastructure, however, is--has been overlaid with sometime by a communications infrastructure. It's a matter of the level of sophistication of that infrastructure that determines whether we want to qualify it as smart grid or not. But for a number of years in the history of electric power management, we have been using communications command and control to manage that very--these assets on the electric power infrastructure. I could spend a couple of days talking about the various standards and protocols that are used in this infrastructure and the shortest I usually can do it is in about a day, but I'm going to try and do it in an hour here. But this diagram here, you know, gives you an idea of the different standards that are involved, the different domains of utility automation. We've got the back office, the enterprise, for which we need to have protocols, information exchange, network communications, the control centers, sort of a specialized element of the back office enterprise. Getting out into the field through wider end networks of a variety of different technologies to connect to those various substations and other assets that are distributed to all over the country where the devices are actually located. Then getting into the local areas, what we like to call the field area network providing us the connectivity to commercial end-users, to consumers, residential end-users, to bulk generation sources, to all the various substations that are out there, to distributed resources; so all of these different elements need to have some level of connectivity for the most basic operation as well as the most advanced operation. What we're--I'm going to, you know, be able to show you is that for large extent IP is already pervasive in all of these. So, in the news and the like sometimes the utility community is portrayed as being a bit backward and, gosh, why don't they just use, you know, IP. Haven't they thought of that? Well, the answer is yes. It's been in place for quite some time. It's just that it hasn't been exactly all that well-known and as sexy as some of these, you know, interesting uses of IP that you guys are responsible for putting together here. So I'll give you a little bit of background of what's been done and some of the reasons which will point to some things that we can do and bring new innovation moving forward. You know, why automation, in the first place? Now, certainly a hundred years ago, we didn't have communications like these. We relied on automation but mechanical, entirely local, no communications. But what--so, you know, what really drove, you know, automation? Well, a couple of key events really drove that. Starting in the '60s with the Northeast Blackout of 1965 that lead to a whole lot of different efforts to change the way that the, you know, power systems is managed, how we measure reliability, the standards we have in place, the need to automate, to understand how various protective devices interact. It was a defining event that resulted in the creation of the Electrical Power Research Institute, EPRI. And that was reinforced in 2003 when we had another north--widespread Northeast Blackout, and it uncovered a whole series of new areas where we weren't communicating as well as we should; human communication as well as machine to machine communication. And I can trace path and have another talks that can show that August 2003 blackout. It was, I think, it was actually August 14th. But we contributed to what became the Energy Independence and Security Act Of 2007. But prior to that 2005 act that states look at smart metering. So a lot of that, well, I can say started from this event. So historically, you know, in the utility automation, we've been dealing with mechanical devices that work just fine. They were very intelligent devices in their own, small mechanical computers, if you will, that perform very specific functions very reliably with a little or no maintenance for many, many decades. And some of these you can walk in to some substations in the country and find some of these, you know, still in place today and working just fine. But, you know, there, you know, was a, you know, an observation that we can use, you know, in technology to make these devices be more cost-effective and operate in more innovative ways as we apply microprocessors to another technologies. So, you know, in the '70s, we started adding electronics to automate these devices, which then led to being able to connect them. So today--and we use the term protective relays. Probably, the most common intelligent devices in the grid today are the devices that keep the grid safe; keep things from blowing up, keep transformers from blowing up in your neighborhood, make sure that if someone or something contacts a line, it very quickly is de-energized and, like, we call those protective, you know, relays. But they're some of the most intelligent things in the grid. And if they misoperate, it has this most significant opportunity to disrupt the grid if an adversary gets in or something behaves incorrectly. So since the advent of cheaper communications, cheaper electronics, there's been an explosion of Intelligent Electronic Devices. You know, pun intended on the--if you noticed the acronym that we use, you know, for these devices is the IED. And in our world, the utility world, we invented IED before the terrorists did. So I'll be using the term IED, you know, quite a bit, you know, here during this discussion. The general concept of managing devices on the power grid that, you know, is generically internationally called Telecontrol. But in North America, we generally referred to it as SCADA, Supervisory Control and Data Acquisition. And, you know, this evolved, take advantaged of these new electronic devices, find ways to remotely control them, monitor their status, changed their configuration. And this was done with very specialized protocols for performing utility operations. The characteristics of these protocols were very dependent on the--its long-lived history of how the mechanical devices evolved, electronics were brought into play the mimicked the mechanical behaviors, added on to them. And so a lot of the way things evolved were because of the long history and long-lived nature of the devices in the field. So I'm going to cover a few of these talk a little bit about the multitude of protocols that we started off with and where we have been moving towards here recently. So in the beginning we really did have the Tower of Babble. There's a hundred or so vendors of different types of equipment. Every vendor, you know, had a protocol. It just seems like a right of passage that if you're a new engineer at a new project, my gosh, you got to create a new protocol yourself. I would just--I don't know any engineer who hasn't done it at least one time before they finally realized that after awhile that that's probably not the best thing to do. So, that was done in those days. It was trying to encourage brand loyalty or we now call it vendor lock-in in order to, you know, get the business with a particularly utility or infrastructure provider. Huge investment was made in the SCADA master stations, that's where all the money was. So it was to the vendor's advantage to lock the utility in to that particular master by using their own protocol to the end-device. So, data concentrators then were created within the substations in order to talk to all the other devices. So we early on had a multitude of, and still today, of data converters in order to provide a well-defined points of interoperability. This is just a partial list. The full one takes many pages of the SCADA protocols that were out there. Many of the protocols are named after the utilities that invented them, okay. So, AEP protocol, PG&E protocol, okay, that are one of the earliest ones, but a whole host of different protocols. Some specific to utilities that invented them, some to computing companies, large equipment manufactures; ABB protocol for example, and others. So we've had a lot of them. And given the long-lived nature of the utility infrastructure, a lot of these are still out there. But they're collapsing pretty rapidly, you know, to a few that I'm going to focus on, you know, here now. But if we go back and we look at, you know, around 1980, and we're trying to do this, you know, automation and replace the electromechanical relays and other field equipment with electronics, you know, we were dealing with--talk about 8-bit processors, 9,600 bits per second was considered, you know, really fast. You know, for the most part, 300 baud was plenty fast enough for--and actually for some things today it's still is. Memory was very expensive. We didn't really have anything in the way of radios, so we were using mainly least serial lines for everything. The case to automate, the business case was unproven; some say it may still be unproven. And, you know, basically, you have the mechanical device, everyone understood a new or this new fang of electronic thing that maybe you could communicate. And they wanted to spend the same money, you know, with it. So, you know, as the two. So the fact--the cost pressure is--made it very difficult for those new electronic device to be very sophisticated. So varied processor constrain had to be brought stupid, you know, cheap in order to produce. In the same context, our connectivity was essentially phone lines; list lines, dial-up lines; very poor quality because the radios were still too expensive, haven't even thought about LANs yet. Power line carrier was used early on, that was one communication mechanism that was used. But the earliest protocols that were developed were not reliable. They couldn't compensate for, you know, the noise on these things. So a lot of the protocols evolved into some very arcane implementations in order to get around these, you know, these issues. So, a major failure is early on, really, you know, caused a lot of paranoia, and things then got over built and over built and over built with extra and more redundancy, you know, built-in. A good thing in general but done--not--it wasn't engineered, it was just done. So, you know, starting in the mid '80s to early '90s, you know, big customers retire to the vendor lock-in looking for reuse and standardization. We're starting to, you know, get new protocols being available. You know, things like these new fangled, you know, thing TCP/IP and other advanced protocols. Users groups were beginning to form to band together as utilities and infrastructures owners to force the vendors to try in and get some standardization in place. We still have some limitations, a lot of limitations, still not much bandwidth in order to--we have more ideas than we have enabling technology to be able to allow us the cost-effectively implement it. So really, you know, moving into the early '90s, so five BG, you know, five years before Google, 1993; you know, SCADA discovered the Local Area Network and that's where we really started to apply it. Now, a lot of people, you know, would thought, between 2003 or 2005, I mean, you were starting this Local Area Network and TCP/IP in '93? Well, yeah, we were. And a number of things, you know, were driving that. Office computing was, you know, starting to spread the SCADA world. We had engineers who were using off as computing networks and, you know, they wanted to go and try to put that in the substation because they can. And so we had more CPU and memory, bandwidth is available; a couple of other drivers. You know, in '93, we had the National Information Infrastructure initiative that Al Gore was involved in. And outcome of that was a Department of Energy grant specifically designed to use Internet protocols for energy applications. And a lot of folks aren't aware of that project, but it was sort of a groundbreaking effort to specifically, you know, look at how to do that. And the screenshots I have here are from the first implantations of that using an old Netscape, you know, Netscape browser and an early Internet Explorer browser in '95 to query data, in this case, from Naztech. You know, pulling back energy usage of information, the power quality of event information in the '90s, you know, in 1995. One of the main things that have to happen, we tried doing this a little bit earlier. But getting reliable TCP/IP stacks to work in embedded systems was very difficult, you know, to find at that time and operate, you know, for more than--in more than a day before, you know, rebooting memory links, all that happy stuff. So that you know, that was a challenge early on. We start--be able to discover that. The big problem that we still have in those times was reliability and robustness in a non-office environment. We had folks developing, you know, office equipment that, you know, had its own reliability in the office environment. Let alone, putting something in the substation for which when there is a problem on the system, thousands of amperes of current flow causing electromagnetic interference that would just totally blow the brains of, you know, most any equipment that was--would be use in an office environment, so that was a challenge. The SCADA protocols we're very simple; memory-mapped to register-based. The meaning was related to a physical memory location, a specific bit pattern. You know, no concept of layered systems, loose coupling, you know, that--this was an incredibly tightly coupled program to the bare the metal, if you will, in order to scrape out every ounce of performance. And, you know, all sort of innovative crazy things were done in order to get the information from point A to point B with very low overhead, because yet things were still very, very expensive. So once we got beyond that proprietary stage, we started to get some cooler lessons around, you know, the concept of developing some protocols with some more modern computer science and communication's theory applied to it. Early protocol that's still around today, called Modbus was, you know, very widely used in industrial automation, but some very basic concept of being able to set the variable and get a variable. It was patterned after physical manifestations. So, when you look at the terminology, you'll see registers referred to as coil registers or input registers or holding registers. It still have mapping to the physicality of the devices that were there. So, you know, they're still very, very tightly, you know, coupled. So we had some of these early protocols but there was no standard for time stamping the data. One thing that you do mechanically in an electric power infrastructure is you want to select something first, you know, basically, you know, prepare the device so that we're ready to operate it. Once we make sure that, yes, in fact, we know this is the device we're going to operate, then you operate; it's so called select before operate. These early protocols didn't have that and there was some rather spectacular failure when, you know, when an addressing scheme inadvertently addressed the wrong device. And instead of opening, you know, this breaker, it closed the other and caused a massive failure. So, you know, some key features were missing on this early, you know, elements but they did survive. And that protocol has evolved to operate over TCP/IP, but it doesn't have any inherent built-in security and, you know, it's not very extensible. The, you know, the protocol that evolved after that is one that's probably still the most pervasively deployed in the world today, especially in North America, DNP3. It was developed by a company called Westronic which later became a part of GE and GE Harris. It was based on some early IEC, International Electrotechnical Commission at work in Europe to develop a protocol, but they weren't interoperable. That evolved overtime to create a users group. It's now being standardized in the IEEE and it's really sort of the work course today of Remote Terminal Unit/Intelligent Electronic Device communications; mainly, again, in North America, South America, Australia, Africa, popular in the water world as well. It's still--it's robust enough, it's design, you know, it was good enough, and that we had communications engineers working side-by-side with the water engineering domain experts that we started to, you know, get some things into it that allow it to be more extensible and long-lived, supporting various mappings, ongoing physical layers. It lent itself to being able to create a TCP/IP based implementation. And we are able to put a security in it at a relatively low level of the protocol. So very reliable, still tight, you know, still more register-based that doesn't have information per se. You still--a vendor A and vendor B, vendor A may represent voltage in registered number 23 and represents the units in volts, but vendor B may put voltage in register 47 and it may be in kilovolts. So I'm still, you know, quite a ways away from the kind of interoperability in self discovery, you know, we see in other protocols in other industries. The final evolution was to the 61850 protocol family which was sponsored by the Electrical Power Research Institute, starting in the late '80s leading up to a specification in 1992 that really merged fully the information technology community, computer science folks, and the power engineers with their demand expertise to come up with a relatively--a pre-elegant system that supports this architectural concepts of layered systems loose coupling and the like and the concept of an information model. The problem was in its--in the mapping of that abstract model was very well-done. The physical manifestation to the MacFire layer was pretty poor. You know, serial was the primary mapping. No TCP/IP support, but it was focused on the--what was very popular at that time in the federal government, you know, the full ISO network stack implementation competing with TCP/IP at that time. So a lot of pilot projects, a lot of pilotitis but no consensus. And--but there's nothing like a real project to focused the mind and actually get something done. So American Electric Power had a need to replace some of their already aging infrastructure, being a, you know, utility with the, you know, one of the older utilities, they needed to replace some equipment. So they had an initiative to bring some focus to it with resulted in taking the best of what was done before and creating a TCP/IP support to the protocol via RFC 1006 that maps the ISO stack, you know, to TCP/IP. That was published as an IEEE standard first as a mechanism to transfer it to the IEC and within the 2004 or 2005 timeframe, the various parts, the subparts where then finally standardized as what we know as 61850. And that's probably the leading standard for new implementations today, definitely in the rest of world, we're a little slower in the US for a variety of reasons. And the most of the rest of the world standards are treated more like law. Once a standard is available and applicable, pretty much right from the get go a new installation is going to use it. That's how they are treated in the electric power industry. In the US, all of our standards pretty much are voluntary and so even new installations today are still using, implementing some of the old stuff because we just don't have a mechanism to mandate or--other than heavily suggest the new stuff. A part of that is another protocol called the Inter-Control Center Protocol. This is the protocol that's used to communicate between control centers and support wide are management that shared the underlying protocol that's underneath UCA at 61850, which is the manufacture's messaging specification, MMS, you know, for, you know, the underlying bits on the wire encapsulization, and that is now a separate standard. So all these standards finally lead us to a point where we have some standards available to us for field device communications that have a concept of standard names for standard things, of an information model that's a vendor independent. So, when I buy a device from vendor A and vendor B, both of them, if they want to represent voltage, the same name string is used to, you know, to reference them regardless of how they're mapped internally to what kind of processor they're using, your memory, architecture, what have you. And that was a big deal for the, you know, for the utility industry. That just shows the mapping, I'm not going to go through that. So what's so great about this? You know, basically, what no other utility protocol had had before 61850 came along is--we now have the ability for self-description. There's an addition to standard names for standard things for the data itself. We also standardized the ability for the metadata. We can retrieve the information about what units are represented. We won't have Mars Lander, you know, problem. You know, we can go and ask the system and, you know, is this, you know, what are the units that we're getting this from? Is it volts? Is it kilovolts? Is it furlongs per fortnight? Okay. So we were able to represent those in any one we want, but we--it's discoverable. Instead of we're dealing with structured data now, we're dealing with data that is derived from common classes so that we can reuse various classes of data. A lot of very fundamental concepts of information architecture, you know, we take advantage or take for granted in most other industries. But the utility industry, bringing together the power engineering world, just trying to keep the lights on with the communications world, that was a big deal. So a number of capabilities were brought--made available to us. The ability to replace traditional point to point copper wiring to interlock one protective relay from another with being able to do that over an Ethernet network, being able to transmit waveform samples in real time. So, instead of having to put a current sensor, you know, and run copper wire to every single device that needs that current information, we can put one sensor in, digitized it and make it available to a whole rack of equipment that can see the exact same data without having to duplicate all those analog sections and the cost associated with that, so a lot of really good things. The--so in general, you know, the main things that are different, we have object models, we've got the "Nouns" that we could--the protocol operates on. We've got application services, an abstract communication service interface that helps us with this layered concept, technology layering and abstraction and loose coupling, and then underneath all that we have the profiles, you know, that can change over time. Today, we may be using MMS to--as the underlying protocol to instantiate the abstract service interface, but we also can use other protocols underneath that, new profiles as they became available, more readily available, have additional features that we want to use and we can map them without affecting any of the stuff, you know, higher level on the stack. When we talk about 61850, particularly, it's a suite of protocols, not just one. It took a lot of years to develop all the different parts, but we've got the sampled measured values and generic object oriented substation events that basically operate directly as their own ether type, they bypass the rest of the stack because we're dealing with submillisecond performance requirements. The difference between a substation responding to a car hitting a pole and causing a short circuit and making sure that that transformer is simply the de-energized rather than it blows up. You know, maybe milliseconds. So some things were done to support very fast, you know, connections. But for the bulk of the work, using the MMS protocol suite with TCP/IP profiles, you know, allow us to provide communications and take advantage of all the infrastructure that we have, you know, in front of us today to deal with TCP/IP based traffic. So the typical substation, we're dealing with, Intelligent Electronic Devices that are connected to generally two busses, what we call the station bus that's used for just polling and reading the various data from these devices, controlling them, opening and closing breakers, reading settings back, changing the settings of protective relays and then a fast bus to process bus that maybe receiving the high speed sampling from those voltage and current sensors in real time to, you know, that drive the way that the sensors all work, the EIDs work. So the standards associated with that, standards associated with the fast side of it, the time is syncing. The time syncing has turned out to be a very important. One of the things we learned in the 2003 blackout is that without time stamping we couldn't really figure out exactly what happened. So, since that time huge driver and values has been associated with trying to get the time stamping right. So the protocols are all based on existing standards at the higher level, the application layer, the object models, again, the standard names for the standard things. That's where the technology independent interoperability comes from, that's what gives us the vendor independence. It isolates us from technology change at the lower levels. When I first was implementing this, we were implementing this on 10 megabit Ethernet, everything works just fine with gigabit Ethernet. It works fine over fiber-wired, you know, what have you. And, you know, so that benefit of layering, you know, comes into play. All right, I'm not going to go into the incredible details here. So what can you do, we, you know, the services provide the ability to name objects, to read and write them, to describe them, to be able to report when information changes, to log data, to do file transfer. You k now, so what we're--we have this thing called the logical node which is just basically an object and it instantiates one or more common data classes. So it's a very classic computer science, you know, things that are very--come natural to an information architect but were new for the utility industry. But we're able to do this in the early '90s, which for the utility industry was, looking back on it, was a pretty big deal. The abstract services isolates the data from the specifics. We can mapped many protocols, handle the future, you know, the future proofing. Let's say, so the data objects represent all manner of things. I talked about voltage and current readings and the like but all manner of sensors, control inputs are, you know, it can represented pretty much. You know, anything that we've historically done with mechanical devices and stand alone electronic devices, we've been able to use this, you know, this protocol to address everything that comes down with the name, so you can think of 61850. To some to extent if you look at the naming convention here, it has a lot of similarities to, you know, to SNMP mapping, a MIB, and it's a hierarchical representation of a device; the farther down you go, the more specific you get. So, you know, you know, here, we've got a top level device that may be an addressable device. You know, this is a name for the third instance of a device that does measurements, then, we can get down to the--on a phase voltage. One channel, phase A, it's the absolute value of the magnitude and the floating point representation of it. So, very--just like you drilled down into a tree, an SNMP, you know, type of tree, similar kind of thing here. I hadn't really thought about it before, you know, but, you know, 61850 could, you know, be thought of this SNMP on steroids, I guess, so you offer my naming point of view. All right. Okay. The other element here is the immunity I mentioned earlier, getting all of this to work in a substation environment is a real challenge. A lot of work has gone on to be able to support that. We now have standards in the IEC that define exactly how this equipment needs to behave in this environment, in high electromagnetic interfering environments, things that, you know, that those of us who've, you know, done that, you know, created the hardware before, products for the corporate environment that never really thought of it before. And it's a real challenge to get this stuff to work in these harsh environments. And it's not--and it's harsh in many ways. We've got wide ranges of temperature, but also, as I mentioned, we have a fall that occurs in the system high current flows, this electromagnetic environments are significant. Security, of course, is a major issue in, you know, in supporting these systems. So once we have communications, we have the ability to control large amounts of energy. There are, you know, clearly security concerns, security, you know, from a safety perspective as well as from a disruption to commerce point of view. In the early days, the systems were stand alone, they were barely interconnected, and that the utility communication's world grow up thinking that security by obscurity was the way you achieve security. And it is taken us a very, very long time to, you know, to address that. And I would say that the problem still hasn't solved--the institutional problem of security. We've still run into engineers that would say, "Oh, well, you know, that's, you know, no one knows about that protocol," you know. And is--if any one knows, I think, about security knows that that is, you know, not a way to secure a system. So a lot of work is currently going on to address that. There are several standards that have been developed recently and it's particularly--in this particular standards arena to specifically secure these core standards that are most pervasively deployed; TASE 2/ICCP, the inner control center protocol, 61850. The European versions of DNP3, which is IEC 102 and three and four, another related one, 870 and then DNP3, so we've managed to create some standards to address the security on these relatively new protocols. Some protocols are considered not worth it, Modbus, so it's not being addressed. The utilities, you know, do take seriously their overall security architecture and almost every utility is different because of how they evolved, how their infrastructure evolved, whether they have a fiber optic rings in a, you know, in a nested, you know, SONET ring system or they just got a few microwave point-to-point links. It's all over the map as far as what their physical communication infrastructure is. But regardless of how they've done that, they're applying best practices that are evolving for utility SCADA system protection involving what you would consider as standard practice in an enterprise environment, but with some very, you know, special needs; recognizing the importance of the energy management system to stay in network for which the wrong button pushed on one consol can, you know, block out an entire city or large area. Particular attention has made to ensure that it runs on a completely separate physically isolated network, and most utilities in this case is depicted by the blue lines and the blue ring versus the corporate firewall, in this case, the, you know, the red ring. And with a lot of monitoring in place and mechanisms that allow specific corporate clients to access aspects of the energy management systems, you can do your work but through some very controlled environments. And there are as many versions of this diagram as there are, you know, utilities and some utilities are in multiple versions of that. We--you have a number of guidelines in the industry that are driving that. The one that most, you know, most of you hear about are the NERC CIPs, the North American Reliability Corporation responsible for the critical infrastructure protection standards. But those standards focus more on accountability and who to blame when something goes wrong than to give a whole lot of guidance, you know, on exactly what to do. So other activities are going on to develop things like what--and this is working on here and now on these released [INDISTINCT] providing specific guidelines on how to secure different elements of the smart grid. So, you know, these architectures will evolve common elements, things that work will be identified and deployed elsewhere. But a common theme in these architectures from what we've had out on the field for a very long time now is our critical devices that do the bulk of the switching of energy in where a lot of energy is concentrated in a small place. And if things go wrong, it can result in, you know, loss of life, loss of millions of dollars of equipment that may take a year to replace. And it's all at the substations. The issue is that, you know, these substations oftentimes are shared facilities. They may be a transmission company like a Tennessee Valley Authority, may, you know, have their equipment there, but it's shared with, you know, [INDISTINCT] Coop, you know, that has, you know, two employees and 700 customers and the facility is, you know, accessed, you know, by any number of people. So even in that environment, we have to treat this as if it is an unsecured, you know, facility. We cannot trust any of the devices that are in that system. So we, you know, our "trust no one", you know, motto here. There's been a lot of discussion recently in the--for smart grid and Home Area Networks. Well, how do you, you know, if the Home Area Network device is somehow connected to or interoperating with a utility, you know, the utility network, you know, isn't that a risk? Should we just not do that? Air gapped it, you know, if you will. Well, I mean, you know, we're able to accomplish that with point-of-sale terminals, ATMs, both of which exist in untrusted environments. Anyone can get their hands on one and tow one away if they want to. But we're able to apply the standards and secure the technologies in order to be able to deal with in untrusted end nodes. So what that really bothers me when people talking about, you know, trusting the end node and, you know--and that's the wrong statement. There is no such thing as a trusted, you know, end node, even in this case where the consequences are much more critical than the home area network scenario. So it's a matter having that mindset in designing the security to address it. All right, I'll get off of my security soapbox. A key thing that happened in the early 2000, starting around 2000 and culminating in 2002, was--to help get the utility mindset to start thinking, you know, even further about structured information management was the EPRI IntelliGrid Architecture. Historically in the electric power back--utility back office did a lot of one-off integration. She would have one engineer develop [INDISTINCT] and one team develop software for managing energy markets, another for, let's say, meter reading, another for SCADA. You know, say, someone found that there are some value in gluing these things two together. So you would do a one-off implementation, get the guy next--the cubicle next door who's kid can do a visual basic to write an application. Any confident engineer, you know, or your neighbor's kid can write a VB app to, you know, glue the two systems together but trying to do that in a way that's extensible, manageable, maintainable, and secure is another story. So there's a lot of systems that have been developed where you wrote some code, extended it. Then you had to write a little bit more code to extend it again, glue it together. You're, you know, taking less and less use of the interfaces of these devices to present themselves. And sooner or later, you run into something that you can't bolt on, like security. And what the IntelliGrid folks, you know, project did was to really begin to make the case, evangelize is sort of one of my jobs, to evangelize the utilities to use this kind of approach. To evangelize, you know, getting them to, you know, architect the system from the beginning. Create a--an infrastructure that have security data management built-in. And so we've created a series of standards now to support this concept of standard names for standard things in the back office. We have our Reference Architecture that's been identified to codify all of this within one of the IEC committees that's responsible for telecontrol. So we, you know, we have identified various standards that are, you know, deal with the information encapsulization at the high level and through middleware, web services, a common information model for the enterprise. Then getting into recognizing the various application domains that we have support, and the unique requirements associated with them recognizing that all of these applications require on data acquisition to send service in the field and control--and actuators in the field for which we've identified a variety of standards in our reference model that meet the disparate requirements of these different applications. You know, the utility enterprise is a very large enterprise. It encompasses almost every kind of back office business model that you can think of, you know, in one place. And the requirements for the field device has covered incredible, you know, wide range from, you know, get me the data once a month. And you know if I'm some data because it's accumulated, I can fill in the blanks. That's metering on one end. Or, by gosh, I'd need four-second data, absolutely reliable because I got a--if a few of state estimates make the wrong decision, you know, I'm going to knockout the North East. You know--and so we've got a wide range of requirements. And the very standards of an identified in the reference model addressed those various needs. This is but one reference architecture of several others that are, you know, that are really necessary to visualize the information exchanges in the utility enterprise. So one of the challenges we have in front of us is to try and find a way to come up with a comprehensive reference architecture, or an architecture of architectures to represent all the different information exchanges and business operations that need to occur. A lot of ways to look at this, so based on what utilities are now moving cards, the rest of industries have been doing this for a while. But Service-Oriented Architecture's relatively new to utility enterprise. But we now have standards that are codifying the Service-Oriented Architecture, type of adaptors, and interfaces, codifying some of the enterprise service bus characteristics that are necessary to support this service orientation of the utility enterprise. Translate that to a real example, Southern California Edison is just about to publish, you know, this as their overarching architecture showing some of the elements. They've got their field area network, IP-based connectivity using 621850, 621850 and IP, again, here with their--for protection and control networks, IP being used in many [INDISTINCT] DNP3 over IP, 621850 again having a common services environment that has a security built-in, you know, intrusion detection built-in, intrusion prevention built-in. Being able to recognize and utilize all manner of networks, use WiMAX where necessary, fiber where necessary, being independent of the underlying physical layer manifestations. And then recognizing that we need an enterprise architecture in the control center that can deal with all the different types of application, energy management systems, GMS, outage management, distribution management. I mean there are hundreds and hundreds of different application categories that was--that have to run in a utility environment. So that's, you know, we're starting to get the early adapter utilities to start thinking in terms of architecture, creating a reference architecture that they can build and design to, to identify their well-defined points of interoperability for which specific standards work today. But they can change the amount a new ones in later because they have implemented the very basic concepts of loose coupling in layered systems. So just to, you know, revisit the picture in this, you know, short about of time, I've really only been able to talk about just a tiniest subset of these. Talked a little bit about DNP3, 621850, ICCP, the common information model, again, standard names for standard things at the enterprise level, we're using things like soap and web services, and restful implementations as well at the enterprise level, the common information model with XML encapsulizations of data to run our various business applications. All of these, we've been doing this for, you know, 10 plus--you know, 10 plus years in one form or another. And there's been a continuous evolution more exponential here recently because we're starting to get, you know, a lot more uptake on it. So, this work is been as pretty mature. We now are able to be totally agnostic of what's in the wide area network. We've got--all of these technologies are being used because we're mainly using IP, we can use whatever is cost effective in our particular area. The--you know, the Wild West still exist right now in the utility mystery in the field area network. You know, that--the last mile, if you will, or last several miles to the end of [INDISTINCT]. This is still the area where it's driven by, you know, vendors in a very vertical market where it's primarily proprietary. The well point--the fine points of interoperability, you know, are here and--you know, and here. We don't have MacFire layer interoperability or even another layer or so above that in this arena. So that's something that people are working on. The value just hasn't surfaced yet completely to--you know, to force that to happen any quicker. You know, we also have evolving standards that are still being worked on, on the side about of motive engineers for the electric vehicle. Got the connector nailed. Some issues with the height--with the fast-charging connector, because SAE neglected to pay a whole lot of attention to little things like UL and some other standards they should have. But that'll be addressed. But the big problem here is these guys don't have a whole lot of domain expertise in understanding what's necessary for demand response and other, you know, managing that load and doing smart charging. So we've got a lot of work to do here yet. We've got a lot of proprietary things happening in the asset management area. And so--you know, a lot of asset management systems out there, but they all rolled their own thing, you know, right now. So there are some work to be done there. In the consumer side, we've got, you know, dozens of protocols. We have an interesting email exchange going on right now where I was able to relate that I've got probably, you know. I've got all of these and a dozen more running in my home automation system, all using gateways. It's very expensive to integrate that way. But, you know, we can finally--you know, we can make it work. But there's work going on to try and at least agree on the information model. Again, if we can get to agree on, you know, the standard names for standard things aspect that map it to any technology. That will be a good thing. And on the distribute resource side. Again, we're not--we don't have a lot of--we have some standards that are based on 621850 to represent the data, but we're missing some electrical standards to ensure that the safety of the network is provided for when we have pervasive deployment of these things. We need new protocols that can interact between a photovoltaic panel that's generating energy back into the system, and the protective devices I talked about that are on the substations. So they know when we're generating. And if something happens on the line that, you know, we have a--car knocks down a pole and we've isolated the system that we are able--that the line worker knows that that part of the line is in fact, you know, dead so they can work on it, you know, safely. So a lot of things have to be done from a protocol point of view there. And we fully expect, you know, TCP/IP, Internet-related protocols will be involved in coming up of that solution. So a lot of work yet--you know, yet to do. And this--again, this is also just scratching the surface of the--probably the most visible, you know, standards that are out there. But again, you know, IP is everywhere. I mean, you know, IP is involved with almost all of these. Not all of them, but almost all of them. Okay. So why did, you know--so some people asked me, well, you know, why--you know, why do we have this mishmash of things that is out there? And how did they evolve the way they did? And why is the uptake on standard protocols and the like, you know--you know, taking so long. And, you know, the key is really looking at the drivers for utility automation. You know the bulk of the decisions are all--and primarily, the first, second, third, fourth, maybe fifth thing are all safety; line worker safety, public safety, and safety of the equipment. The equipment that we're dealing with here is the massive amounts of energy are flowing through it. And some of the equipment literally takes a year, or year and a half to build. In my early engineering career, I worked for Cooper Power Systems, McGraw-Edison where we built the 500kV and 765kV transformers. From beginning and to end, that takes a year and a half to build one, okay? And you know, so you know, you go to extraordinarily lengths to prevent from having one of t hose fail. There's, you know, only a few spares in the entire country. You don't want the, you know. So there is a--and it's--a lot of attention has to be paid to that. Reliability, just keeping the lights on. You know, the US has probably the, you know, the most reliable electric power in the world. As much as we talked about aging infrastructure and equated nature of things, it's still a pretty reliable system, you know, because of the distributed decision-making that it's done that the mechanical relays as well as the new ones. But--so the next, you know, object is for--is optimized for reliability. So a lot of redundancy, a lot of cost, being, you know, the tight coupling was considered as a way of managing some of that reliability. The security--in the utility world, you hear people talk about security. Security has a different meaning to a utility engineer. It means really the ability to meet demand and be able to meet the future demand is what security means to utility engineer for--you know, us information geeks, we think of cyber security. But we have to be able to address, you know, both. Stability, being able to deal with the load/generation balance in real time. These protocols are designed to, you know, recognize that, you know, we don't--without a major breakthrough on bulk storage, in general, the bulk of the utility system. We deliver energy in real time. You know, millisecond-by-millisecond trying to maintain that balance. A lot of the decisions are made. A lot of requirements are driven by that real time nature of things. Only when you really get down to--you know, getting near the end of the list. You know, is it for convenience and for--you know, for making it easy for end users to, you know, be aware of their energy consumption, and to make it easy for vendors to participate, you know, in a market. You know, it--you know, that gets lower on the list. I put regulatory at the bottom just because I hate regulation. But a lot of things are done because the utility has to. You know, there's a fundamental concept of the obligation to serve universal service. So things that you might say, but why don't, you know--you know, why can't we just apply this technology and then automate it and just require that we have a gateway in the home, and we've got PCTs, and then we're done. Well, you know, we can't do that and mandate it across all socioeconomic classes and put requirements on the end user, you know, to do that. So we've--one of the reasons utilities, you know, end up getting inside the home area network is because of that universal service, you know, requirement. It's not as if they want to be able to get into the home area network and provide thermostats to an end consumer. But in many cases, they have a regulatory driver to be able to do that. So there's a lot of unique drivers that exist for--you know, for utility. You know, one thing that wanted to, you know, get across is when we have a failure in a router or in the Internet, you know, we inconvenience someone, I can't run that Google Search right now. I'll have to do it, you know, a little bit when the network comes back up, may lose a transaction or two. Some of it loses some money. But when we're dealing with the--when we're dealing with the electric power infrastructure, you know, we have physical manifestations of these errors and issues that we've got to--that we've got to be able to deal with. So when things go wrong, we, you know, we cause lots of noise, lots of heat, you know, things that have real physical manifestation as well as cost associated with them. >> Really, really bad. >> GUNTHER: This particular failure [INDISTINCT] is continuing to burn because those protective devices I mentioned failed to communication with the neighboring device. In this case, because of disconnects [INDISTINCT] the second contingency of streaming from that, that communication didn't occur so the backup protection fail and resulted in the tertiary protection having to [INDISTINCT] and finally clear [INDISTINCT]. >> There it goes. >>There goes the car. >> GUNTHER: And there it goes. >> That was beautiful. >> GUNTHER: So--and, you know, part of their--you know, the other aspect of things, you know, we prefer to do things remotely when we add communication so we can prevent things like this from happening. Again, you know, a failure of a mechanical device that's being operated locally can have catastrophic effects on the--on human life as well. He is okay. But--here, hear the car alarm's going off in the background. This is a relatively small fault, but a significant electromagnetic pulse disrupting equipment in area, in this case, a car. This kind of thing happening in front of your house who doesn't exactly give you a warm fuzzy feeling, and you know, so--you know, again, when the failure occurs, our protection doesn't work, communication goes down, you know, things like that, you know, happen. [INDISTINCT] on a small-scale, there in a large scale. That line probably was out for, you know, a couple of weeks replacing all the insulators. Here's a--you know, here's another example where the protection inside the facility failed. It never [INDISTINCT]. And in this particular case, we have a rather spectacular failure of a large transformer. And that--you know, that took--you know, that took many, many, many, many months to replace that. So when we have a failure in the communications network, in the utility infrastructure, there's real consequences. And that's why, you know, we--you know-- every industry says they're special and they're different. But, you know, there's not too many that, you know, have that--those kinds of consequences. And that's what we--you know, why we pay a lot of attention to very carefully and deliberately adopting new technologies moving forward. That's it. Any questions? Go for it. >> So, you mentioned IP a lot. >> GUNTHER: Yeah. >> IP is one of those protocols that has a built-in number scheme, which means that essentially there's a bunch of [INDISTINCT] you know, the [INDISTINCT] all these opportunities of things to go wrong. So I'm wondering have you actually considered prodding to people who develop [INDISTINCT]. >> GUNTHER: [INDISTINCT]. >> [INDISTINCT] but have you considered prodding them so that you know that means our [INDISTINCT] in order to go from the end node all the way to somewhere else. >> GUNTHER: There's a lot of folks in the utility industry who point to, you know, TCP/IP and say, Man, we could do a lot, you know, better. But will all of the failings and known issues and vulnerabilities, you know, we've managed to engineer systems around them to be able to address and deal with a lot of those issues. Yeah, absolutely, we'd like to solve some of those very fundamental, you know, problems. The--you know, the issue is, well, you know, who? How much money? Is it worth it? And, you know, the utility industry learned a long time ago, we don't want to, you know, do another one off. We want to use what's pervasively deployed. So pervasively deployed infrastructure that has known flaws that we have worked around as far--for is better than a new approach that fixed the known ones. We have a whole host of unknown one. You know, issues and flaws. So that's sort of the trade off. >> [INDISTINCT] >> GUNTHER: [INDISTINCT] talk about that. Yes? >> So, during the California energy crisis, on of the big issues was this keep the lights on phrase, because the government couldn't distinguish between who really like the air condition and somebody running a [INDISTINCT]. Is there anything in this that leads us to be more [INDISTINCT]? >> GUNTHER: Yes, I mean we have--with the newer technology being deployed will give us that--the capability of doing that by, you know, most of the utilities have a good enough customer information system. They know where critical loads are. But up until recently [INDISTINCT] smart meters with integrated disconnect, the best we could do is disconnect all the load of the substations. So we go into rotating blackouts. We got to shut a whole substation down, meaning all theaters, meaning fire stations, police stations, people on life-support, as well as all the folks who are just watching their plasma TVs, that was the best granularity we have. With smart meters with integrated disconnect, we can do that. But of course, we have a large faction of folks who, you know, think the fact that, you know, the ability for a utility, for some reason, they relate the utility to government. But being able to first of all--determine what your usage is and whether it's critical or not, and then be able to turn it off is a big brother issue. So we have. you know, political and social problems to solve to be able to allow us to use the technology that's presenting itself. >> These smart meters, are they per house or per device? >> GUNTHER: Per house. Probably, a per house. And so, you know, basically on a--you know, household-by-household basis instead of going in, shutting down the entire sub--you know, the entire substation, we could go and send a signal to turn off the disconnects on all the houses that are not--you know, have a medical exemption, you know, turn off all the--we can keep all the grocery stores, the convenience stores in the neighborhood on, we can keep the police station, fire station, you know, other elements selectively, you know, all energized. And accomplish our object for reducing load for peak management. >> So, my thinking was, you know, just looking at my house, there's very little power that it can gets consumed in at any one time, except for the dryer and the stove. So, I prefer a system that just came in. And its first step was to turn off everybody's dryer and their stove... >> GUNTHER: That's the thing--that's exactly their next steps. See, the first step, you know, just takes the--what has classically been within the scope of the utility, the service to the house, you know--and let them control that, which has high benefit. The next step is allowing the utility to be able to send either a price signal or an emergency load control signal saying, you know, for any appliance that knows about--that is able to hear me, go ahead and defer the defrost cycle. You know, go ahead and, you know--shut the dryer to tumble so we don't mess up the clothes. But, you know--but, you know, don't use--don't turn the heating element on. We have the ability to do that. But again, there are social and political, you know, questions as far as how much into the home should the utility, you know, go. Where does its responsibility end? You know, should it just provide the information but not be concerned about how the information's consumed internally? Well, you know, you could argue if they don't have some visibility into there--you know, and not knowing what's responding or not, they can't manage the system. It's a lot of social and political issues to be addressed on that one too. Yeah. >> And your diagram of--you know, the end of a smart grid showed the home and the car connecting up through the utility's private network that you see it is also equally viable for those--a home and a car connected through regular Internet connection? >> GUNTHER: Absolutely. The field area network is generic that that field area network could be one or more private networks by third-party aggregator, a utility, the Internet. You know, it doesn't really matter. And, you know, and how that's managed depends on if that kind of consumer is subscribed to a special utility rate program. In the--in which the utility needs to be assured that the signal has gotten there and they are meeting the terms of the contract, you know, for having a special rate for responding to a load control segment. In the ultimate, if we had nothing but pure real time prices, so we delivered pure prices to devices, you don't have to create artificial programs that require a two-way acknowledgement of whether, you know, did you do this or not. Well, you know, if you didn't respond to the price and stayed on anyhow, you have more power through that you paid through. But if it's a direct load control program, you want to make sure--the utility wants to make sure that they really did respond, didn't put foil around the radio and prevented the, you know, the device from being turned off. So, again, you got, you know, some social issues that have to be addressed here on rate reform and the like. Yeah. >> Before [INDISTINCT]. How far away do you think [INDISTINCT] realistic solution to this problem is [INDISTINCT] solutions to this problem that you're talking about, once when utilities switch prices and then, you know, people can opt in and act voluntarily [INDISTINCT] utilities been [INDISTINCT] devices in your home and [INDISTINCT] it can make everything a lot more efficient. And potentially there's economic incentive if they set up the right kind of system with the energy market and they regulate maybe people can opt in or take in profits from turning off air conditioner throughout the day because it's too expensive and so forth. >> GUNTHER: Right. >> And give that power back to the grid and paid for it maybe even. >> GUNTHER: Well, technology is not the problem. We've got a lot of ways to do it. Yeah, so there's--we've been implementing demand response. You know, I implemented first demand response of a system, you know, like, 30 years ago. You know, so the concept has been around for a long time. What's making it more feasible now is the technology is cheaper, ubiquity of communications, multiple ways to do it. We're creating architectures that realized that, you know, just codify the information; don't tell me how. But, you know, the rate reform is a key element to that to make sure that we can, you know, have a mechanism where the true cost of energy is actually represented. So there's an incentive, a business incentive, you know, to do that. And that just takes time. And we have a situation in the US where every state has different, you know, regulation. It makes it difficult for a national or international market to develop, you know, to produce cheap equipment. So there's a lot of barriers and most of them were not technical. I mean we've got plenty of good technical problems to solve to, but they're less daunting than some of the--some of the rate and regulatory issues, >> So I guess in the [INDISTINCT] right? Like if lights aren't actually going out in my home, I can run my air conditioner the whole day, so while we'd like to change the [INDISTINCT] load and it will be cheaper for the utility so they take more money. It's not going to benefit [INDISTINCT] at this point. >> GUNTHER: Well it's... >> How far away are we from really caring about...? >> Well, the business case is very difficult. In California, it's a lot easier. Your rates are high for all sorts of reasons, your capacity is constrained, because in California, other parts of the world too, but in California in particular, you know, there is, it's difficult to get approval to build anything new. Nimby is a--and banana, you build absolutely, you know, nothing, anywhere, anytime, you know, are huge problems. But people, but the energy is--the demand is still increasing. California has the lowest energy demand than the--per capita in the country, but it's still going up. And if you're not allowed to build anything, you know, it--that makes rates go up, and we have to use this technology to manage, you know, that peak loads. So there's a lot of drivers. So if you can't make it here, we're not going to be able to make it work anywhere. Where I'm from in Tennessee, it'd be a long time before I see that. You know, I pay six cents per kilowatt-hour. We have no congest in problems. So, I mean, even if, you know, even if we did have, you know, a five to one price there. Let's say of a buck or two a day. >> Why is it a better solution? Like, why build a smart grid? Why just not build a three nuclear power plant in the middle of the states somewhere and increase the generation capacity? >> GUNTHER: Okay. No, it's--like I said these are not technical problems. You know, these are socio, you know, political economic, you know, issues more than anything else. Good question. >> I mean, what about your [INDISTINCT], we think it would [INDISTINCT] have some program where you can [INDISTINCT] real time updates. >> GUNTHER: Yup. >> You know, if somebody... >> Because you can't. >> ...use a hairdryer or not. So I actually picked the [INDISTINCT] people who hack into these networks and if you build them, they will hack them. So people will then go, what you're doing? And if you're home, and it's a--I think it is a kind of bigger problem. And so I feel like people are kid of right to resist that psychology and [INDISTINCT] real benefit. >> GUNTHER: Did you hit the, you know, the nail on the head with a lot of these, you know, a lot of these questions. You know, the core for many elements and aspects of smart grid on the end-consumer is consumer empowerment; being--providing information so that you can make more informed decisions. It's really not much more complicated than math. We have the technology to do it. There's way easier ways technologically for, you know, for adversaries to find out your, you know, you're living patterns and the like. And there are ways like we managed to secure ATMs and Point of Sale. We can do a pretty good job at securing it, not going to prevent every, you know, issue from occurring. But, you know, those are--that's a manner of managing your risk, but, you know, is it work it to provide the information to consumer to save, you know some amount of money. And, you know, what we found is that end-consumers don't want to be energy managers. After the first couple of days that nifty display and watching your energy use on your pc, the novelty wears off pretty quickly. You know, but if you do have pure prices to devices and the devices are smart enough just to simply listen and when they can make an intelligent decision and you save a few bucks because you don't have to do anything special, that has some value. >> Exactly what I mean. I could install something in my home for a hundred bucks and have to deal with the power company where I pay $20 less a month, and in return it pays for itself, but it also [INDISTINCT] ability to mange the power of load that my home draws at certain time. So I mean I think people would install that, but--I know it's doing it, you're right that technologies existed forever. I was just trying to figure out... >> GUNTHER: In the regulated environment, I mean, it's not as if you're saving the utility, you know, any money. You know, sooner or later any costs, sooner or later, any savings sooner or later are reflected in the rates. So, you know, just give in your time window for the rate cases, you know, the consumer pays no matter what. You know, what we've been able to demonstrate that smart grid is able to do, it makes it sac less. In other words, you know, basically, energy supplies continue to, you know, cost more, more people are using so demand is going up; we can't build anything. You know, that has, you know, price pressure goes up. All that smart grid can do is make that sac less. It can make the prices not increase as fast. Now, a lot of folks, you know, don't listen. You know, lots would say, don't [INDISTINCT] want to say that. You know, it's--everyone expects smart grid to make things, you know, go dramatically down in price. Well, for some individuals who are making, you know, stupid decisions because they didn't have any information, you know, probably can save a whole lot of money. But in the grand scheme of things, I fully expect energy cost to continue increase just with smart grid and this technology they'll go up a lot less fast. >> Question on the other side, which is the generation side, and I'm concerned about [INDISTINCT] and then also on generation this is probably also [INDISTINCT] in competing all that in... >> We call it renew--we call it variable generation and renewable variable in a generation. You know, some of it is very, you know, very unpredictable. For example, we know solar panels are not going to produce energy at it very well-defined points in the day, so, you know, there's some things you can do. Actually, in bulk wind with the weather forecasting, with the tightly interconnected grid, it's more deterministic than you would think. Even with bad forecasts, you know, for one winter but I can't predict it, but, you know, in the Midwest all are interconnected together, I got a pretty good idea of--and can predict hour by hour pretty well what it's going to do. The thing is, it turns out load is way more variable and unpredictable than some of the generation sources are. And, you know, so we just publish a study called the Eastern Wind Interconnect Transmission Study, EWITS that demonstrates what we need to do in the Eastern U.S. to get 20%, 25% wind integration. And, you know, some new transmission is needed in order to couple things together tightly enough to manage that variability. Storage, obviously, can come into play, but a lot of folks talk about, "Let's locate, you know, battery storage, you know, at each of this variable generators." Well, you know, that, you know, that doesn't work all that well because the bulk of the variability is in the load. And it turns out that, you know, storage is always good but it makes more sense to analyze it as a system and figure out where the optimum place is to put that, you know, the storage, you know, where it can most cost effectively be used. So that's something that we have to look at as, you know, how do we architect the system to support thermal storage, electrical storage in appropriate places in the grid to handle higher--higher and higher penetrations of those types of generation, but it's not an easy solution. >> You know, helping it out in the load side, are you willing to have a dial-on like dryer that says, you know, "Start whenever the power company says it's a good time to start." >> GUNTHER: And that's exactly we're talking about. We're talking about the, you know, smart appliances, smart devices. You know, that's some of the work going on, the standard said, you know, organizations here today is to figure out what is the information that we need to get to an appliance to make that kind of a decision. You know, a dryer manufacturer or washer manufacturer does not want to have an external devices turning something on and off because you got a consumer, you know, consumer interface problem there with, you know, destroying clothes. So tell me when the price is high or tell me when there's a grid event and let it make the best decision on do an orderly shut down or just keep things tumbling but turn the element off. And, you know, that's what we're working towards to develop the standards on how to encapsulate the information so that those appliances can be built. Because you don't want to have to program it, we all know what happened with, you know, with the VCR and your program or thermostat, you know. The thermostat's on hold all the time, okay. I mean, you know, a survey, you know, demonstrated maybe only 20% or less 15% were actually being programmed. And you know, you got to get the person out of the loop. >> When they built the system like this for the utilities is kind of give [INDISTINCT] messages about the state of the grid and the price of power and so on. Do you think they would deliver this over some kind of broadband on the power lines themselves or do you think they would use the Internet? >> GUNTHER: It will be delivered on whatever makes sense in that particular area and it's cost effective and can be, you know, relied upon to be secure. Initially, because of the investment that's being made for smart meter--we need smart meters to reduce, you know, to--in order to--we need to replace the meters to capture the interval data so that we can have innovative pricing. And it turns out that by putting the smart meters in place and putting that integrated disconnect, we get a whole bunch of value by being able to do that selective rotating blackout, to be able to save a pile of money on, you know, new homes and in move in, move outs and the like. And coming along with that, there's a communication infrastructure that allows us to deliver some data that way. So early on, demand response signals will likely be delivered over the utility network just because it's available and an obvious thing to do, but you don't have to. You don't need advanced metering infrastructure, smart meters to deliver prices to devices. There are other means through, you know, through other external networks broadband to deliver that. The thing is you got to find a way to do it in a way that you can trust the information. >> Sir, [INDISTINCT] at the end, right? If I agree to turn off my air conditioner and make a payment, then I might need to turn off my air conditioner. >> GUNTHER: Right. And that's the difference between the program approach, which is that, you know, that's--you've struck a deal. I'll do this, you know, if you do that, and now it's got to be audited. That's what we call--that's the energy, you know, a demand response program. If instead, you simply, you know, there were prices for, you know, for, you know, hourly prices. If you choose to use energy, you know, when the price was high, so be it, we don't care. There's no contract, you know, you know, we're charging you for the cost of energy. You don't have a program, so you don't have to verify it. You know, we need to measure your energy use, but we don't have to be aware of all the devices and how they're behaving and auditing your behavior. The prices get adjusted, you know, to influence the behavior and overtime, you figure out what those prices need to be and let the market decide. We're long way away from that though. >> [INDISTINCT] written about it or discuss the protocols, so I don't know. It's kind of like cyber paths where you can probabilistically enforce the rules across on some of the people such that any individual's privacy is protected, but at any given time, they might not offer up some proof and if they fail then they pay a high penalty? >> GUNTHER: I'm sure--I mean, I'm not aware of any specific ones that are going on. There's one, you know, but I'm sure there are some folks looking at that. You know, fundamentally from the privacy, you know, point of view, the utility always has and still does, you know, if they want to go and look in the data, you know, when you, you know, when you're, you know, you're using energy or not. You know, with the smart meter, they simply, you know, know it to a--or can know it to a higher degree. And it's no different than your credit card company being aware of every transaction, you know, that you make if they choose to go and look at it. So there's a stewardship of the data issue that's required to support that privacy. >> There's also, at least, a sort of [INDISTINCT] in some respects [INDISTINCT] systems that are collecting data and putting it in a central repository, it would then be searched in mined by computer program. >> GUNTHER: That's right. >> That's actually information you can discover though that you couldn't before. >> GUNTHER: Absolutely. >> And I challenge the meter readers to collect enough data to learn the same kinds of facts. >> GUNTHER: There's huge value in those datasets that, you know, that could be mined, but this is where policy is required. This is where as already has exists in the banking industry especially the credit card industry, you know, to what level they're allowed to mine that data for marketing purposes and provide that, you know, to other vendors. That's something that probably--I'm not a big, you know, regulation guy, but that's probably something that makes sense to have some rules on. And I fully expect to see that in the, you know, in the utility world as we have that more granular data. All right. I'm a little bit overtime there, but thank you, all. I appreciate it.