Enchant (software)

Transcription

JEREMY WALKER: Hi, my name's Jeremy Walker. I'm a developer advocate here at Google. Welcome back to our series on the Three Flaws of Software Design with Max Kanat-Alexander. This is on the third flaw, so why don't we just jump right into it? MAX KANAT-ALEXANDER: Sure. So this is the third of our Three Flaws of Software Design. And this third one is, being too generic. Often, this is called over-engineering. It's an attempt to accommodate every future requirement now even though you can't actually know all the future requirements now. This is a flaw that senior developers frequently fall into. So junior developers often fall into our previous flaw, which is writing code that is too rigid or that doesn't have enough design. But senior developers have done that, and they've been bitten by it. They've been bitten by change. And they want to try to accommodate every possible change now. And this sounds a little bit like our first flaw, which was writing code that you don't need. And it's sort of a specialized case of that. But there's a little bit more to know about it, so I pulled it out as its own flaw. The problem that it has is that you've put in a ton of effort for very little value. Basically, like when you've design something that's very, very generic, you're not delivering the value of the code that you wrote to the user. The amount of work you did, and the amount of value that the user actually gets out of it, are not proportional to each other. Because you've done a lot of work that doesn't actually deliver any value to the user. Now, sometimes that's OK, because you're doing good design. But over-engineering, or being too generic, trying to meet requirements, future requirements, that you don't know about yet is not good design. Developers do this all the time by trying to be good. They've been taught certain practices, and they want to follow these practices. But they don't understand that there's certain times when you don't need to follow those practices. People want to catch every exception, but there are some exceptions that don't need to be caught that you really should just let bubble up to the surface, because otherwise you're hiding errors. And it actually makes it harder to debug the system. When you try to handle input that you never get-- when we did our code review for the first flaw, you saw some of that. Handling situations you're never in, like trying to make thread safe code that doesn't need to be thread safe-- I've done this. It's a nightmare. You're adding locks everywhere, and it really just adds a week of work to a project that should have taken a day. Or dependency injection, when there's only one choice that there could ever be-- this is probably most common in the Java world, because dependency injection is very popular. And I'm not saying that dependency injection is bad. But I'm saying that dependency injection when you don't need it is bad. You shouldn't be doing speculative dependency injection, basically. Like, you shouldn't be guessing, oh, well, in the future, we might need to do dependency injection, so I should just do it now. In general, these can all be summed up as, one day, we might need to do foo so I should do foo now. Let's give an example. So let's say that I want to write an HTTP server that communicates over a TCP socket. And I have a bunch of time on my hands. And I think that I am the world's best developer, and I just want to do an awesome, awesome job on this. So what I do is, I'm going to be, like, OK, I'm going to really, really design this, super design this, but I'm going to over-engineer it. What I'm going to do is I'm going to have multiple-- you know, there might be other types of sockets, other than TCP sockets in the future. So we have a socket, and a socket type, and a socket factory to make our sockets, and then have a TCP socket. So by the way, any time you see a factory they can only produce one object, that is something to be suspect of. Or we want to say, oh, but maybe we'll support other protocols, other than HTTP. So we should have an abstract request class, and then a type for the request, and then a request factory, and an HTTP request, and the same thing for the responses. And then, that means also that the request has to understand the response it creates and the socket type that it came in on. So there's going to be all this complexity in implementing, sort of, generic socket handling and generic request handling in your generic response, and also in your generic request. And then, you know what, everybody likes configuration files. So instead of writing code to make our HTTP server behave a different way, let's write configuration files, not because we need them, just because we think we might need them in the future. And specifically, let's make it so that the sockets can configure themselves. Because we want to have different configurations of sockets. I don't know why. But, you know, you just want to configure them to have different time outs, or something. And so, we need to be able to read socket types from files, basically, not like a serialization format, but just a configuration. And you know, because we like being efficient, in production, we want to have a binary version for this configuration, but binary configurations are hard to debug. So in development, we want to have a text version of this configuration file. And then, in production, we probably want to store this configuration file in a database, but in development, we might want to store it in the file system. So we should have different ways to store this HTTP server. So, what's wrong with this picture? It required 15 classes. And HTTP is a complex protocol to implement, particularly if you also have to implement a lot of TCP underneath it. So I would say that all that complexity is going to make it take one person-year to do this and maybe 30,000 lines of code. And also, you're going to have to write way, way more tests than you would have to, otherwise, because you're going to have all these generic situations that you have to make sure are actually valid. But, let's say that we did this more simply. What's the non-generic implementation? The non-generic implementation is three classes-- socket, request, and response. How long is that going to take? I don't know, maybe two to three person-months, maybe even less if you don't implement the entire HTTP spec. And the reason it's going to take that long is because HTTP is genuinely a complex thing to implement. But the work you'll be doing is all work that actually delivers value to the user. Those two to three months are all valuable time spent working, as opposed to the other eight or nine months that would have been spent making this thing generic, which, ultimately, the product is identical. The product that you're delivering is identical. I'm not encouraging bad design and saying you should hack things out quickly. What I'm saying is, I'm discouraging over-design. And the rule that you can apply to get out of this flaw, to resolve this flaw, is be only as generic as you need to be right now. And how generic is that? Well, you need to be generic enough so that you never duplicate code, so that your design is simple, and so that your system is stable. So my rule for duplication is that two is too many. Once I want to basically copy and paste a piece of code, ever, even once, then I start to think, OK, how can I have a super class? Or how can I have a utility function, or something like that? You want to keep it simple, so that it's understandable. So sometimes you want to abstract things out of the classes, because otherwise it would be very difficult to understand the code. Or you want to abstract thing out of the functions, because it makes it easier to understand the code. And you want the system to be stable. So sometimes there's a certain amount of complexity in handling error conditions, and you need to handle those error conditions, even if you haven't seen them already in production. The fact that somebody sends you a bad HTTP request should not be a security vulnerability. So you might have to handle that. And that sort of wraps it up for our third flaw. Any questions? JEREMY WALKER: Yeah, so again, just like the other videos, you had a great example in the book-- I think it was related to email-- that I was hoping you would cover. MAX KANAT-ALEXANDER: Sure, so this was actually another Bugzilla example. So what we wanted to do was every time that you change a bug, Bugzilla sends out a bunch of emails. And this is very slow, because Bugzilla used to do it synchronously. It would actually update the bug, then send an email. And if you were updating a bunch of bugs all at once, it would update the first bug, send the first email, update the next bug, send the next email, update the next bug, send the next email. And email sending is actually very slow. Like talking to Send Mail, and actually getting it to acknowledge that the email has been queued, actually, it can take a couple seconds. So if you were changing 1,000 bugs, literally, people would just open a browser tab and leave it there all day. JEREMY WALKER: Oh, wow. MAX KANAT-ALEXANDER: Yeah. And so, one of our big feature requests was, can we do this asynchronously? And so we found a little demon that could do it for us, a little queuing demon, that could do it for us. And it could store little blobs in the database that you could read back out and send an email. So basically, there was an asynchronous demon that sat alongside and would just pull all the database for emails that needed to be sent. And we had this all spec'd out. And then, the developer who actually came around to implement it said, well, maybe people are going to want to use some other queuing system. Maybe they have their own queuing system. And they don't want to use our queuing system. So he made the whole thing able to have plug-ins, basically, to have plug-ins for other queue management systems, other than the queue management system that we had chosen. And even in the code review, I was like, I can't understand this code. I can't understand this change. I can't review this change. And eventually, he got called off to do other work, and another developer took it over. And that developer ripped out all of the generic bits and the change became 100 lines of code or less, maybe like 75 lines of code, to implement this super-critical feature. And I immediately approved it, basically. I was like, this is really easy. It's going to go in. And the fact of the matter is that in the three years after that that I worked on the Bugzilla project, nobody ever requested to use another queuing system. And if they had, they were probably bright enough developers to write custom code themselves, because this was an open source project. So it ultimately ended up not being necessary and way, way simpler just to have non-generic code. JEREMY WALKER: Wow, great. Well, that covers our third flaw in the series. We have one more video, where we're going to talk a little bit more about some stuff and some wrap up materials. So make sure you tune in for that. Thank you.