The catch-22, of course, is that at some point, MySQL was the "new thing" to Diane, so how did she learn it in the first place?
The answer of course is that judgement is less about making decisions upon familiarity per se and more about knowing when unfamiliar technology seems to provide good trade offs given what is known about it. For example, PostgreSQL may be unfamiliar, but choosing it is a wildly different type of decision than choosing another less mature unfamiliar data store, since at a high level PostgreSQL is known to be a well-understood, mature technology that may provide significant leverage over MySQL in certain scenarios. Being able to understand the various dimensions of technology choices and make good ones with limited information and experience is what makes good judgement, not just an outright aversion to unfamiliar things.
2nd dimension: ecosystem of the tool (developers/libraries/support/etc)
3rd dimension: developer's ability to use the tool (time to learn, current ability, etc)
If you think on factors going from 0 to 10, MySql could be [4, 7, 8] and Postgres could be [7, 7, 4] for that developer.
Of course the 3rd factor depends on the developer (and on the time available), for example I know Go is very powerful, but I wouldn't use it, unless the available options for me are not good enough for a given problem (so the time to learn compensates for the loss that using other technologies would incur)
I think another important dimension is the suitability of the tool to the technical culture and composition of the organization. For example statically typed java-ish languages tend to be more attractive if you have many disconnected groups contributing to a code base, and if I'm being cynical, overall less average skilled programmers just hacking out features. Whereas a small team of top-tier developers are more well positioned to leverage the benefits of more fancy tools and languages.
Diane most likely chose MySQL because it was the most popular open-source database. Popularity is often reason enough to choose a particular tool, because it means that that tool is more likely to be supported by other good tools.
"Popularity is often reason enough to choose a particular tool"
Widely available, many eyes searching for (and patching) security bugs, many people can understand and debug it, probably the most documentation out there, its the simplest possible overall way to do it, she's not exactly doing anything new, so using a tried and true successful solution is good business and sound engineering practice.
So, the cub scouts are building birdhouses this year, again. Last year we used hammers. I found a nailgun recently invented in the black forest region of Germany which shoots a special birdhouse domain-specific-nail thats a precise metric size specialized for this kind of wood and building birdhouses, unlike local nailguns which use an air compressor or .22 blanks, this one uses compressed unicorn gas, which everyone who's cool knows it doesn't smell, and its brand new and I'm the only guy this side of the pond who knows anything about this new nailgun and the only guy who can use it or maintain it, and it cuts hammering time down from 5 minutes per birdhouse to 5 milliseconds even though theres no business case to care. Oh no wait I'll just bring some hammers again this year. But old fashioned obsolete hammers don't scale to 10B units! Oh that's OK the school gym where we have pack meetings only holds 500 and usually only about 20 scouts show up anyway and the countries population is only 400M, so I'm all good. And given other business constraints the scout pack is permanently constrained to grow slower than I can buy more hammers. Yeah, looks like a job for a hammer, its hammer time... Would I roof a house with a hammer, well no, but I'm not roofing a house so I don't much care.
Personally, as a dev, I push myself a little with each new project. I've been using Sinatra for all my web projects, with this latest one I took the jump to Rails. Otherwise the project fell squarely within my realm of knowledge, as a previous project showed me how the Facebook API works.
The time before that I tried to incorporate good OO style.
Next time I think I want to incorporate testing. Or maybe I won't feel that the next one is the right time to incorporate it, and instead I'll pick up a new Javascript library.
As devs, we have a remarkable ability to shape a project by choosing which technologies to use. So long as we exercise good judgment, we can learn new things on the job and not have to bet the farm.
As for MySQL / PostgreSQL, that decision probably didn't matter all that much for the given problem. Abstracted over with Sequel, they're pretty much interchangeable until you get to heavy, industrial-scale use. A lot of people can go an entire career without needing to figure out the use case for each.
I think that asking why a programmer knows any piece of knowledge is a very good glimpse into how competent she is. I don't care that somebody likes Git more than SVN but I do care why (on a technical level) somebody likes Git more than SVN.
This is a problem because writing working, maintainable code that solves the given problem rather than an imagined problem should be the trait of a competent developer.
I had the pleasure of going on a course run by a German developer recently, and his brutal focus on quality and maintainability made me rethink what we class as a good developer.
Brutally in the article Jack is not a rockstar - Jack is incompetent. A great developer solves complex problems in a way that is maintainable - the problem is that most problems aren't that complex. Dianne however didn't do anything that made her "great" - any developer who can't do the kind of engineering practices she did should be fired.
Hacking on something in your spare time is completely different, but as soon as you want to work on something that others will work on as well you need to be into that engineer mindset. It's one of the things that impresses me about Patrick McKenzie's writing, and his experience of learning that the hard way in Japan.
Maybe we should be asking why the things we make need to be maintainable. Why can't I just make program module x to input a and output b and never touch it again.
I believe modular code is the future and the sooner we get there the better.
Here's how I read your comment: why do we worry so much about making things maintainable - why don't we just make things maintainable? Building a system out of small modules is what makes it maintainable.
Because x(a) -> b might need to be understood by everyone working with your module at some point. What if you leave the company and someone inherits your module? What if they decide to change their codebase and need to rewrite your module?
Writing modular code doesn't magically give you maintainable code either. It needs documentation, proper use of design patterns, and needs to be readable.
What do you do when bugs show up in that module? rewrite it from scratch, introducing god knows how many new bugs? Or only address bugs when a rewrite of the entire module is necessary?
Joe asked the same questions and then wrote a collection of CGI programs/scripts in one of those languages which start with the letter P. It was dull and uninteresting. There were a half-dozen commands the devices needed to call, so he hand-wrote the SQL, making sure to use the appropriate techniques to insert parameters instead of "string building" and opening the client to injection attacks. His code thus resembled very small filters: HTTP requests mapped into SQL, and then results mapped back to the client's output requirement (unspecified in the story).
His code wasn't super fast and wouldn't win any awards for being clever, but he got it done quickly and it worked. They were able to ship. Over the years, as they moved from one web server to another and from one platform to another, the maintenance programmers they brought on later (for Joe had moved on, as people do) were able to adapt it without too much trouble. The CGI aspect meant it could run on IIS or nginx or whatever else they found.
It would never scale to huge numbers of queries per second, but it wasn't supposed to. It needed to handle 500 devices in the first year polling about once an hour. That's 12000 queries per day, or .138 QPS (queries per second). You could almost service those requests by hand if the timeouts were high enough and people were cheap enough: just cut and paste!
Bob asked the same questions, wrote CGI and hand-coded SQL too, but made a mess of it. It took him three times as long to write. It had numerous security problems and took the site down when someone discovered they could pull a "Bobby Tables" attack and change everyone's recipes to involve dog poop. Customers rioted and demanded refunds. The company went out of business.
They used the same technologies and got two different results... and this happens constantly.
Some technologies are used to create only garbage, sure, but the technology itself won't guarantee success.
Dianne's solution creates a maintenance problem if the number of devices grows too much or if network stability is a problem, or if there is an expectation down the line of, for instance, push sync.
My point is not that her solution was not valuable, but that to suggest that the maintainability of the two solutions is black and white like this is wrongheaded.
The really unfortunate implication here is that learning new technologies is either something that shouldn't happen or something that shouldn't happen on the job. I'd say that something with these requirements would have been a perfect opportunity for Diane to, "learn postgres," if she felt that it would have been better. The extra day or two of effort would likely pay off – if not on this project, then on the next.
> but she also knew that anything much more complex would be beyond her current skills.
And with this mindset, her skills for the foreseeable future.
> Dianne's solution creates a maintenance problem if the number of devices grows too much or if network stability is a problem, or if there is an expectation down the line of, for instance, push sync.
None of those are maintenance problems, they weren't part of the additional spec. For custom to-spec development, if the requirements change then it's expected that the code will have to change as well, incurring a change order.
A maintenance problem would be something like MySQL being an unreliable DB or the code she wrote being unreadable. If someone needs to go in and fix inevitable bugs then there will be issues.
If the customer wants to add features later then they'll have to pay for the changes and if the new features are different enough from the original spec then the changes might be quite expensive.
The ideal solution would be somewhere between a massively scalable system and a system that just barely meets expected capacity. When an engineer is designing a bridge they over-build just enough that the bridge is solid even when packed with 18 wheelers in a snow storm. Most of the time the bridge won't be anywhere near that load. A great developer builds to the expected capacity + some additional capacity based on the uncertainty of the prediction of capacity. I don't see either Jack or Dianne as perfect developers.
I agree that the "bridge" should be built to the expected capacity + additional due to uncertainty, but that's a slippery slope and one that this article is trying to address. It's a judgement call on what amount of work is appropriate for what amount of capacity, as adding on more capacity due to uncertainty might delay the product or solution from launching.
It may be more appropriate to properly separate concerns and build a flexible architecture that can be upgraded over time to meet needs as they come, or left as-is if those needs never manifest.
MySQL can easily scale to well beyond the "500 users" unless each of those users puts an especially heavy burden on the data store. The scenario at hand is synchronizing recipes and that is not a heavy burden.
Further, with a high-performance web framework, foregoing resilience (not recommended, but just for argument's sake) a single web-application instance should be able to handle a much larger number of users as well.
But all of this isn't really relevant to the point made by the OP. He's just using these fictional data points as an illustration. Although I am personally in favor of using examples that are more representative of reality, I think we can mostly ignore the fictional data points to appreciate his point.
I guess it depends on how efficient you want to be regarding your hardware costs. I have had to replace systems built in exactly this way because of traffic from fewer users than you'd think.
There's a difference between technology and implementation though. If Dianne had chosen a solution that fundamentally can't scale easily then that is a poor decision, whereas a technology that can scale written in a way that can be upgraded later is much more flexible.
As for the MySQL v Postgre I see that as a sensible Agile decision - much more confidence in quality now, with the knowledge that in the future a move to a different DB is beneficial.
The REST/Sinatra/MySQL solution was chosen in contrast to protocol buffers and Cassandra - the scaling potential of the systems are very different.
If you haven't worked on a project of this nature then the complications are likely not to be obvious to you. For instance, once this ships, they will be required to maintain that version of the REST API that was designed forever, including any side effects or semantic weirdness. If they determine that some changes are required for the sake of scale, they need to cross their fingers pretty hard that they can squeeze all of those changes into the existing API.
If this system were truly designed with Dianne's current abilities in mind, we don't know what that REST API looks like. Is it possible to imagine that it sends the full listing of recipes on every request? Absolutely. Are you going to be required to do that forever once the device ships? Yes.
The ideal way is to build for either postgres or mysql, using vanilla SQL, therefore both shipping something that is known to work (mysql, in her case) and learning something about postgres. My observation is aside from anecdotes and peculiar corner cases which should be very strongly avoided, the philosophy of mysql is always the cub scout motto "try your best". The philosophy of postgres is more like "It shall be perfection at all times or I will curl up and die horribly as the ultimate prima donna". When times are good and there's nothing unusual, there is absolutely no practical difference. Its a meaningless personality test to walk up to a dev and ask which outlook on life matches their own outlook and which makes them feel like vomiting. It tells you something about the dev and nothing about the problem. Rarely, if ever, does actual business need enter into the debate.
Postgres/mysql is a business needs decision that needs to be made at a higher pay grade than hers, most likely. Assuming anyone at a higher paygrade knows anything about databases or specifically mysql vs postgres. If no one at a higher level understands, its her job to put it into simple little one syllable words even an overpaid MBA can understand, or they simply have to trust an underling to make a higher level decision.
Something not discussed in the article specifically is this behavior comparison is classic bodger/tinkerer behavior aka Jack vs true engineer behavior aka Dianne. Most people are flexible between the two, some are notoriously inflexible... I have a backyard garden swing built to hold 4500 pounds because that reflected my junk pile at the time and desire to never maintain the swing after installation, although over-specification behavior like that at work would lead to my firing or the financial failure of the company, so I gotta know when to switch. This should have been explicitly discussed in the article.
> I'd say that something with these requirements would have been a perfect opportunity for Diane to, "learn postgres," if she felt that it would have been better. The extra day or two of effort would likely pay off – if not on this project, then on the next.
Perhaps on a product team but for an agency dealing with clients your time is your money. You spend an extra day or two on the project and the client is getting charged for that. Now they're wondering why we're thousands over their budget when the initial perception was we could work within it.
I'm all for learning new things on the job but that opportunity is only for certain situations.
I agree with the op but have a different spin on it. It depends on what you see as your core business. Is your business about the technologies used to build the application or are you using technology to let, "kitchen devices synchronize their list of recipes". If your business is the later then you are better of using tried and tested technologies your developers are familiar with and can maintain without detracting from the core business.
I think the exact opposite holds true. If you're a MySQL / Sinatra shop, being the best at MySQL is the goal - double this if you're working for clients.
If your job is to provide the best possible kitchen device sync platform, then doing on the job research into improving that is essential to the business.
I do not disagree with you. It is all relative and the most important thing is to know what your core business is. Using a car analogy, car salesmen and engineers both need to have technical knowledge of the cars they work with. One focuses on selling cars and the other making the cars. I think too often start-ups are not too clear on their core business. Engineers know the cars better than the salesmen but does that make engineers better equipped to sell cars?
Dianne knew it wasn't the most elegant solution, but she also knew that anything much more complex would be beyond her current skills.
Since when are complex solutions elegant? I always thought the simple ones were the ones deserving to be called "elegant". Also, the cheapest, fastest, and most reliable components are those that aren't there.
I don't agree with the article at all. At least, not in the way it was presented.
Sure, designing such a small system to use Cassandra and Protocol Buffers might be overkill. Then again, it might not. Without more information, it would be impossible to know which one is the best developer.
For instance: how much time did the Node+Cassandra+Protobuf implementation take? How is that, compared to the junior dev that only knows MySQL? Was Jack on a deadline, or did he have time to explore alternatives? Was Jack really inexperienced, or he did have experience in the new tech, but wanted those in his resumé?
Were they billing hours? What about the coworkers, do they only know Ruby and MySQL too, or are they confortable about learning new things? Do they even have coworkers?
Was Jack's solution modular enough that components could be replaced, if they were a problem? (Cassandra, for instance)
And most importantly: what was it that made Jack's implementation unmaintainable? Sure it wasn't because they are "new"(the article is rather old) technologies. Was the code badly documented? Was the design flawed?
Jack's biggest flaws, as can be inferred from the article, is not asking enough questions before diving into the implementation, and not getting permission before using "new" tech.
I know the author did not get in depth about the personalities or the project itself because he wanted stereotypes to illustrate a point, which was taken. But the problem with it is that other people could get the impression that "new" technologies should never be considered, or that any attempt to make the application "scalable" from the beginning is a "Jack trait" and thus, wrong.
Not to mention that Jack might even be right, should the company's expectations prove to be too low.
Any article stating one trait as being most important for anything is probably being hyperbolically reductive, but in this case it's particularly painful because "judgement" is absolutely meaningless. Every single thing every person does requires judgement, and if you don't have it you'll probably get hit by a bus before you have a chance to get a job.
What this article is really arguing against are early-adopter developers who chase every new fad for its own sake and prioritize their own amusement over the best solution at hand. But this says about how good the developer is, only how good their actual contribution to your company will be relative to their potential. A shit developer can do their best but it still won't measure up to a great developer's trend-chasing effort.
Now even assuming some base level of competence, there are many different mindsets and personalities that lend themselves to different types of programming. I could go on and on about specific skills but we're mostly programmers here so I'll just hint at it: debugging skills, low-level skills, architecture skills, analyst (business requirements) skills, hci skills, modeling skills, clairvoyance, luck. All of these things have varying degrees of importance depending on the job and team at hand, and some combination of them will make someone the best developer for some specific job.
But if you find my answer too wishy-washy and you want specifics about what makes a great developer I'll boil it down to two things. First, to achieve competence as a developer, one needs have the tenacity and logical thinking to debug any issue no matter how arcane. Second, to achieve greatness one needs to be able to comprehend the entire problem space in such a way as to account for many more factors than an average developer let alone a non-technical person could ever hold in their head at once, and have the ability to translate those constraints into a system which is at once easier to understand and more elegant than the vast majority of competing solutions.
Nearly all the comments so far focus on neophile vs. neophobe. But I took the main point of his post to be orthogonal to that: Dianne took time to ask questions. She didn't immediately dive in and start using her {comfortable old tool, sexy new tool}. She asked about the requirements: Estimated user count, load, connection quality, etc.
That's the key point. A developer with good judgment will start there. A developer with good judgment might even raise their hand and say, "As I understand this problem, it requires technologies I'm not so good at. What should we do?"
There are exceptions to every rule, and all sorts of projects. But in general, that seems like the sort of developer you want to hire.
There seems to be a self-imagined divide between these two "camps" of developer. We see:
1. Hipsters that throw tools at a problem, and are eager to know as many different languages/frameworks/tools as possible, to stay ahead of the curve.
2. Developers that have picked a framework, and stick with it for life. They learn when necessary, but their framework can handle most problems thrown at it.
I would consider myself a part of both camps. I read Hacker News because people talk about these tools in use, and I love sitting down at home and toying around with new JavaScript frameworks, and different environments/tools. However, I have a job where I use a proven framework, a solid language, and a role where I model problems with a team of like-minded developers.
Now, imagine the scenario. I toy around with a new framework and absolutely love it for some use case. I decide to write a tool using Node.js, I use Bootstrap, and then I post it on my blog, Twitter, and on Hacker News. Some will check my work out and think it's cool, and others will look at it and will vomit at the very thought of JavaScript running on a server when another server-side language/framework would be far more suited to the problem at hand.
The point I'm trying to make is that developers are so keen to slate projects/code that was written for fun. My Node.js application isn't very good. Hell, I can probably write it a million times better in the language I actually use day-to-day in my job. I just wanted to release it so others can learn from it, and to show that I'm capable of keeping up with everyone else that is releasing apps-a-plenty in all these crazy new languages I see every day.
I agree with the general idea of the article, but only because developers are starting to embody these camps. I see entry-level/junior developers that apply for jobs with CV's that list every language they've touched for at least half an hour, and I see battle-hardened developers complaining about people trying to legitimise new tools because they're "not as good" as the tools that everyone else uses.
How is Dianne solution not maintainable or scalable? I have used flask gevent and mysql to take similar load. MySQL is not bad at all if you check everything you send it with explain detailed while developing.
Jack drops a couple turds into your codebase, but so does Diane. Hers just take longer to arrive. Inability to seek actual best tools based upon personal fear of the unknown does not a great developer make.
Basing one's code off the familar or the purported "best practice" without actual knowledge leads to drek down the road. A great developer chooses the right tools for the task (along with other things...) Diane's, for instance has these flaws:
- Requires a relatively heavy runtime.
- Requires HTTP stack.
- Wastes CPU/power
These are not negligible considerations for embedded devices.
Someone who knew what the crap they were doing would likely have chosen a binary protocol standard (many such exist), then pumped that information out over an appropriate topology - someone suggested UDP. The database should probably be SQLite; maybe berkDB, depending on the complexity of the data model and the CPU used. Bluntly, the proposed "great" solution isn't. It's an inelegant solution putting heavy demands on the hardware unnecessarily.
Judgement is fine. What you use as a proxy in hiring for judgement is experience. E.g., you don't hire a Ruby dev to do embedded work. Experience tells you that.
What makes a developer great is great wisdom: knowledge applied skilfully.
n.b.: I wouldn't hire either Jack or Diane based on this article; Diane is fearful and Jack picked a javascript engine too immature for embedded. Jack might be led to understanding and might be worth more; I don't want to go around trying to convince people that better is better - I have no hope for Diane if she can't spend a day learning postgres.
I thought that in this contrived example, Jack and Diane were writing the software that runs on the server, not on the kitchen devices. Also, what if the "devices" are small-form-factor PCs? That would relax the constraints considerably.
Ah, see, I read the spec as saying that it would be things stored on the kitchen devices (toaster, etc).
You do want to make sure your comms layer is easy to make reliable when talking from the device, regardless. Might be HTTP, might not be - I'd likely take a gander at ProtoBuf myself for feeding over TCP/IP to minimize comms complexity on the phone home code.
I think I'd use HTTP as the synchronization protocol, because GET and PUT are specified as being idempotent, meaning it's safe to retry them. Also, HTTP is a ubiquitous standard, meaning there are several implementations to choose from, on both the client and server sides.
The ubiquitous of HTTP is indeed a definite point in its favor(as is its popularity and level of documentation).
> GET and PUT are specified as being idempotent, meaning it's safe to retry them.
Entirely OT: Abstractly, yes, that's how they should be implemented, but they don't have to be and consequently it's entirely up to the receiving server whether they are or not. Which, in terms of assuming that someone will make unverified and bad assumptions, means that the implementer at some point will let them slip and become non-idempotent.
:grin: Yep! As the original article said, digging into the spec and clarifying is absolutely a trait of a great developer (even an average one).
Possibly being able to write a very clear requirements/spec with very few words is one of the hallmarks of the greats. (The initial RFCs for TCP/IP are amazing).
I think one of the big valuable traits of a good programmer is a real fear of complexity. But I don't think the biggest problem is the complexity that comes from "rockstars" and over engineering. I think it's a way bigger problem that complexity tends to grow uncontrollably as features are added and bugs fixed. Especially when the domain is poorly understood and when the team isn't focused on keeping a coherent and simple system.
This kind of creeping complexity isn't introduced by mavericks because it's fun. It arises because of a conservative tendency to never throw anything out, to avoid refactoring, to be afraid of changes and simplifications. And also because of simple incompetence in design. Special cases multiply, no fruitful abstractions are introduced, the team's language is confused.
It's not as simple as that though - it all depends on the context of the project. Sometimes it is perfectly OK to go crazy and use bleeding edge technology other times that would be daft - it all depends what you are trying to achieve mixed in, as the article says, with a heavy dose of judgement.
It's about how you use bleeding edge technology though, especially after playtime is over and you've established it does what you want. All too often the engineering of things like abstracting the bleeding edge tech, or documenting what is going on gets forgotten. Then some other poor soul is left to pick up the pieces when said bleeding edge tech released a breaking API change.
Raw UDP+checksum might have been even better. Easier to build on a tiny microcontroller. Lighter on the network. Offers an expansion path to NAT hole punching for push updates.
I agree with the author. A few (actually still good) devs I know prefer to solve a problem in a sophisticated manner, scalable and all the stuff in 160 locs instead of using the non-scalable solution of 40 locs that will also solve the problem, if only for the next 12 months or so.
I think the problem is the incentives. As a founder I need the stuff done until some deadline while as an employee I prefer learning new stuff in a company.
So I found myself on both sides of this story and I have never been more productive than as a founder because I only choose the small scale solution because I can complete two tasks per day instead of one task every two days.
Note that 40 locs is meant as a placeholder for "quick to program and easy to maintain" code and 160 locs for "fast to execute, scales, worse to maintain"
EDIT: Worse to maintain as in "the solution is not immediately obvious or I need to understand the abstract concepts employed or I need to know some special lib to understand it". All three forms of "worse to maintain" are examples of a greater investment of time and brain to understand what's going on.
This kind of judgement is not a sufficient and not even a necessary trait of great developers. It's nice to have, but 10 such developers will not allow you to build anything better than Dianne's solution.
Many great developers have misjudged their abilities or the practicality of particular ideas and solutions. I won't name (and shame) any, but look for examples among famous game developers.
I would disagree in that's the working definition I've most commonly heard. Think of rockstars doing the same old thing but now with a cool trendy new sound. Yup, its just "happy birthday" again, but I've got a rockin' hairdo and a wild guitar solo (its all about the solo with these guys) and I added some pyrotechnics to the show, although it does nothing productive. None the less, at the end of the day, other than an impressive show, it was still just "happy birthday".
I don't think it involves wearing leather pants to work, although it doesn't exclude it, I suppose.
Listen to the why's, and not the what's, and you'll hear judgement.
In my experience this is very ill-advised. The answer to "why" questions varies so dramatically that it's extremely difficult to get an unbiased, realistic answer to those questions. I normally ask a bunch of "how" questions and eventually you unravel the "why".
The questions presented sound good on paper but don't matriculate that way in person. For example:
What's your least favorite part of Ruby and the Ruby on Rails framework, and why?
Tell me about the last time you used an interesting bit of technology, and what you learned from it.
Are much more naturally spoken as:
How do you feel about your least favorite parts of Rails?
How have you benefited from an interesting bit of tech?
It's amazing how much more human responses become when you use the words "how" and "feel".
Some people (you call them Rockstars) are interessted in the newest technology and they want to use them to solve problems. No matter how efficient it is or how good somebody can maintain in.
But you forgot one thing: the "Rockstars" use technology, that will be normal for the masses 2 or 3 years later.
Ruby, for the first time, was used by Rockstars too.
It depend on your company and the size, but you need some rockstars and a mass of good developers. The rockstars look for new things, they try them, they show how to use it. The mass of good programmer can look at the results and choose what make a sense to use in productive environmnet.
There is nothing bad with a rockstar programmer, only with people that think they are rockstars and talk like "i am the king baby". ;)
"But you forgot one thing: the "Rockstars" use technology, that will be normal for the masses 2 or 3 years later."
Or sometimes they pick the wrong tech, and it dies, and someone else gets to pick up the pieces 3 years later. Predictions are difficult, especially about the future.
The problem they forget to mention is that Diane's skills stagnate. Sure she did better this time, but Jack has learned more. Next time through his solution will be faster and have less errors.
If everyone was like Diane and just chose what they knew we'd still be coding in Assembly and be nowhere near the throughput and capability we have now.
I often will try multiple implementations if I have the time, break the problem down to something simple I can do with a few languages and tools in a day, contrast them when I'm done and then go full scale with the one that seems a good fit. More times than not it ends up being a combination of all the solutions I tried because it gives me a chance to discover the strong points in each.
Yes. Because good judgement requires understanding complex situations, and that requires skill. And judgement is improved by experience.
For example you see a bad thing in the code? If you can understand the problem and the risks and benefits, you can make a good decision. If you can possibly make it better, understanding the time available to you, and the risk to the product, you should make it better. Not put it off and let the bad thing become more entrenched technical debt.
Making that kind of decision, requires good judgement. It requires understanding your own skill, time management, risk, and more.
Being able to exercise good judgement comes with extreme rewards for any project, as there will be less costly mistakes, and more successes and better code.
In my opinion a great developer is the one who finds a solution to a problem. It doesn't matter which technology he/she uses. If your organization is already using some tech then you expect the developer to use the same tech so you don't have to hire 10 people for 10 different techs. Any any decent developer should be able to look at any code and figure out what it does and be able to tweak a few things here and there.
More important things are proper comments, readability of the code and documentation. If there's a bug and you are not around, someone else should be able to fix it by looking at your notes. The bug fixer doesn't need to be a guru of that tech.
I find the list of comments regarding this being "narrow minded", "contrived" or generally this being "a bad article".
It isn't, it's spot on. If you're outside the bizzaro world of overvalued, under-producing nitwits, the vast majority of technology is written on what's already understood by the team/developer. Because, you know they have budgets, and project managers, and families.
Jack isn't a rockstar, he's a self-important ass using his employer's/client's funds to learn a new technology that no one can maintain.
Maybe it's just because the post is out of date now (2011) but I'm not sure why "new technology", "rockstar" and "produces unmaintainable code" are all conflated together.
Does "rockstar" mean "bad" in this context? Because "using the latest technology for the sake of it" is not a trait of a good developer, and I have no clue what the hell "rockstar developer" means any more (unless they're also in a major rock band, then I know what it means).
This is a pretty bad article. Sure judgement is important, very important but sometimes using a new better set of technologies is the better judgement. That doesn't make things inherently un-maintainable just because it's new. That's where the judgement part comes in, knowing which new technologies to use in order to build a better/more maintainable system. This smacks too much of the author saying this is what I know so it's the best and everything else is crap.
Hacker news commentary options on coding advice:
1. Spin the advice to make it sound like the advice is bad
2. Turn the advice around to make the author look bad
3. Criticize technology choices ( promoting your own choices of course )
4. Some variant of "duh" or "don't reinvent the wheel"
5. Lament the sad realistic state of the programming world
6. Cast the author and others into various categories ( such as what I'm doing now )
On this subject. What I find funny, slightly scary and hugely rewarding is visiting old code and removing massive parts of it only to leave it doing pretty much the same thing.
It touches on a coding horror article about wanting to write code. Don't 'want' to write it, want to write as little as possible of it. If you do have to write it, write as little as possible and with the notion you'll be looking at it in 6 months going wtf.
Learn new technologies on your own time. Don't use the client to fund you picking up the latest-greatest whatever. That's what personal projects are for. You take the knowledge gained from your personal projects (and unforeseen circumstances in paid projects that you had to code your way out of) and do what you know will work the requirements, versus what will be exciting for you.
These articles always should be taken with a grain of salt. Maybe a few grains when they don't have a real story to back it up. I want to hear about 2 real developers, what they did, and why one result was better than another.
Diane will be happy to stay for 10 years while getting average salary. Jack will move on to a cooler place while bumping his salary quite a notch because he knows latest tech.
Hire engineers that will optimize your apps and systems for the highest business value and the lowest cost.
It seems silly and obvious to say that, but many engineers seem to have forgotten that goal.
The 2 mistakes I see over and over are systems that are optimized for Novelty and systems that suffer from Neglect.
## Mistake: Optimizing for Novelty
Bored engineers often optimize for novelty. That means they add technologies to your systems not because they're needed, but because the engineers want to play with them. Rather than consider the implications of making the systems more complex and more costly to manage, they are easily lured by shiny new technology and come up with justifications to use it even if it provides negative business value.
These engineers aren't bad people. They will legitimately think that the new technology is a good idea. They may even make an impassioned business case for it. But listening to these engineers is the equivalent of a naive young girl believing a teenage boy "really loves her" at the end of a first date when he wants to go to Make-Out Point "just to talk". The boy may actually believe this in his heart, but any woman with more experience will see exactly what's really going on.
To see companies that suffer from this, it's as simple as looking at a few job postings for developers. Look for the postings that list requirements for waaaay more technologies than the company could possibly need. It's a sign that novelty has cursed that company.
If someone offers to make your systems "exciting and cool!", be very afraid. Your systems are the foundation of your business and they should be simple, secure, scalable, and generally pretty boring.
## Mistake: Neglect
The other problem I see frequently are systems that are simply neglected.
No one knows if the database backups work. No one knows if there are critical security updates that need to be applied. No one knows if the site is down until a customer complains. No one knows if the guy who quit a year ago still has a copy of the production database. On and on...
You will often see neglect in systems that became complex due to novelty. Every new technology that is added to a system requires monitoring, documentation, and security updates. The more technologies that get added to a system means the more technologies that are at risk of becoming orphans and neglected.
## Shameless Plug
Manage servers? One of the biggest wins against neglect is to use a configuration management tool like Ansible, Salt, Chef, or Puppet.
If you want to make sure your systems are fast, scalable, and secure, the first step is having full control and power over them.
Tomorrow, Sept 4th, I'm launching my book "Taste Test: Puppet, Chef, Salt, Ansible" which is designed to save you the days or weeks of research when picking one of these tools.
In the book, I implement an identical project with each tool so you can see what each one is like to work with. You may be surprised at which ones were super easy and which ones were really difficult to work with.
I completely agree that pragmatism is essential to being a great developer. When designing a system, you should be asking yourself questions like:
1) What sort of load will this system need to handle now and in the foreseeable future, and what sort of latency can it tolerate? How will that load flow through the system? In this contrived scenario, the server and database need to be able to handle an average throughput ~9 requests a minute with lax latency requirements (it's okay if things are backed up by a couple minutes). The devices only need to handle one operation per hour. Planning a safety margin of perhaps 10-100% above that is prudent; more than that is premature.
2) What are the upgrade characteristics of each component: how will you upgrade or replace each component? how long would it take? can you even upgrade them? Some parts will be in a central location, and can be easily upgraded with only minor downtime. Some will be deployed on hardware you don't control or otherwise can't be easily upgraded. In the worst case, they're embedded in firmware and you need to support them as-is for their entire lifespan.
3) What are the failure modes of each component: what could cause them to fail? how likely is that? what will happen if (when) they do? how will you recover?
The answers to those questions determine how much risk you can take when designing each component.
Also, look at where the answers are different for two components that talk to each other. Any time there's a boundary between two components that can't be upgraded at the same time, or where one component talks to another with different failure modes, there's an interface. You need to think hard about the interface between them: how will they communicate? how will you extend that communication protocol over time while maintaining backwards compatibility? how will they behave if the other has failed? Those interfaces are locations where over-engineering a bit can pay off.
So for this example, I would think hard about what runs on the devices and how they talk to the back end, because that code is probably very difficult to upgrade. I would use rock-solid, dependable technologies for those. Similarly, the database probably stores important information, and one of its failure modes is that all the data is lost due to hardware failure. So I would use a mature database there, either MySQL or PostgreSQL.
But the back-end service? It's easy to upgrade and because it's stateless, its two failure modes are that it crashes and it needs to be restarted, or that the hardware fails and it needs to be re-deployed. Neither failure mode is that bothersome, so it doesn't matter what technology you use there. Use whatever the team is most familiar with and will let you iterate the fastest.
This is one of those "why I am a great developer" sorts of posts that allows people to pat themselves on the back and congratulate each other on how great they are, passive-aggressively sending it out to the team in hopes of denigrating some team member who likes newer things. In this case "judgement" means "uses the things that I know", and there seems to be little evaluation beyond that high-level use case.
Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
From a management perspective, it is easier to hire competent new employees for established, popular technologies.
Employees don't stay around for long anymore, especially dynamic, innovative employees who stay up to date on the newest technologies. People like that get recruited away all the time. Then who's going to maintain your bleeding-edge system? How long will it take to hire another employee of the same caliber?
If you're a top company who can afford to recruit and keep the best employees, then by all means, build on the latest and greatest. There is certainly nothing inherently wrong or unmaintainable about technologies just because they are new.
But if you're a medium or middling company with significant turnover in the tech ranks (i.e. most companies), then it's probably safer to build with the most popular, "standard" tech you can get away with.
I hated this perspective as a 22 year old when my (prior) company practiced it, but I can see the wisdom of it now. Not everyone should be on the bleeding edge, or even leading edge. It's important for both employers and employees to be on top of how close to the edge the firm chooses to be.
Yeah, I agree with you: 'just because' .. the author of the article doesn't know anything about protocol buffers, node.js and Cassandra. Probably because those things are 'too new' and 'not proven' technologies - to him.
But if I had to take over a project from someone, to maintain it, I'd MUCH RATHER have it be done using those technologies than MySQL and Sinatra. Yikes!
I recently had to make a bid on a project that would give the users the ability to share their works, produced in an embedded environment, with their communities - a "Cloud" for the users. I designed the system architecture of the embedded system to be as self-maintainable as possible, and as simple as possible. Since its an embedded OS environment that the users would be using, instead of writing code I integrated such tools as git, and designed a system topology that would allow the user to be shielded from all the git hassle, while still getting a push-button experience.
I lost the gig to someone who decided that it would be better to re-invent "topography ordering" (his words) in a custom MySQL+PHP application, and when I was given an opportunity to challenge his design, I said "great, you're going to re-invent Git, poorly".
He spent the rest of the day "pissed that you compared his project to .. GitHub", which is what he thought I meant when I said "git". Nope, I said "git", I mean "git" - not 'github'. He actually didn't know how to use git.
Either way, off they go .. re-inventing the DAG with MySQL+PHP in an embedded environment, when they could've just wrapped the feature in a git script ... yikes++!
So this article just demonstrates to me that there are a lot of blow-hards out there who just don't get why its so important to apply the NIH principle to themselves, not just others ..
But if I had to take over a project from someone, to maintain it, I'd MUCH RATHER have it be done using those technologies than MySQL and Sinatra. Yikes!
Except that isn't really an option in most cases.
The two most realistic choices are:
Take over a high-quality implementation using somewhat stale tools or an average to shoddy implementation with newer tools that the developer was learning on the fly.
Coming from a functional programming perspective on code quality, I think new code written with a modern understanding of software engineering generally is better quality than old code backed by tech written with last decade's understanding.
.. or, take over a project that was done by a competent developer who knew what they were doing and were able to communicate that knowledge through their codebase to anyone else who might pick up the project later on - which is the real #1 Trait of a great developer, whatever technologies they're using ..
The quality of an implementation is rarely tied to the tools except in extreme cases. A good Developer in general builds good solutions no matter the toolset. A good python developer is likely to become a good Objective-C developer if they invest the time to learn and because they understand the need for clean modular code the iOS code is likely to turn out pretty well.
Take over a high-quality implementation using somewhat stale tools or an average to shoddy implementation with newer tools that the developer was learning on the fly
My experience has been that developers who are most interested in innovations in this industry are dramatically more likely to build better solutions -- the passion and interest cuts both ways. Versus the career developer who does the absolute minimum necessary.
But of course with any given situation and set of developers your mileage will vary (I've known on-the-edge developers who build ridiculously nice code, and hanging-two-generation-back coders who make disgusting code. And vice versa). This particular scenario bothered me because the nodejs / cassandra canards were propped up under the illusion that the only advantage they offer was nefarious "Web Scale". Only in reality for many projects they make it ridiculously efficient to build solutions. The same goes for Go right now -- there are practitioners who can build in moments what would have taken months on a, for instance, ASP.NET team.
My experience is that for every developer who adopts a new technology for its true merits, there are at least two others who use it only because it is at the top of Hacker News/Ruby Weekly/other trendy publication, and those developers take advantage of very few of the technology's benefits.
> Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
> The result was a combination of Google Protocol Buffers, node.js, and Cassandra. Elegant, scalable, and totally unmaintainable.
If I can venture a guess, Jack went ahead (without asking further questions, as opposed to Dianne) and took what was very hot at that time.
2 years ago, Node.js was extremely hot, in a sense that it was the new thing that somehow combined the hot javascript and made something easier. I remember my collegate excitedly talking about the amazing serverside javascript.
I don't know about Google Protocol Buffers and Cassandra but I assume they were what the author had in mind as 'very-hot-but-not-widely-used-yet', and most technology that fits such description generally falls into 'not-reliably-stable' and 'not-many-resources-yet' category.
Think how you would do Java EE without help of Javadoc.
I got the same feeling as you niuzeta, and I feel the article may be getting judged a little too harshly (although I find the assertion of this being the best quality in a developer a little too absolute for my tastes).
I think the author stated in the article that the fictional character Jack ".. makes a point of starting each project with at least three new technologies."
I personally understand this as three new technologies Jack has not previously worked with, which should be a definite indication of some risk/unknown factors. Plus given the context of this fictional scenario, it seems unlikely that other developers associated with the project/company know them.
Now if everyone were so conservative that they'd never try anything new, there would never be any progress, but I think the point of the article is that something isn't better, just because it is new. In fact, to me the rings of the "worse is better" philosophy which I didn't think was taboo in any way here on HN :-)
Finally I'd like to add that the author didn't say this was a "maintenance nightmare", he did however say the solution was elegant, scalable and totally unmaintainable.
>Finally I'd like to add that the author didn't say this was a "maintenance nightmare", he did however say the solution was elegant, scalable and totally unmaintainable.
Which is impossible, right? If it's elegant, it's maintainable. That's part of the definition of elegant software. A on liner that does amazing things, that very few people understand is not elegant. The same thought scales up to a whole solution IMO.
> Why was -- in this contrived scenario -- Jack's solution a "maintenance nightmare"? Just because.
Because apparently the author didn't have the necessary skills to maintain it. Cassandra isn't easy to master, especially if you only have cheapo developers who are reluctant to switch to PostgreSQL from MySQL.
But yes, it's a highly opinionated, narrow-minded piece.
Having only used Oracle, MSSQL, and MySQL, I can't imagine a small 500-user app would be that hard to implement in in Postgre even if I'd never touched the platform before - does Postgre have some terrifying gotchas, or does it work as a reasonably well-behaved SQL server? That is, if I naively installed a PostgreSQL server on my box and started creating tables and indexes and users, would I be destroyed by some horrible gotcha as I failed to account for some critical config (like selecting the right storage engine in MySQL)?
It's my understanding that there are no config gotchas in PostgreSQL (unlike MySQL). I've definitely found it more of a pain to set up at first. This is because the default role is very restricted and defaults to safe.
As someone who moved from MySQL to Postgres, the only thing that took some getting used to was users being actual Unix users that you create and then grant access to. At the time it seemed odd considering what MySQL did but it ended up being a feature I like quite a bit.
PG roles are stored in the DB, not Unix. You can set up pg_hba to use ident authentication so that users are automatically granted access by their Unix login, but it isn't the default (and usually not recommended), and you still have to have PG roles in the database for them to correspond to. So I think there's been a misunderstanding.
They're similar, but there are some pitfalls you need to be aware of: count(*) optimisation in mysql for example. Different behaviour in the query optimiser. Most cases are easy to handle, but if you're used to mysql, there's still a learning curve.
Agreed. If anything the number one trait of a developer is that they solve problems. However, that is not unique to developers and is required of non-technical people as well. What it means to be good at solving problems seems largely context-dependent to me, but if you are really good at solving important problems in a valuable domain you are great at that thing (developer or not).
Such an argument would be rich coming from someone building a solution on Ruby. Indeed, such arguments essentially limit your technology choices to Oracle, SQL Server, or DB2, with an app layer built on .NET or J2EE.
The answer of course is that judgement is less about making decisions upon familiarity per se and more about knowing when unfamiliar technology seems to provide good trade offs given what is known about it. For example, PostgreSQL may be unfamiliar, but choosing it is a wildly different type of decision than choosing another less mature unfamiliar data store, since at a high level PostgreSQL is known to be a well-understood, mature technology that may provide significant leverage over MySQL in certain scenarios. Being able to understand the various dimensions of technology choices and make good ones with limited information and experience is what makes good judgement, not just an outright aversion to unfamiliar things.