Skip to main content.
Features: Calendar | Interviews | || Login

Interview with Damian Conway

Damian Conway is a frequent speaker on Perl (and often topics unrelated to Perl, such as quantum computing, Harry Potter, or the Klingon language), has written several Perl modules, and is the author of Object Oriented Perl (Manning Press) and now Perl Best Practices (O'Reilly Media) which will be available soon (although you can pre-order it on Amazon (hint hint).

The Perl Review: Perl Best Practices has 256 guidelines. Is that number magic? Did you have to leave off some practices to make it work out?

Damian Conway: The weird thing is that it was totally accidental. Originally I had about 220 guidelines planned, but that slowly grew to about 240 as I as I wrote the examples, watched how I coded, and noticed that I was doing things (good and bad) that I hadn't realized I did, which I then needed to address.

The remaining guidelines were added during the technical review phase, as I discussed and debated suggestions with our 27 technical reviewers. Several of them had extensive industry experience, and were able to point out common mistakes that they saw all the time, which I was able to then cover in the book.

I didn't have time to track the number of guidelines during that last phase of writing, so it actually came as something of a shock when the number came out to precisely 2**8. I had to count again to make sure.

TPR: Who should read "Perl Best Practices"?

Damian: Well, naturally I hope every Perl programmer will read it.

Novices should read it because it highlights many of the "gotchas" that can make Perl harder to code until you're familiar with its mindset and idiosyncrasies. Intermediate programmers should read it because, along with experience, coding discipline is the main thing that separates a good programmer from a great programmer. Expert programmers should read it because they're the ones writing large, complex systems in Perl, so they're the ones who benefit most from the consistency and reliability that rational coding standards can bring to their source.

And, regardless of the level at which they're currently programming, I'd like everyone to read this book--not so much for the specific guidelines it suggests, as for the deeper issue it tries to raise. I'd like them to contemplate the notion that you can make choices about how you code, and that those choices really do make a difference to the quality of the code that you produce.

As I say in the Preface:

Just thinking about these issues--becoming conscious of the way you currently write code--can be of enormous benefit, even if you don't adopt a single one of the recommendations that follow.

TPR: You say that the guidelines are not ivory-tower. Is that ironic coming from a published (former) university professor?

Damian: And not quite "former" either. I still hold an honorary academic position. Yes, I suppose there's a certain incongruity there.

It's certainly true that academics--even nearly reformed ones--can be prone to ivory-tower thinking. But only if they stay in their ivory towers. And that's nearly impossible to do in modern economically rationalized, product-oriented, market-driven academia. "Publish or perish" has largely been replaced by "profit or perish" and that doesn't leave much room for wishful thinking.

Long before I reached academic escape velocity a few years ago, I was already earning most of my income from teaching and consulting into some of the biggest Perl programming shops around the world: Fortune 50 corporations, large scientific organizations, major government departments. And I hope that I've learnt as much from that experience as I've taught during it.

When you're given the (often nerve-wracking) privilege working with people who process transactions worth a billion dollars a day with Perl, or people who use Perl to track and cross-reference hundreds of thousands of major criminal and civil trial precedents, or people who filter vast quantities of scientific data through Perl, you quickly discover which techniques scale up to the real world, and which are best left in the classroom.

So when it came time to writing down all that advice, I was particularly careful to only include things that I know actually work and which scale up cleanly to large systems and real-world datasets. Several of my technical reviewers were particularly valuable in that regard; it's hard to argue with the practicality of advice from people who have actually built aircraft control systems, or real-time stock-trading agents.

TPR: Some of the practices seem to say "stay away from things you don't understand".

Damian: Well, I'd say that the message in those cases is more often: "Stay away from things that those who will subsequently look after your code don't understand." You may be the best programmer in your entire organization, but all that means is that whoever takes over the maintenance of your code won't be.

But, yes, it's also true that you shouldn't be using any tool unless you understand what it does and how it's likely to malfunction. Using the production code of a major project as a testbed for teaching yourself threads or ties or coroutines or OO or closures or continuations is generally a very bad idea.

TPR: So can "best practices" for an individual evolve as they understand more of the language?

Damian: Most definitely. For example, I offer several guidelines that are based on using closures. Now, if you don't know what a closure is, those guidelines clearly aren't going to work "best" for you.

As a specific example, although the book strongly suggests using the "inside-out" technique for implementing classes (Chapters 15 and 16), some less-experienced Perl programmers will probably find the interplay of object data distributed across multiple variables, the use of references as keys, and access control via named closures to be individually incomprehensible and collectively unmaintainable. So they would be much better off using ordinary blessed hashes as objects instead, until they have that "Aha!" moment and grasp the power and elegance of the more advanced technique.

A slightly subtler point here is that "best practice" is only rarely specific to an individual. Because very few programmers work as individuals. Most of us develop code in some kind of team structure, and "best practice" has to be what is most comprehensible and efficient for the team as a whole. It's definitely the case that a team's overall level of understanding and experience can evolve over time, in which case what works best for them may likewise change.

That's where I hope this book can also help. Because what doesn't change over time are the issues I'm trying to address: the importance of consistent naming and layout; the care that's required when handling Perl's various kinds of aliasing; the features that produce an intuitive and extensible interface; how, when, and what to document; efficient control structures and I/O handling; and so on.

Far more importantly than "laying down the law", I hope this book "opens up the discussion" and allows programmers at any point in their personal and group evolution to find reliable and maintainable ways of coding.

TPR: Do you think programming practice is more nature or nurture?

Damian: It can't possibly be nature: programming simply isn't a natural act.

And it can't really be nurture either, because most of us simply don't care enough about how we code, and certainly don't try to nurture better programming habits in ourselves.

I think it's more a matter to evolution under various internal and external selection pressures. Most of the things that most programmers do when they program is the unconscious expression of a learned behaviour or a conditioned reflex. As I say in Chapter 1, the style that most developers use...

...will be based on their earliest experiences in programming--the linguistic idiosyncrasies of their first language, the way in which code was presented in their initial textbooks, and the stylistic prejudices of their early instructors.

That style will develop and change as the programmer's experience and skills increase. Indeed, for most programmers, their style is really just a collection of coding habits that have evolved in response to the opportunities and pressures they have experienced throughout their career.

Just as in natural evolution, those opportunities and pressures may lead to a coding style that is fit, strong, and well-adapted to the programmer's needs. Or it may lead to a coding style that is nasty, brutish, and underthought.

But I do think we can nurture well-adapted habits, rather than "natural" ones. Culture can overcome instinct. If we choose to let it.

TPR: Have you noticed different trends in coding behavior between "real" and "accidental programmers", or does everyone have the same problems?

Damian: I'm not sure there are "accidental" vs "real" programmers. If you're a stenographer who is just writing a four-line Perl script to convert line-endings on documents, that's every bit as real to you as a 20-Ksloc-a-year hacker's codebase is to her. Maybe more real.

Of course, the level of experience, the depth of understanding, and range of tools available in those two situations may be dramatically different. But the problems aren't. They're always:

And, of course, six months later:

TPR: Which coding styles have your personally evolved through?

Damian: My first programming language was Pascal, so the style I started with was very structured and formal. Then I taught myself C, so my style became a very structured and formal variant of Kernighan & Ritchie's. Then I rapidly picked up Fortran, COBOL, HyperTalk, Lisp, Prolog, csh, AWK, C++, Eiffel, Icon, PostScript, Miranda, Java, and AppleScript. So by the time I came to Perl my style was already seriously eclectic, possibly even postmodern.

In a sense, that was the original motivation for developing these guidelines. I realized that my own code--written in some weird creole of a dozen distinct language styles--was completely incomprehensible to anyone but myself...and sometimes not even to me! So I set out to reinvent that style, to morph it into an approach that still felt comfortable to me, but also helped other people understand what I was doing.

TPR: Which of your former styles do you consider the most disastrous (in any dimension)?

Damian: I think my greatest stylistic fault has always been my bad habit of letting short, tight subroutines gradually inflate into uncommented, unmaintainable, monolithic monsters; failing to adequately factor out the various subtasks into separate utility subroutines or, at least, not breaking the subroutine body into distinct "paragraphs" to make the subtasks easier to discern.

That's a common failing in many programmers, I think. We're so busy trying to write the code, that we forget that we're going to have to spend 5-to-100 times longer reading it. So we optimize for ease of immediate coding, rather than ease of long-term comprehension. A large fraction of this book is devoted to exploring ways to get past that (ultimately counterproductive) trade-off.

TPR: You stray from, delete from, or add to perlstyle in your recommendations. Does perlstyle, which started as Larry's recommendations and pretty much stayed there, need an update?

Damian: Actually, I think perlstyle stands up pretty well against this book. Or vice versa perhaps, since Larry's advice came first.

I deliberately didn't reread that particular manpage before I started writing, but looking back on it now I see that, of the forty or so substantive suggestions it makes, "Perl Best Practices" only disagrees with four of them. Of course, that may just reflect the inherent similarities in the way Larry and I think about programming, but I'd like to hope that it also indicates that the advice in both documents makes sense.

Of course, the book has over 200 other suggestions as well, some of which probably could be profitably added to perlstyle. But if you had to pick only forty of them, there are very few I'd change there.

TPR: You advocate names that read more like English, but a lot of words have ambiguous meanings. Will there ever be a perfect naming convention?

Damian: Sure. There's been one for the better part of a decade (Lojban), but only a few hundred people in the world can actually understand it.

I certainly do advocate using names that make code easier to read, and I specifically discuss the problem of ambiguous words. Fortunately it's only a very small list of troublemakers that create most of the problems: last, left, right, no, abstract, contract, record, second, and close. And, of course, the single worst offender, set, which has perhaps a dozen distinct meanings related to programming, and well over a hundred in general English usage.

But, if you avoid that last set (ahem) of words, ambiguity generally isn't a big problem.

TPR: Do the best practices change if you are coding in another language, say, well, I don't know, how about Klingon? Or Latin?

Damian: I'd say that at least half of the guidelines in the book are applicable to any programming language. Things like:

These are universally good developer habits, no matter what languages you use.

As for your specific suggestions, Latin is much more free-form than Perl. Because the grammar is inflected rather than positional, you can basically arrange the components of any statement in whatever order you like. That does imply, however, that you need to take a great deal of care with naming variables and subroutines, since they need to be unambiguous in all contexts.

Programming in Klingon is a much more...err...vigorous past-time. I'm sure everyone has seen the The Klingon Programmer's Code of Honour. There's hardly any point worrying about coherent variable naming conventions when the code it doing it's best to kill you. One thing stays the same though: Klingons know that a true warrior doesn't cuddle his elses...he dead-grips them!

TPR: One of the practices involves adding _ref to the end of reference variables. Why not something like Hungarian notation to also denote the reference type? Or did you have to stop the madness somewhere?

Damian: You can safely have one piece of Hungarian notation. But as soon as you have two or more, they start breeding like rabbits and pretty soon you find yourself using variables like $rgddataszMax_ref, and spraining your eyes whenever you try to read it.

I certainly did consider extending that particular suggestion to _aref, _href, _sref etc. and I know several very good programmers who do exactly that. But, for the book, two considerations stopped me.

Firstly, I didn't feel that the extra information solved a real problem. The reason to add _ref to the end of a variable is that it can help you detect and avoid the very common mistake:

     my $samples = get_samples_ref();

     # and later...

     $initial_value = $samples[0];   # Should be: $samples->[0]

use strict usually catches bugs like that, but not if there happens to be a valid @samples array in the same scope. However, if you consistently mark references with a distinctive suffix:

     my $samples_ref = get_samples_ref();

then it's easy to train yourself to see _ref[...] or _ref{...} as a "red flag" (MJD's excellent concept!) that something is amiss:

     $initial_value = $samples_ref[0];

So the suffix is designed to highlight a problem that perl can't always diagnose by itself. On the other hand, the interpreter can always diagnose a problem like this:

     $initial_value = $samples_ref->{first};   # Should be: $samples_ref->[0];

You always get an exception if the type of reference doesn't match the kind of dereferencing you're doing:

     Can't coerce array into hash at line 42.

So adding a type specifier to the reference marker solves a problem that is already solved automatically. Not that there's anything wrong with that kind of redundancy. As I said, several programmers whom I deeply respect do find it useful.

So I probably would have suggested it in the book except for one problem that kept cropping up when I tried adopting the idea myself: I kept misinterpreting the _sref suffix. Hungarian notations like this only work if they're unambiguous, but I kept using _sref ("scalar reference") when I meant _cref ("code reference"). In other words, I kept thinking of _sref as "subroutine reference".

So I only recommend _ref, because the type-nonspecificity of that suffix forces you to think about what kind of reference you're using, and the interpreter still catches any errors that creep in if you don't.

TPR: Some of your advice is likely to provoke fist fights. What's the best practice when reasonable people disagree over something such as punctuation?

Damian: The easiest way to resolve these kinds of honest disagreement is to fire one or both of the people involved.

Seriously though, it certainly is the case that a few of practices suggested in the book have very reasonable alternatives (which I usually discuss). Often the "best" practice in such cases depends on what's most important to you: efficiency, maintainability, or robustness. In such cases, each of the disputed alternatives probably represents an optimal solution when these three criteria are weighted differently.

So the way to move forward is to decide in advance what your "best-ness" criteria are, and how important each of them is. In other words, to specify your metrics before you start measuring.

Another important point here is that the most heated arguments are usually over the least important issues. For example: where to place block delimiters, or whether to cuddle an else, or how many columns to indent nested code.

The solution in such trivial disputes is to get everyone to acknowledge that their arguments are largely "tribal" or "religious" in nature ("K&R was good enough for my Granddad..." or "Larry uses four-column indents!"). Then get them to agree that it doesn't really matter whether everyone uses K&R or BSD style bracketing, cuddled or uncuddled else, or 4- or 8-column indents; what matters is that everyone uses the same bracketing, cuddlosity, and indentation.

At which point you simply choose: either by flipping a coin, or by taking turns to make "draft picks" for these minor decisions, or by sticking with the defaults provided by your pretty printer, or simply by accepting the suggestions in my book.

TPR: When can you break your own rules?

Damian: Any time you like! This is Perl, after all, not Fascism.

The real question is: when should you break the rules?

And the answer there is: only when the following the rules would result in code that's less robust, less efficient, or less maintainable.

For example, I strongly recommend using exceptions (and the excellent Exception::Class module) as your standard error handling mechanism. Exceptions are a great deal more reliable than special return values, because it's much harder to ignore an exception (you actually have to write code to do so) than it is to quietly disregard return value (which happens automatically in void context).

But if you're running some long automated process, like a daemon or a complex batch computation or a heavily threaded application, then a single uncaught exception can ruin your whole day. If uptime is your number one priority, you might choose to avoid throwing any exceptions anywhere in your system, so that there's no chance of an unexpected error killing your entire process. Of course, your code would become much uglier, as you'd then have to manually check every possible error in every scope, but at least you'd be confident that no unanticipated problem would be instantly fatal.

The whole point here is that one word: "choose". The guidelines I suggest are meant to function as reliable and safe default behaviours, as a set of good habits that avoid common problems and improve the resulting code. They're not meant to be seen as natural laws or supernatural Commandments. They should help you, not constrain you: guard rails, rather than cell bars. To use a different metaphor: to be an artist sometimes you do have to draw outside the lines; but if that's all you ever do, then you're just scribbling.

TPR: Some of the "Patterns" people also like to talk about "Anti-patterns", or "Worst Practices". Is that your next book?

Damian: No, though some might argue that it's the bulk of my CPAN directory. ;-)

Practically speaking, I hope this book addresses the dark side of programming too. The vast majority of the guidelines in the book don't just show a right way to do it, they also show one or more of the wrong ways. And, more importantly, I hope, explain why they're wrong.

That's essential, in my view. I'm asking people to change how they write code, so the onus is on me to both offer a better alternative and demonstrate why their current practices are flawed. Often the best argument is simply to take existing coding habits and extrapolate them to their inevitable unpleasant conclusions.

TPR: What would you advise those people who want to write a guide for their own team?

That's easy. What I do advise at the very start of the book is this:

As you consider these pieces of advice, think about each of them in the context of the type of coding you typically do. Question your current practice in the particular area being discussed, and compare it against the recommended approach. Evaluate the robustness, efficiency, and maintainability of your current coding habits and consider whether a change is justified.

But remember that each of piece of advice is a guideline Whether or not you agree with all of them doesn't matter. What matters is that you become aware of the coding issues these guidelines address, think through the arguments made in their favour, assess the benefits and costs of changing your current practices, and then consciously decide whether to adopt the solutions offered here.

Then consider whether they will work for everyone else on your project as well. Coding is (usually) a collaborative effort. Developing and adopting a team coding style is too. Mainly because a team coding standard will only stay adopted if every member of your team is willing to sign off on it, support it, use it, and encourage other team members to follow it as well.

Use this book as a starting point for your discussions. Negotiate a style that suits you all. Perhaps everyone will eventually agree that--although their personal style is self-evidently superior to anything else imaginable--they are nevertheless graciously willing to abide by the style suggested here as a reasonable compromise. Or perhaps someone will point out that particular recommendations just aren't appropriate for your circumstances, and suggest something that would work better.

Above all, keep in mind that the goal of any coding style is to reduce your development costs by increasing the maintainability, robustness, and efficiency of your code. Be wary of any argument--either for or against change--that doesn't directly address at least one of those issues.

A review of Perl Best Practices appears in the Summer 2005 issue of The Perl Review.