Interview with Mark Jason Dominus
Mark Jason Dominus has written several Perl modules, has written for Perl.com (which he used to edit), PerlMonth, and The Perl Journal. Mark's book, Higher Order Perl, was recently published by Morgan Kaufmann and is on sale now.
The Perl Review: Why did you write Higher Order Perl?
Mark Jason Dominus: My main reasons were as I explained in the preface of the book: It had become increasingly clear to me that most Perl programmers seemed to be writing C programs in Perl, and that it was possible to do a lot better than that. We have a really good tool, and it's a shame that we're using it the same way that we have to use the old crappy tool.
But I had a few hidden agenda items as well. The book is about how to use closures, and I think there are too many languages around that have no easy way to use closures. Java, in particular, seems to me to be a giant step back towards the languages of the 1970s. My first draft of the the preface had a long screed about how back in the 1950s, recursion was unavailable in common industrial programming languages, such as Fortran, and if programmers understood recursion at all, they thought that it was an unnecessary, inefficient, ivory-tower technique. But eventually languages like C and Pascal appeared, and if you tried now to popularize a recursionless language, people would laugh at you.
I think the same process happened with automatic garbage collection. There, Java was the big turning point, and I think that in another ten or fifteen years nobody will bother much with languages that have no automatic storage management.
I think the same thing is happening with closures, but we haven't gotten as far. It's still possible to promulgate a language like Java that doesn't have practical closures. At the time I started writing, Python didn't have closures at all and I heard Guido van Rossum say that they weren't important. I think that's wrong, and that in another thirty years people will laugh at anyone who tries to invent a language without closures, just as they'll laugh now at anyone who tries to invent a language without recursion.
TPR: At the start, you compare Perl, Lisp, and C and say that Perl is more like Lisp than C. Do you think Perl would be as powerful if it was closer to C? Could it be more powerful if it was closer to Lisp?
MJD: I don't think anyone thinks that Perl would be more powerful if it were closer to C. In the 1980s, Unix sysadmins wrote in C and in Bourne Shell, and Perl has largely replaced both of those for those sorts of tasks, leaving C a fairly small niche.
In the preface of Higher-Order Perl, I said that Perl shares six of Norvig's "seven features that make Lisp Different". The seventh one is the one that Norvig calls "uniform syntax". That's the one that led Larry to say that Lisp looks like oatmeal with fingernail clippings mixed in. But it enables language features that are so powerful that they're almost unimaginable to people who haven't studied Lisp. It's really hard to do justice to the huge benefit that Lisp gets from its simple and uniform syntax, which Perl just doesn't have at all. Any attempt to construct Perl code dynamically is fraught with error, and we take it for granted that it won't work. For example, in the documentation of the Perl6::Subs module that was recently released, Chip Salzenberg says "This module is a source filter. Source filters always break." And I think that's true. But source filters in Lisp are easy to write, and they never break. Lisp's main assignment operator, "setf", is implemented with a source filter. So if Perl were more like Lisp in that sense, I think it might be more powerful. On the other hand, I don't think it would be very Perl-like.
TPR: What's your relationship to Lisp? Do you still use it?
MJD: Occasionally. But I find Haskell and SML more attractive, for various reasons. One is that the Lisp user community is severely dysfunctional. Another is that I find the strong typing of Haskell and SML fascinating.
TPR: Do you wish Perl was more Lispy or Lisp more Perly?
MJD: Neither, really. I like having more than one language. Sometimes I wish Perl's syntax were less verbose.
TPR: Did you learn more about Lisp as you worked on this book? Did you learn more about Perl?
MJD: The book took five years to write, and I learned a lot about a lot of things in five years.
TPR: Readers learn a lot about functional programming from Higher Order Perl. Do you think programmers should still learn Lisp or some other functional language?
MJD: First I need to put in a disclaimer that I don't think Lisp is a good example of a functional language and I don't think most Lisp programmers would consider it one. "Functional language" suggests a firm philosophical stance against side effects. You can see this stance in true functional languages, like Haskell, where all side effects must be quarantined in a special kind of data structure called a "monad". I think most of the Lisp world doesn't care about that stance.
That said, I do think that all programmers should learn as many different languages as they can, and I'm puzzled when they don't want to. It's like being a carpenter and saying that the hammer and the chisel are good enough for you, you don't need to learn to use a saw or a screwdriver.
The best thing about going to learn something new is that you never know what you're going to find out. HOP came about because I discovered all these techniques that work well in Perl that I had never seen done in Perl before. It's hard to think these things up by yourself; it's much easier to borrow them from someone else.
Programming communities tend to be isolated. We all know the same languages, we all read the same books, and we all learn how to solve the same problems in the same ways. If you want to be able to solve the problems that have stumped the people around you, you need to know something they don't know. One way to do that it by knowing how things are done in a different community.
If you're going to learn a new language, Common Lisp would be a good choice. So would Haskell. They're both powerful and useful, and they'll get you thinking about things in a different way than you're used to.
TPR: Are you surprised that after a couple of years of working on a book about functional programming in Perl, Autrijus Tang has lit a fire under the community by using Haskell to write a Perl 6 compiler?
MJD: Everything Autrijus says and does astonishes me.
TPR: A lot of the techniques in the book seemed like it was geared to programmers writing things for other programmers, such as tools or modules. Was that your intended audience?
MJD: Yes, and I think I said as much somewhere early in the first chapter. Part of that is my own bias, since I'm just not very interested in user interfaces. But I think most programmers are involved in writing code for other programmers, rather than for end users. And almost all programmers end up having to put programmatic interfaces on most of their code, because that's the only way you can write regression tests for it.
TPR: When you discuss memoization and caching, you say that the ability to do this easily (as with your Memoization module) should be a major point of Perl evangelism. What other major points of advocacy can we get out of Higher Order Perl?
MJD: Memoization is a great example because everyone anyone can understand it in five minutes. If you have more time to explain, I think the overloaded parser stuff from chapter 8 might be a good thing to try. Writing functions and operators that combine simple parsers into complex ones is a really useful technique that should appeal to almost anyone who has ever had to write a parser.
TPR: Why did you choose Morgan Kaufmann as a publisher? As far as I can tell, this is their only Perl book.
MJD: I think it is their first Perl book. I chose to publish with MKP because I felt that there was a really good personality match between myself and the people with whom I would be working. I'm really glad I did; it worked out very well.
TPR: Did you design Higher Order Perl as a possible textbook?
MJD: No, just the opposite. I was terrified that people would see it as an impractical academic treatise with no relevance to professional programmers. I tried hard to rid it of any academic aroma. So, for example, there are no exercises at the end of the chapters.
The publicity people at MKP have asked me if I think it could be marketed as a textbook, but I have a lot of trouble imagining what class it could be used for.
TPR: Did your idea of the book change a lot since you first started? What happened along the way to get the book to its final form?
MJD: No, it came out pretty much the way I planned it. Even the table of contents isn't too far off from what I originally proposed back in late 1999.
The only big surprise was that in several places I found that there was a lot more to say than I had anticipated. For example, the chapter on iterators was at one point about 160 pages long. I just kept digging deeper and deeper. I cut some of it out before the book went into production, but most of it is still there.
TPR: As you wrote this book, you gave out chapters on a semi-public mailing list. How did that work out for you? Would you do it again?
MJD: That appears to have been a really outstanding success, and I will absolutely do it again. When you circulate the drafts to six hundred people, some of them read those drafts really carefully, and you get excellent early feedback. For example, there's a section in chapter 6 about sorting a log file into a different order while it's still being written. That's there because Dave Turner said he found my other application of cutsorting unconvincing and abstract. I decided he was right. There's a big chunk of chapter 9 that I rewrote because Greg Bacon sent me a long letter asking for clarification, and it's much better as a result. That's the sort of thing that you hope you'll get from the official technical reviewers, but you hardly ever do.
TPR: Higher Order Perl is (or will be) free on your website. Will it always be the published version, or will you update and add to the content?
MJD: I'm not sure yet. One idea I've had was to turn the web version into a wiki, with a few paragraphs on each wiki page, and let people discuss the book, and add clarifications and criticisms on the web site.
TPR: Giving away your book is a huge gift for the community and represents several years of your personal time. How do you hope people give back? Tipjar? Speaking engagements? Free dinners at conferences?
MJD: I hope people will buy copies of the book to keep. MKP is nervous that having the book on the web site will hurt sales. I don't think it will, but I'd sure hate to be wrong.
Since you asked, though, I would like to put in a plug: I used to earn my living by traveling around and teaching Perl programming classes. I couldn't have written the book if I had had to hold down a regular job. And not having a regular job also made it possible for me to visit conferences and user groups and give talks for free. So if people want to support me, they might consider getting their company to bring me in to do some classes.
TPR: In the past, you have expressed strong feelings about Amazon.com. Where would you like people to buy your book? Do you have an affiliate link?
MJD: I'm still upset about Amazon's ridiculous "one-click" patent, and I'm still boycotting them. Software patents like Amazon's threaten the livelihoods of all programmers, including mine, and I don't want to give my support to a company that means me ill. They dropped their suit against Barnes & Noble, but the terms of the settlement were never disclosed, so we don't know what happened, and Amazon could turn around at any moment and sue anyone for violating their patent. So the issue is unresolved.
I'd like people to buy my book from anyone but Amazon. My web site has an affiliate link to either to Barnes and Noble or to Powell's Books, depending on who isn't out of stock at the moment. I get a small kickback from this, but I really don't feel strongly about it. I do feel strongly about not supporting Amazon. There are a lot of places to get books online; it's not hard to buy books from a company with less abusive business practices than Amazon.
A review of Higher Order Perl will appear in the Summer 2005 issue of The Perl Review.