I'm a compiler geek, with a slant towards scripting languages.
I work on the Javascript team at Mozilla. Before that I did a PhD in compiler optimizations, static analyses and scripting languages.
I’ve worked on phc since 2005, and been its maintainer since 2007. I wrote the optimizer, and nearly everything performance related.
I had mixed reactions upon hearing about the release of HPHP [1], the new PHP compiler from Facebook. There are a few aspects to this so I’ll start with the technical stuff. I always love the social aspect, so skip to the bottom if you like whining and tears.
I don’t know the answer to this. I haven’t seen anyone even mention PHP’s references, which are incredibly gnarly for a static analyser [2]. HPHP might just ignore it, which isn’t necessarily a bad idea. I’m wary of ignoring edge cases, as they tend to interact in horrible ways, but I guess Facebook already run all their code off it, so it can’t be that bad.
In general, I’ve found that ignoring the edge cases is bad when compiling PHP. There are a million of them, and they all interact. They interact worst of all in static analysis, because you have to consider all possible paths. Its the sort of thing where if you nail it 100%, then you have something amazing and widely applicable, so that’s what I was aiming for with my PhD. I suspect HPHP doesn’t consider all paths, and makes all sorts of hacky assumptions. [3] This is probably a really good idea. I did the opposite in the optimizer, and the result is instead immature and slow.
Facebook said HPHP reduces by half their number of servers. PHP’s libraries are already written in C, which gives it the appearance of being fast, even though the interpreter is dog-slow. This implies that HPHP-compiled PHP code is much more than twice as fast as the PHP interpreter. This probably means HPHP is way faster than phc‘s compiled code as well.
I saw some criticism for not building a JIT on LLVM. But:
PHP is not like other dynamic languages. Duck-typing is possible, but most of the community best practices come from Java (along with the class system), and so its not used that much. Monkey-patching — switching out classes and methods from objects at run-time — isn’t possible, except with the hackiest of hacky unsupported extensions. Dynamicism in PHP tends to involve templates instead, like with Smarty. If you want to analyse it, then you just need to run it a few times, get all the templates to be instantiated, and compile all the generated PHP code. I’ve started calling this "deployment-time analysis", since in server-side, you probably know all the code you’re going to compile at deployment-time. So a compiler is a perfectly reasonable approach for PHP, and a JIT is probably not needed.
People seem to want to know if HPHP is widely useful outside of Facebook, and some people are saying "no". I disagree strongly. In order for HPHP to be useful, you need to have a PHP application which is suffering due to PHP interpreter performance. That matches Facebook perfectly, and they’ve always been the canonical example I use to explain why PHP compilers are interesting. But you don’t have to be Facebook size or scale to have performance problems.
I’ve heard the argument "you don’t need a compiler, since PHP is rarely the bottleneck" for many years. I think its complete bollox. But I wrote a compiler for PHP, so I would say that.
Unless your PHP server is sitting there idling (which is probably the case for many PHP servers out there), then you could make use of a PHP compiler. For small timers, all components of your application are going to be sitting on the same box, contending for the same resources. Even if you assume the DB is the bottleneck, the resources the interpreter consumes could be more profitably spent on the DB.
The PHP interpreter is also quite memory hungry, as interpreters go. Any PHP value in your program uses 68 bytes of overhead [6]. An array of a million values takes over 68 MB. If HPHP is able to convert your million value array to native C types, it will only take 4MB. I’m sure your caches could make good use of that savings.
However, optimization isn’t only about speed. The main value is that they give you freedom in how you code.
There is a meme in the scripting language communities that {PHP,Ruby,Python,Perl,etc} are "fast-enough". If you need it to go faster, then you should take your hot-loop and rewrite it in C. HPHP will free you from such concerns.
You should consider also that PHP is considered relatively fast. Its not — the interpreter is dog-slow — but programs written in PHP are typically not that slow. This is because most of PHP’s huge standard library is written in C, with a thin layer of PHP over it. Anytime you call a string function, your PHP C string is passed into the C library, the pointers are manipulated and the bits are twiddled, and then it’s handed back to your code. Its a bit like driving in America: it takes a few minutes to get on the freeway, but once you’re on it you’re there in no time.
This is not necessarily a good thing:
Believe me, if your entire application just ran PHP interpreted code, it would not be fast at all. But people who write PHP functions and libraries don’t want to write C. They like PHP, are productive in it, and any time spent arsing around in C is wasted when there’s a website falling apart and a long list of features due yesterday. HPHP will free you from such concerns.
Compilers also provide other niceties. You don’t have to unroll your own loops, or move constant expressions out of loop headers. I don’t know if HPHP supports these, but I’m sure it could.
Allowing your existing code to go faster is hardly the point though. Really, the point is that you can do more in less time. Suppose you’ve decided that your application needs to response to the user in 500ms. The DB takes 200ms, the request takes 200ms, the framework takes 50ms and your code only has 50ms to run [7]. That’s quite a constraint. This leads to people using PHP as a simple templating layer, instead of as the Turing-complete langauge it is. I expect we’ll hear a lot more about HPHP, simply because of how freeing it is to the user.
So even if you only have a small VPS, instead of massive server farms like Facebook, you’re likely to find a use for HPHP. I’m sure shared hosts will set it all up soon for their users, and everyone will be happy.
A better question is, how widely applicable is a compiler which doesn’t support all of PHP’s dynamic constructs? Funnily enough I did a bit of research on this. We chose the opposite tactic for phc: trying to stay 100% compatible with Zend, all the extensions (even the 3rd-party, unpublished, top-secret, proprietary ones), etc. You would expect we would first research whether this was useful? Well no. We built it, and then I did some analysis to check whether it was useful. You can read it in depth [8]. Basically, I downloaded 700 packages off sourceforge, and wrote some phc plugins to check for evals and dynamic includes (a dynamic include uses a variable as its parameter, instead of a constant or literal).
Result: 40% of PHP packages use dynamic constructs. Now this isn’t quite as scientific as it should be. Lots of those programs were old, and styles have changed. eval is discouraged these days, but it’s probably still used, if only to get around the weaknesses of the PHP parser. In particular, this doesn’t imply that HPHP is somehow fatally flawed for not supporting dynamic constructs. It just means it might not be so useful to you, the common PHP programmer.
An easy way to get round dynamic includes is to just consider the PHP files in the directory structure. There was a good research paper by Wassermann on supporting that, but I find it very hard going, so you probably will too. Still, a naive approach is to just stick a switch statement in, and compile everything that makes sense. This is how you would deal with things like WordPress plugins. It does mean that if you change your plugins, you’re just going to have to recompile. If you’re using a compiler, I doubt you would find that a problem.
But not everyone is happy with this new compiler, such as, well, me.
Lets start with a quick whine.
You would think they could at least invite me to the party. Those bastards.
More seriously, I actually was annoyed at all the news reports about HPHP, principally because they were largely bullshit. And I knew it, and I couldn’t say more because I told Facebook I wouldn’t. There is very little more irritating than idiots being wrong on the internet, and the news stories brought out thousands of them! Reddit and Hacker News were literally covered with stories about HPHP. Hundreds of trolls emerged from under their bridges, not knowing the difference between a bytecode-based interpreter, a caching PHP accelerator, and a native compiler (which is fine, until they start saying they’re all the same). Think about it now still makes me angry.
I’m also slightly annoyed that people all of a sudden care about PHP compilers. I worked on one for 4 years and I could not convince anyone to give a shit. But now that its got the Facebook logo on it, all of a sudden PHP compilers are the greatest thing ever. Bah.
One saving grace is that they didn’t patent it. I have an email in my inbox from one of the HPHP developers saying he couldn’t talk to me about the compiler because they might patent it. That’s pretty shitty. Thank god they open-sourced it instead [9]. It sounds like it was a bit touch and go for a while.
Which bring us to the question of whether they should have used phc?
Obviously it would be great if they had used phc, and I’m not privy to the reasons they didn’t [10]. The design decisions we made in phc were aimed at maximum compatabilty, and the performance suffered [11] as a result. The optimizer was designed to solve these problems, and I believe it would have, but it is not mature enough now, and was still a twinkle in my eye when HPHP started two years ago.
Facebook was solving their performance problems, not building a PHP compiler for general use. If they were doing the latter, it would be much easier to criticise their approach, but for now I can’t say I would have advised them otherwise. On the other hand, they probably didn’t need to build their own parser – its a tricky problem and phc‘s parser and front-end are excellent. Had they gone another way, then they could probably have started to use phc‘s optimizer [12], which while immature and slow to compile, is pretty state-of-the-art and has great potential (if I do say so myself).
A better approach would probably have been to hire all the programmers who worked on PHP compilers, to get that expertise in house. They did try to hire me, but only recently. I’m honestly surprised that they haven’t tried to hire the Shannon Weyrick, who is currently working on rphp, his second PHP compiler [13].
When they annouced HPHP, I would have said it was phc‘s death toll. The original phc authors, Edsko and John, have moved on to other projects, and I’ve run it mostly solo for about two years. But I havent worked on phc in about 6 months, and my hatred of PHP makes it unlikely I will again. My requests for new contributors to step up has fallen on deaf ears, and my summer intern hasn’t decided to take over either.
Since no-one wants to take on the compiler, the new competition from Facebook should probably kill it, right? Maybe not. Over the last week, traffic to the phc website has increased by five times [14]. Facebook has unleashed some sort of latent interest in PHP compilers that I haven’t been able to extract from people. So perhaps this might be the rebirth of phc, not its death.
And phc is better than HPHP in some ways. HPHP is almost certainly faster because they didn’t have to deal with eval, dynamic stuff, and because they don’t use the Zend libraries. But phc was specifically designed to work with the Zend libraries, with eval, with everything. So it’s probably a better fit for most projects than HPHP.
If you want to take over phc, then join the mailing lists, download the source code, read the death notice and contribution page, and email me for commit access.
phc will likely live on anyway. The front-end is pretty slick: Facebook ran it over their million lines of code and only had one or two problems. It gives a lovely AST to allow all sorts of code transformation tools, has a nice plugin interface (for C++ lovers) and an XML interface (for the rest of you), and will spit your code out largely as you put it into it. Its certainly the most mature and well tested part of the whole project.
The optimizer is pretty slick as well, but in a different way. I’m pretty sure its the most advanced static analyser for PHP, and it’s waiting to be put to good use. That said, it’s damn slow, and not mature (read: pretty buggy), and itself doesn’t support eval and dynamic includes (surprise!!). The optimizer is waiting for some love — I could imagine it making a pretty nice "automatically find out what types your function may be passed" kind of linter.
Otherwise, phc will only live on as part of the Roadsend Raven compiler. I understand that they’re going to take the optimizer and the parser from phc, and that will be really interesting.
Finally, what does it mean for me? Well, I’ve left that ship already. I’ve hated PHP for a long time, and have no desire to go back to it. I’m doing a startup now, but when I go back to regular employment I will be looking for another scripting language run-time. There are plenty to choose from, in particlar Unladen swallow and TraceMonkey. Mozilla looks like an amazing place to work, so I think I’ve worked out my backup plan.
