⇥ RIAs in the (small) enterprise

November 30, 2007
3 comments
 
⇥ Permalink

One of the most critical decisions that we had to make when redesigning our entire technology stack was how to replace the backend systems that we use internally for managing our business. As I mentioned in my previous post, much of what goes on behind the scenes at php|a is automated, and much is handled through a management interface that our staff uses for purposes ranging from customer support to getting a new issue of one of our magazine up on our website.


Although we are a very small company, we are very distributed—with people working in as many as ten different locations at the same time, we have to deal with some challenges that most small businesses rarely encounter. One of these is having to support a wide range of operating systems—most of our dev team is on OSX, and most of our customer care team is on several versions of Windows. The other is having to deal with a curious mix of requirements: access live data securely, comply with merchant security standards, print well-formatted documents (such as invoices), manage customers, upload all sorts of creative, handle news posts, and so on.

Our old system used an extranet for all these tasks. This worked well for the longest time, but the entire concept was beginning to burst at the seams; printing from a web browser was never easy, for example. File uploads are, for lack of a better term, painful to handle. In fact, the entire in-browser user experience doesn’t lend itself well to operations that require referencing multiple information screens at the same time (say, editing an invoice while looking at a customer’s record). Naturally, it is possible to visualize all the relevant data in the same screen, but then you have to deal with things like information overload and, well, the fact that you are presenting the same data in more than one way (which, in my opinion, is a major no-no of application design).

From a strategic perspective, a “classic” web-based application (of the non-Ajaxy variety) looks easy to maintain on the surface—but a simple cost/benefit analysis quickly reveals problems like decreased productivity and increased development costs when your programmers are trying to work around the browser’s limitations in order to give their users the tools they actually want—not to mention the cost in terms of mounting frustration on both sides.

Looking for a better solution
When we started redesigning our technology, one of the big decisions that we knew we were going to have to make was replacing the management systems. Having decided that a plain-old website wasn’t going to do the trick, we eventually narrowed our choices down to four possible alternatives:
  • Native Application
  • RIA
  • XUL
  • AIR/Flex
I have to admit that the native application was, for a short while, the front contender. There was a period of time when almost the entire company had (by accident) standardized on OSX, and applications on Macs are both a pleasure to develop and a pleasure to use. On the other hand, none of the developers had strong skills on OSX (or, for that matter, on any other operating system natively) and, of course, locking ourselves into a particular operating system seemed like a potentially expensive thing to do—what would happen if we were to hire someone who didn’t use OSX? We’d either have to purchase a new Mac, or scrap our application. Worse yet, there’s a good chance that we’d get used to the OSX app, so that replacing it might prove to be impossible (at least without inciting a riot), and we’d end up having to maintain two different applications.

RIA schmia
The other almost-natural choice was some sort of AJAX-powered Rich Internet Application. We never seriously considered this solution, mostly because we thought that it would bring all the headaches of a GUI application without most of the benefits. AJAX is not a technology—and, therefore, there is no unified toolset on which we could standardize. For example, I use Safari almost exclusively, Sean prefers Firefox, Paul is known among our team as “the guy who files all those weird Opera bugs” and most of our customer care team is on IE.

Of course, we could force all our users to access our system using a particular web browser—but that would be silly; browsers are notorious memory hogs, and the level of complexity required by an AJAX application is only going to make things worse (just do a quick search for AJAX memory leak and you’ll know what I mean).

Finally, having dealt with browser quirks far more often than any of us cared to remember or admit, we weren’t about to get tangled into a web—pardon the pun—of badly-supported standards and inconsistent behaviour. AJAX was out.

Gozer the Gozerian
By the time we started considering building our application using XUL, we were fast approaching the level of desperation that was bound to drag us all the way back to our plain-old-web solution.

I will spare you the weeks of misery, but the bottom line is that XUL is an acceptable solution only under three circumstances:
  1. You like pain
  2. You are writing Firefox (or another Mozilla project)
  3. You employ someone who fulfills (1) and (2).
Unfortunately/luckily, we didn’t fit any of these categories. XUL is nowhere near ready to be used in a solid application outside of Mozilla unless you can get your hands on a good developer who has already been exposed to the technology, and those are hard to come by. Documentation for the project is sparse (and that’s being very generous), and the end result is just as much of a memory hog as Firefox.

Adobe’s Flex was the last technology we looked at—almost as an afterthought. We had, of course, heard about it (although at the time Adobe had not quite put as much of an effort into pushing it), but our combined prior experiences with Flash and Macromedia held us back considerably.

Then, something interesting happened: Sean rebuilt what it had taken him three weeks of XUL research and development in a little over one day using Flex 3 beta. I really wasn’t expecting anything nearly as quick—or as successful—and this made us take notice.

Flex and AIR are interesting technologies. They seem to have been designed with developers in mind in exactly the way that HTML/CSS and Flash were designed for… somebody else. Like XUL, Flex lets you lay controls over a form in a logical fashion without having to be too concerned about hacks, but unlike XUL, Flex is well documented and has a powerful IDE that is on par with the latest Visual Studio or XCode. Like Flash, Flex is based around a technology that is (relatively) lightweight and can be programmed using a language most developers are familiar with—ActionScript is, of course, very similar to JavaScript—without significant sacrifices in terms of performance. To give you an example, as part of our original platform tests, we built a prototype VNC client and a semi-working IRC client. These are both examples that have complex requirements (the IRC protocol is simple, but one does need to display the output in a pleasant way and provide a good interface for the user) and, at least in the case of the VNC client, are very intensive from a processing standpoint.

Coupled with AIR, Flex allowed us to build (in a matter of a few weeks instead of months) and deploy a whole new management system that can be installed (and developed) on any of the platforms we work with. Even in its current beta status, and with all the shortcomings that come with it, it is a very powerful development environment that combines most of the “cool” features of web applications and client-side GUI software in a package that uses technologies a wide range of developers are familiar with.

As testament to how good this approach to software development is, consider what happened last week, when we finally deployed our management console company-wide. Despite the fact that very little work had been put on creating a “visually stunning” interface, everyone, from customer support to shipping to the members of the dev team who had not actively developed it, was able to install the application and learn how it works in a matter of a few minutes.

Granted, we don’t do anything super-fancy. For the most part, the application fills forms and exchanges data with our service layer (powered by a super-simple, JSON-driven interface). However, this is true of most applications that can benefit from a technology like Flex—and, frankly, I wish Adobe would place more emphasis on this fact, which would open a significant market to them. Being able to develop MP3 players, word processors and time management tools with Flex is great—but the vast majority of potential users of the technology are more likely to have problems of the create-an-invoice type that need solving, and Flex is passing them by while they bang their heads against a wall trying to write HTML and CSS that makes printing decent documents possible (good luck with that). I will bet any amount of money that a developer can build a better interface with Flex—even without using the visual IDE—in a fraction of the time than it would take him to do the same in HTML, with the added bonus that the resulting code would be significantly easier to maintain.

The Silverlight at the end of the tunnel
You have heard me and my colleagues push Flex quite strongly of late. Naturally, Adobe is a client of ours, and so you’ll have to take what we say with the due grain of salt. At the same time, our enthusiasm is not fueled by sponsorship money, but by a genuine appreciation of what Flex has done for us, and by the fact that we believe that technologies like it are going to have a big impact on our industry if we give them an opportunity to.

Flex has the great benefit (for us) of being completely cross-platform—and its key components are even open-source (which, curiously, comes in very handy when you have to fix bugs in the framework). It is, however, not the only platform that is taking web development in the direction of sidestepping the browser in favour of an environment that is more conductive to a rich user experience. If Windows is your platform of choice for development, Silverlight may be a viable alternative, particularly if you already have the .NET experience required to take advantage of it (of course, Microsoft is also our client, so you’ll have to take that with a grain of salt, too!). 

Regardless of the tool, what you really can’t afford to do is ignore these technologies—if your business needs computer, they bring huge value to the table.

⇥ php|architect’s new clothes

November 26, 2007
15 comments
 
⇥ Permalink


As you may have noticed, php|architect has a new website to replace our five-year-old ball of code. This change has been almost eighteen months in the making, and one of the main reasons behind my general absenteeism from almost any form of non-necessary social activity during that period. If I haven’t answered your e-mail, returned your phone call or otherwise ignored your existence, it wasn’t on purpose—a man’s brain only has so many CPU cycles.


Here, therefore, are a few observations on our new platform, its development and our strategy. Consider this the “how we did it” piece, with its “why we did it” companion to follow in a future post.

What’s in a website?
I am constantly surprised by how many people don’t seem to realize how complicated the technology behind php|architect really is. By the time we replaced it last week, our old system consisted of over 3,000,000 lines of PHP 4 code responsible for everything from backend management to distribution to online sales.

Even before it became that big, the entire application started showing signs of code rot—I have never made any secret of the fact that php|a grew, at least initially, without the benefit of any great strategic plan, and that was reflected in the way our code was written. Five years of functionality added only in response to immediate needs eventually creates all sorts of problems—the same functionality repeated over and over, monolithic pieces of code difficult to maintain and impossible to debug, and so on.

Eventually, the cost of adding new features—always a requirement of remaining competitive—becomes to be comparable with the benefits that these bring, and the business starts suffering.

Development Unstrategy
Arbi and I first started discussing a rebuild of the site back in 2005, when it was still much smaller and not quite the incredible bowl of code it would eventually become by the time it was replaced. The first difficulty we encountered was decided what the website was supposed to do—and, by extension, how the company itself was supposed to present itself. Were we a publisher? Or a training provider? Perhaps a conference organizer?

You can see that, eventually, we decided that we are a knowledge company—but, believe me, the road that led us to this decision was long and torturous. I will only say this: at some point, I had managed to convince myself that our homepage should work much like a web-based OS. I am glad the rest of the team fed me just enough rope to eventually hang myself and realize what a horrible idea that was.

As our work progressed, the site went through many different incarnations. Throughout, we kept what I like to think of as a “fluid set of specs” that changed as we were trying to assess exactly in which direction we wanted to take things. This may look like a poor way to build a piece of software—and it is certainly inefficient compared to having a solid strategy and running with it—but it did allow us the flexibility of experimenting and researching what the best way of doing things was for us without ever boxing ourselves into a bad decision we couldn’t get out of. In other words, it took a while, but we eventually came up with something that reflects our needs: it provides us with a solid base on which we can build our future (sounds lame, perhaps, but it’s true).

The Software Dilemma
I suffer from episodic “rejection periods” during which I become particularly hostile to a certain idea. For a while, for example, I couldn’t stand working with databases; therefore, I became obsessed with the idea of storing and organizing data in the filesystem. At some point, I even became really fed up with PHP and decided that we should look at Python as our platform of choice for web development.

In the end, however, we did use databases, and we did use PHP 5. After all, it is a really solid platform, and most of our expertise lies with PHP anyway. Mucking around with other technologies, however, gave us some great ideas on how to implement certain functionality in the best possible way.

We started by developing our own framework. We stayed away from existing frameworks for a number of reasons—primarily that our needs are so unfortunately unique that we would have only been able to use a small portion of any other system anyway, and what good is a system that can do 100 things “well” when you only need four or five done “perfectly?” Similarly, I dislike the idea of loading up a million classes to perform one operation, and I find it an extremely odious way of developing software. Therefore, we established a single, simple rule: no more than three levels of inheritance. This may sound crazy, but it’s helped us keep our code nice and shallow with no repetition whatsoever, resulting in high performance and a very simple execution model that can be easily debugged.

Next, we needed a frontend templating solution. Over the last few years, I have been particularly partial to XSL, and all of the new system uses XSLT exclusively for the frontend presentation layer (as well as some e-mail work). I cannot stress enough just how good this has turned out to be for us. XSL transformations are powerful but limited, thus forcing the developer to keep the frontend layer and the business logic (or the view and the controller if you live in MVC world) completely separate. PHP 5 has some incredible XML and XSL manipulation libraries, and the end result is a system where the frontend developer is free to much around as much as he wants without encroaching on the backend developer’s turf—all the while using skills that are completely transferable, so that practically any frontend developer could step in and completely change the way our site looks without a single line of PHP code needing changes.

Finally, for our backend we went to a completely service-oriented architecture. No matter what you do on our site today, the actual functionality is provided by an underlying service that handles the real business logic—in fact, by hitting a single page you could be triggering a request to multiple services, all in different physical locations and therefore completely scalable. This was perhaps the most difficult choice we had to make, because we wanted to avoid the overhead of existing remote service systems like SOAP and therefore had to come up with our substitutes. 

Service orientation poses some unexpected challenges—like, for example, debugging—which take some time to solve. In the end, however, the payoff is terrific, particularly when you consider that a well-designed web service can be used by multiple frontend systems for multiple goals. For example, our backend management system is now based entirely on Adobe Flex, which has allowed us to combine the ease of Flash-based development with the convenience of a true GUI application and all the privileges that come with is—such as direct access to the network and nearly unrestricted control over the filesystem. Despite its power, the entire service management on the Flex side still takes less than one hundred lines of ActionScript code. I should here, publicly thank both Adobe and, in particular, Mike Potter for helping us take full advantage of their upcoming Flex 3 platform, which I think is the best thing to come along since sliced bread.

A new pair of shoes
The end result of all our hard work stands before you today—still a little rough around the edge, but already well-rounded and quite well tested by our faithful users, who kept us busy through the last week by pointing out bugs and suggesting changes to improve the overall user experience.
When I say “end result,” of course, I use a term that is a bit misleading—the importance of the new system is in the fact that it represents a new platform on which we can build our future products, and that was the reason why we got into this messy business to start with.

* Image by chris27