Pytheas & the Problem of Phoenician Tin: ancient lessons for navigating the high seas of Open Source software development, building library systems, and circumventing trade blockades

Art Rhyno, Leddy Library, University of Windsor
arhyno@uwindsor.ca

Beginnings

One of the most important and promising recent trends in technology has been the emergence of Open Source software. Loosely defined as a model of software development where applications are created and maintained by a community of developers, Open Source applications are made freely available in source form for modification and peer review. 

Open Source is most closely identified with the Linux operating system, a flexible, high-quality system developed, distributed, and supported through the Open Source model. It is safe to say that anyone who has used the Internet has benefited from Open Source software. Linux is a popular choice for hosting Internet applications because of its strong stability and worldwide support structure. Other Open Source applications in widespread use include Apache, the most popular web server in the world. 

Open Source is, in many ways, a reaction to the perception that consumers pay too much for commercial applications and don’t gain enough flexibility in the resulting product. Microsoft is most often blamed for this situation though there was also a time in the not so distant past when Bill Gates was the white knight leading everyone out of the dominance of the IBM empire. Of course, humans have been dealing with monopolies and trading inequities for far longer than the existence of the Department of Justice and shrink-wrapped software. 

More than two millennia ago, the ancient Greeks were faced with a Phoenician blockade that was preventing direct access to the tin routes. As a result, the Greeks were forced to buy tin through overland suppliers at a premium price. The Greek colony of Massalia (now Marseilles) was particularly hard hit by this blockade.  Massalia had no free access to the ocean, which was difficult for a colony that depended heavily on trade, and was squeezed not only by the supply problems that were caused by the blockade but also by the limits imposed by the Phoenicians in trying to set up contacts for trade with other lands. 

Libraries are in a similar position as Massalia, but for acquiring software rather than tin. Like the citizens of Massalia, we know that this critical supply goes through a series of dealers and is sometimes marked up along the way. Library systems could be compared to the Phoenician tin used by the Massalians 23 centuries ago, in both cases, representing products that are vitally important for day to day operations, yet somewhat mysterious in their journey from source to deployment. Library system vendors must deal also with the consequences of IT supply lines, and most are almost as dependent on outside technologies as the market they sell to. If libraries can be compared to the Massalians, library system vendors may be much like the suppliers in the middle of the tin routes: not exactly an unprofitable position, yet far short of the returns that await those who control the blockades. 

Alternatives to Phoenician Tin

In about 325 BC, an explorer named Pytheas was charged with the task of finding the source of the tin routes. Although the records are unclear and Pytheas' monumental work describing his achievements called On the Ocean has been lost, it appears that he was funded by the town council and merchants of Massalia. The town council of 300 BC was a bit like the World Wide Web consortium is today in that it provided direction for the activities of the community it represented, even though the merchants of the time didn't always heed its advice. Much like the computer industry’s history with standards, everyone appreciated the need for a somewhat democratic organization to put order on the world but that didn't necessarily mean everyone followed its directions.

A similar though somewhat more modest journey began in the library world in 1998 with the establishment of the oss4lib web site and listserv by Dan Chudnov at Yale. Initial and continuing discussions about the possibility of an Open Source library system on the oss4lib listserv and the linux-in-libraries list led to the creation of the OSDLS (Open Source Digital Library System) site by Jeremy Frumkin at the University of Arizona. Much like the task that lay before Pytheas, the quest for a truly Open Source library system was not completely without precedent. Pytheas would have been aware of many of the voyages that had ended in failure before him and had first hand knowledge of some of the routes where his journey would begin. For their part, libraries had been major adopters of the "in-house" system trend that had marked the days of the mainframe from the 1960s through to the eighties, and had experienced the backlash against in-house applications when public institutions increasingly turned to the marketplace for technical solutions. 

Yet if the waters on the journey to Open Source solutions were not completely unknown, there seemed to be enough differences to make the trip worth attempting one more time. Open Source software is fundamentally based on the availability of ubiquitous networks to foster wide scale collaboration and feedback beyond library walls and institutional borders. Furthermore, the nature of software development has also changed dramatically in the last decade. Object and component-based programming models have led to greater freedom in building applications from a variety of sources and with much greater efficiency since the entire life cycle of developing the component has already been completed by someone else. There are also a growing number of Open Source success stories in the library world, including jake, MyLibrary, and the Greenstone Digital Library Software.

One of the most important lessons for embarking on a project on the scale of an Open Source library system or to circumvent a trade blockade is to leverage existing knowledge and tools as much as possible. There is speculation that Pytheas may have been successful because he was willing to tap into solutions that were outside of the Greek sea-faring experience. He may have utilized the talents of sailors from other countries and gathered information from the inhabitants of each strange land he passed in order to get a better sense of what lay ahead. Similarly, Linux benefited greatly from the building blocks constructed for BSD, and as Eric Raymond points out in his groundbreaking Open Source essay, The Cathedral and The Bazaar, part of the genius of Linus Torvalds, the creator of Linux, was in his ability to find the minimal effort path from Point A to Point B. 

It is critical that libraries not make the mistakes experienced in previous decades when institutions resorted to building everything in-house because so few external building blocks were perceived to be available elsewhere. Pytheas knew better, and in order to follow his example, one of the projects of the OSDLS was named after him in acronym form. Of course, it took some time to determine what exactly the acronym should stand for but the title that emerged was “Powerful Yet Tactfully Helpful Electronic Arranger of Sources”. More than anything, PYTHEAS tries not to get in the way of using existing web and mainstream technologies.

One Step Forward, One Step Back

Pytheas was no doubt much wiser in his selection of a language to use in his journey than the first architects of the system named in his honour. There were a number of common dialects for some of the lands that Pytheas would pass on the way to the tin mines and he would have made sure he was as ready to communicate with the inhabitants along the way as he could possibly be. The initial developer’s release of the PYTHEAS system in 1999 was written in several environments. No matter how logical this seemed at the time, it simply confused some of those who wanted to contribute to the project. The first release offered several C libraries, some JavaScript, and Java, as well as PHP for a searching interface. The appropriateness of any of these tools could be argued on its own, but together they made for a strange kind of murky developer’s soup. It quickly became clear that constantly putting on a different kind of developer’s hat to work with PHP for one task, and then C for another, was throwing too much disparate technology into the mix.

For several reasons, Java became the solution to the multiple-language problem. Java tools were available for each part of the project, from establishing a middle layer that could handle multiple connections for staff updates to working with JDBC database drivers for loading data into the system. Java also provided a technology called Enterprise Java Beans  (EJB) which had the double benefit of both supplying a powerful middleware framework and hiding most of the messy details that had added so much bulk to the original C version of the middle layer. A related and java-based toolkit from Exolab called Castor enhances the capabilities of the EJB server while XML-enabling many of the components of PYTHEAS, which in turn, allowed SOAP/Web Services support to be easily incorporated. Finally, Java was one half of a technology originally conceived by Netscape called LiveConnect, which was an essential component for making the web browser a viable platform for staff client software.  The overall architecture of PYTHEAS can be seen in the following diagram.

LiveConnect provides the glue for letting a JavaScript application maximize the power of a web browser, in this case, allowing an applet to hold a stateful connection to a server somewhere on the network (see illustration below). For PYTHEAS’ purposes, LiveConnect also necessitated a fair amount of JavaScript on the browser, as did Mozilla’s XUL (XML User Interface). This somewhat violated the dream of one language for every aspect of the system, but JavaScript is so necessary for utilizing many aspects of a web interface and HTML forms that it looked possible to achieve forgiveness for this deviation from the one language rule. It is hoped to eventually utilize XForms, the evolving XML-based enhancements to HTML forms in order to provide a rich and flexible editing environment.

Of course, no journey is complete without several interpretations on the optimal route to reach the destination. It seemed unworkable to use a standard web form/cgi-bin setup for the level of updating that would be required for staff to do MARC cataloguing or ordering library materials, yet several observers sent discreet e-mails questioning if the architecture was too complex and “over-engineered” for a library system. Some of the components certainly required the consumption of more caffeine than is recommended for most humans but almost all of the technologies were selected in order to tap into mainstream applications. 

In other words, the architecture was designed to maximize contributions from other communities, so that, for example, the general ledger required for Acquisitions could be plugged in using EJB and free up the resources required to develop the ledger from scratch. EJB and SOAP, in particular, free a developer from spending enormous amounts of time implementing tricky programming constructs like transactions and resource pooling.

This did not mean that there were no dilemmas. One very difficult question to answer concerned the goal of using a browser interface for staff clients. It was pointed out that a fully Java-based client could offer a rich interface and a stateful connection, without being tied to a particular platform and with no need to use JavaScript. Still others noted, quite sensibly, that in the library world general scripting languages like Perl might offer some advantages over Java because more developers could be encouraged to contribute.

Pytheas must have reached a point in preparing for his journey that he finalized on a specific direction and sailed into the unknown. The project had already turned back to shore once in order to revamp everything in Java, and the notion of the web browser as a container for application interaction was so compelling that the LiveConnect/JavaScript “deviation” remained part of the architecture. If others build stellar Java clients for PYTHEAS or construct responsive middleware in Perl, then this will be welcomed, but it was time for the project to proceed on to the open waters. 
 
Surprises on the High Seas

No major trip would be complete without a few incidents that the travel planners had not anticipated. When deciding on a browser environment for staff client software, it seemed obvious that the browser which made the most sense in an Open Source project was Netscape. The Mozilla project had won Netscape a lot of support in the Open Source community. Even if alternative browser users might argue this point in favour of Opera or one of the many other browsers listed at BrowserWatch, it seemed highly unlikely that there would be any support for considering Microsoft’s Internet Explorer (IE). After all, IE was tied to the platform that the Open Source community is seemingly the most opposed to. 

Yet there were strong arguments that Internet Explorer needed to be as capable of supporting an application like a MARC editor as Netscape. In part, this had to do with the realities of IT support in libraries, many libraries are often dependent on a municipality or computer center that dictates a particular layout on each staff desktop in the library and Internet Explorer could often be part of this setup. Others just liked Internet Explorer better, and since it seemed that Open Source should be about offering more choices rather than limiting options, an attempt was made to come up with an environment that could be made to work with both Netscape and Microsoft.  This effort was ultimately abandoned, the advantages of XUL as a platform-independent toolkit for generating rich browser-based interfaces were just too great.

Another surprise was the level of evangelism that marked comments on the project. This was a surprise because of the contradictions between the enthusiasm exhibited in the comments and the realities of the context they were made in. Receiving diatribes about avoiding Microsoft solutions carefully written out in a Word attachment via e-mail sent with Outlook seemed counter-intuitive though this may have been an example of a library straining against an external computing policy. Strangely enough, very few of these messages have appeared on the OSDLS list and most have been private e-mails. Some commentaries were critical of existing library systems and there seemed to be an underlying impatience with most library standards, especially MARC, and extended library functions, such as authority control. 

Despite the mild bashing of some venerable library technologies, completely ignoring MARC and other existing library standards would be comparable to Pytheas tossing his compass overboard at the beginning of his journey. It might make for an exciting trip but libraries have made huge investments in standards and formats that can’t be left behind at this point. PYTHEAS is specifically designed to fully leverage the power of MARC while defining an XML view of bibliographic data and we watch with great interest an exciting project at the Lane Medical Library at Stanford University to unite MARC and XML that may eventually redefine how MARC is used and shared between libraries.  

Ancient Tin Miners & the Modern Programmer

If Pytheas did eventually see the tin mines in Britain, there might have been a point after his initial wonder that he thanked the stars he wasn’t one of the workers who toiled to extract it.  The backbreaking labor that went into retrieving tin may have seemed remote when admiring the latest creation of the tinsmiths in Massalia, but Pytheas no doubt appreciated that sooner or later, someone needs to do a lot of hard work to get the process started. Programming is undoubtedly much more rewarding than mining for tin was in 300 BC, but it is probably as badly understood today as tin was to the ancient Greeks. 

The evidence of our inability to properly plan and put together computer applications is everywhere in the IT world; constant delays in release dates, budget problems, and never-ending beta processes seem to be the hallmarks of application development. Libraries tend to be somewhat removed from the realities of the software development cycle and it probably is no surprise that developers are a little rare in the library world. 

Creating a system can be no less ambitious than extracting tin before the invention of large-scale mining equipment. Linus Torvalds was able to offer enough existing and new building blocks to catch the attention of a ready and willing worldwide audience to put together one of the most impressive collaborative projects in the history of humanity.  An Open Source library system has to appeal to a much smaller audience and one that may not be as enabled to build momentum in the way that Linux has come together. An entire community of developers may be galvanized to build the world's next great operating system or spreadsheet, or almost anything else that Microsoft or Oracle actively markets and profits from, but far fewer developers are interested in creating an alternative cataloguing editor or library circulation system. To address this, the PYTHEAS system had utilized as many freely available building blocks as possible but the number of functions an ILS is expected to supply looked daunting when compared to the developers available for the task.

The popularity of XML and the amount of work done on metadata standards make some initial steps look more achievable. The Dublin Core/MARC/GILS Crosswalk supplies a straightforward way of mapping MARC to a generally understood metadata format that can form the basis of searching, and the NACO normalization rules provide a consistent approach to mapping queries to relational database layouts. Using the Resource Description Framework (RDF), which is a standard for describing metadata in XML, records could be described and enhanced in ways that might be difficult in MARC, such as adding annotations and reviews to bibliographic records. Castor makes XML easy to produce and work with at almost every working level, so that XML can be used to “wire in” many other applications (see illustration below).

A system that requires a million lines of code is probably as unworkable to libraries as mining tin without a large group of willing miners, but XML and java-based technologies like EJB open the door to leveraging the efforts of a worldwide community to the task at hand. Phoenician tin was often overpriced and of inferior quality because it went through a series of suppliers on the way to market. Library systems built on web-related standards and close to a rich source like XML may be able to use technologies that have not yet lost their luster.

From Massalia to Alexandria

Pytheas' statements about his voyage flew in the face of the Greek philosophy and geographic theory of the time and after his death he became thoroughly discredited. Yet we know today that Pytheas was remarkably accurate in his estimations of the circumference of Britain and it is very likely that he made the longest voyage on record in the ancient world. One of the first scholars to give Pytheas credibility was Eratosthenes, who became the chief librarian at the great library at Alexandria in about 255 BC and is sometimes referred to as the "librarian who measured the earth" in recognition of his achievement of being the first human to correctly measure the circumference of the world. Eratosthenes treated Pytheas’ On the Ocean as a serious work and utilized some of Pytheas’ observations in his own studies.

Libraries have a natural synergy with the Open Source movement. Library collections, library staff, and even the physical structures themselves are usually widely available to a community of users on a non-profit, publicly funded basis. Like Eratosthenes, we have the opportunity to boldly recognize and support a movement that is turning a lot of conventional wisdom about how systems should be built on its head. The start of the journey can be as simple as evaluating the use of Open Source solutions in your library. Who knows? It could change your perception of the world.