Coyote Web Server

Pick Systems has recently released a new product called FlashConnect to provide a mechanism where D³ applications can be retrofitted with an HTML or web user interface.  Modular Software has received a number of queries about how Coyote and specifically the Coyote web server component of Coyote compares to FlashConnect.  This paper tries to compare what FlashConnect is and attempts to accomplish and how Coyote approaches the same problems.

Information about FlashConnect comes from reviewing Pick's literature, attending some of their short seminars, and talking with others who have used FlashConnect in a development setting.  Modular Software does not currently have FlashConnect running in-house and no Coyote elements are in any way based on any FlashConnect code.

The General Architecture

At first, it may appear that both Coyote and FlashConnect accomplish the same tasks.  This is true in general, but the manner in which these tasks are approached is very different.  Coyote is completely self contained.  With the exception of configuring IP addresses in the underlying host O/S, all Coyote functions run in Pick.  There are no external servers to purchase, configure, and maintain.  The opposite is true for FlashConnect.  FlashConnect runs as a slave application to an external web server by running as a CGI (Common Gateway Interface) application within the host operating system.  This implementation, while functional, is much more complicated than Coyote's Coyote web server and introduces operational limitations and severe performance penalties that Coyote simply does not need to concern itself with.

The Complexity of Setting up the Server Environment

FlashConnect requires that you first configure the external web server. This process by itself is the equivalent of configuring Coyote without even touching the Pick environment.  After the external web server is running, you must then configure the CGI directory, compile and configure the interconnection API application.  Finally, you are ready to configure the Pick software including setting up the process pool, security parameters, and other issues that FlashConnect requires.  If you read Pick's FlashConnect documentation, more than half of it is devoted to configuring and installing FlashConnect.

How the Environment Impacts Performance

Another approach that Pick Systems and other vendors uses to help web integration performance is to talk about "scalability".  This is achieved by running multiple systems, some with web servers and some with Pick data engines.  Again, Coyote makes these so-called solutions unnecessary by making the underlying engine as much as an order of magnitude more efficient.

The reasons that Pick's FlashConnect's performance is not optimal is because of the presence of independent layers operating in multiple environment. Instead of enhancing performance, these layers actually extract a performance penalty which can be significant.  Each layer has it's own processing overhead and each step of moving data from layer to layer only adds more CPU cycles and processing latencies.  The result is an approach that does function, but at a performance level that is far from optimal and that requires far more hardware to service a given level of requests.

If you want to look at how each of these layers takes their toll consider the process of handling a single web request.  The web server receives an HTTP request.  If must receive the request, parse the header fields, and then pass the data off to the CGI interface.  It is here that things start to slow down.  CGI, by it's very nature, requires that the web server create a new host process (or at least link to an existing process).  This is a quite slow task.  In Pick terms this would be the equivalent of logging a port on, running a program, and then logging the port off for each page access.  The overhead does not stop there. After the CGI process is started, Pick's FlashConnect transport application has to load, initialize, and open a TCP/IP connection to the D³ multiplexer application that is running either on the local machine or across the network.  Once this connection is open, data is transferred from the CGI application to Pick.  Once in Pick, the page hit is compared with a table of current "web applications" and the data is sent to the Pick process so that you application code can actually run.  Of course, once your code has generated some data for output, that data must flow along the same path to finally end up in the external web server and eventually get transmitted to the user's browser over the network connection.

If this convoluted path sounds complex and inefficient, it is. Pick Systems does not quote reproducible performance figures, but the number thrown around in their FlashConnect classes indicates that they fell comfortable with handling several hundred page requests per minute.  In web server terms 200 pages/minute is less than 4 hits/second which is really quite dismal.

If you are now thinking that FlashConnect's logic is somewhat contorted, you will be happy to learn that Coyote is amazingly simple and elegant. Instead of layer upon layer built outside of Pick, a pool of Pick phantom processes silently waits for TCP/IP connections, processes HTTP requests, and run your application code.  There are no host processes created on the fly as is the case with CGI and interprocess communication and latency are completely avoided.  The result is that Coyote can process active application page hits at over 50 pages/second or 3000 pages/minute. This is ten times the reported speed of FlashConnect.

The Applications Programming Interface

Coyote builds a similar applications programming environment, but there are differences.  You still write programs top-down, but the execution environment and some of the rules vary.

Application Persistence

• The contents of the applications variables.
• The current execution location.
• The contents of the program's return stack (what subroutines were called to get here).
• There must be semantics for handling users that "disappear".
• There must be semantics for handling application locks when users "disappear".

The Web Interface

With FlashConnect, you build the web page's HTML by first loading a "template" and by then merging your custom HTML code into that template. FlashConnect provides subroutine calls to "initialize" the template and then replace your strings into that template.  This mechanism is workable, but inefficient in several areas.  First, the mechanics of reading the template and then performing insertions is somewhat cumbersome with a lot of subroutine calls involved.  Second, it is your applications responsibility for maintaining HTML data formats including converting ASCII characters like < into the &lt; sequences that HTML expects. Finally, the process of building HTML input fields is also quite manual requiring a lot of Pick/BASIC source code to handle even simple program logic.  The result is a programming environment that does not meld the features of HTML very well with the features of Pick/BASIC.

Coyote also gives you the ability to generate web pages based on templates with your own data inserted.  Instead of requiring a long process to initialize the template and then making subroutine calls to replace your data, Coyote lets you directly include the Pick/BASIC insertion code directly into your HTML "template" pages.  This hybrid HTML/BASIC document makes the task of laying out dynamic HTML pages much easier. Building INPUT logic is also much easier.  With Coyote, you can create an HTML page with form <input ...> statements and simply declare the input statements with names that match the Pick/BASIC variables or dynamic array references that are to receive the data.

The end result is that Coyote lets your programmers do more real work and less busy work managing the web environment.  HTML documents are authored as HTML documents and Pick programs are authored as Pick programs. In fact, Coyote lets you use different personnel to author programs than you use to build HTML layouts.

Run-time Performance

The second area where Coyote is more efficient than FlashConnect is in how HTML documents are produced by merging templates with application variables.  FlashConnect approaches this task by performing a series of string searches and replaces with variable data.  Coyote takes a different approach.  Instead of searching strings for insertion points, Coyote is more like a code-generating 4GL.  For each HTML document, Coyote generates a running Pick/BASIC subroutine and insertion points are actually a part of the generated Pick/BASIC logic.  The result is that Coyote generates dynamic HTML pages faster than most hand-tuned programs could.

Deployment Costs

The Coyote run-time environment is different.  Instead of using a dedicated Pick process for each web application, a small number of Pick phantom processes share the workload for all web users.  Because these processes are running on Pick phantom ports, they actually meter as zero Pick licensed users.  If you were to configure the same 100 user web application with Coyote, the cost would be $5995 total assuming a full T-1 bandwidth requirement for Coyote.  This is 83% less than the cost to deploy FlashConnect at 100 users and would actually support many more users at this bandwidth.  Here is an estimation of costs for a number of user sizes:

Using FlashConnect in a Public Web Site Environment

Scalability

Coyote deals with scalability with a more straight forward approach. For most users, their current Pick host system is more than adequate for deployment of web applications.  Remember that Coyote is as much as 10 times as efficient at processing web requests as FlashConnect. This means that nearly any current Pick host can drive an internet T-1 line to saturation without any additional effort.  This is adequate for the vast majority of Pick applications.  If you application is larger than this, you can still use traditional web scalability solutions including reverse proxy servers and parallel Pick hosts.  Just remember that these steps are usually not necessary for most application environments.

Conclusion

Otherwise, FlashConnect is best viewed as just another tool kit that lets external web servers call Pick.  Not that using an external web server is unworkable, it just adds several layers of complexity, overhead, and inefficiency that are best done without.  By bringing the entire HTTP engine into Pick, Coyote and Coyote end-run all of the complexity and performance bottlenecks by keeping all of the processing local to the Pick virtual machine.  The result is a single system that is fast, flexible, and performs as much as ten times faster than competing product offerings.

Comparitive Summary