ScalaServer: System Support for Scalable Network Servers

Department of Computer Science
Rice University

The ScalaServer project considers a range of related issues in the design of scalable, high-performance and highly available network servers.  Internet server systems (e.g., Web servers) serve a virtually unlimited client community, they provide an increasing volume of data, and they must support a set of Web based services that increases in variety and sophistication. The performance of such servers is critical to the success of the evolving information infrastructure.
 

This project is supported by the National Science Foundation, the Texas Advanced Technology Program, the Sloan Foundation, Compaq, HP, and IBM.

Project Members:

• Mohit Aron
• Gaurav Banga
• Sitaram Iyer
• Juan Navarro
• Vivek S. Pai
• Darren Sanders
• Peter Druschel
• Willy Zwaenepoel

Contents:

• OS support for network servers
• Systems support for cluster-based servers
• Current directions
• Code Releases
• Publications

OS support for network servers

We have developed a set of new OS facilities that provide effective support for network server applications in general-purpose operating systems, such as UNIX and Windows NT:
 

• Superpages are memory pages of large sizes that are provided by most modern processors. While they have the potential of improving applications' performance by reducing the number of TLB misses, supporting superpages poses several challenges to the operating system in terms of allocation and promotion tradeoffs, fragmentation control, and page eviction. We developed a general and transparent superpage management system that achieves sustained performance improvement (often exceeding 30%) for realistic workloads. The system balances various tradeoffs by using opportunistic superpage allocation policies, incremental promotion and speculative demotion techniques, and a contiguity-aware page replacement algorithm.

• Anticipatory disk scheduling is a novel disk scheduling paradigm that solves a problem in OS disk subsystems. When applications issue synchronous I/O, the system is induced into a state of `deceptive idleness'; we deliberately delay requests by calculated amounts of time to avoid this. Our FreeBSD prototype implementation shows improvements of 30-60% with common applications like file servers, web servers and databases. Results of this work are being implemented and tested by independent parties for inclusion into Linux.

• Lazy Receiver Processing (LRP) is a new network subsystem architecture that provides stable overload behavior, increased fairness, and improved throughput under high load. LRP allows general-purpose operating systems to perform differentiated network service, even under conditions of overload, and helps to defend against denial-of-service attacks from the network.

• IO-Lite is an I/O subsystem that unifies I/O data buffering in the filesystem, file cache, network subsystem, IPC subsystems, and applications. IO-Lite eliminates all data copying and multiple buffering not required by hardware, independent of the path that I/O data takes through the system. The resulting reduction in CPU and memory overhead can lead to substantial increases in server throughput on real workloads.

• Resource containers (RCs)  is a new OS abstraction that enables fine-grained resource management in network servers. RCs address the problem of providing differentiated QoS in network servers, independent of the concurrency architecture used bthe server application (e.g., multi-process, multi-threaded or event-driven).

• General-purpose operating systems like UNIX lack an event-delivery mechanism that can efficiently scale to thousands of event sources. This is a problem for high-performance event-driven network server applications that handle a high volume of WAN-based client connections. To this end, we have designed, implemented and evaluated a new, scalable  event delivery mechanism for UNIX systems.

• Soft timers   is a new operating system facility that allows the efficient scheduling of software events at a granularity down to tens of microseconds. Soft timers can be used to avoid interrupts and reduce context switches associated with network processing without sacrificing low communication delays. Applications include rate-based pacing of packet transmissions and network polling.

We also investigated issues in benchmarking WWW server performance under conditions of high load and when clients are connecting via wide-area networks.  This work shows that current Web server benchmarks fail to evaluate important aspects of Web server performance and we propose improved benchmarking techniques that address this problem.

Finally, we developed a new architecture for network server applications, called asymmetric multi-process event-driven (AMPED), and implemented and evaluated a prototype server called the  Flash Web Server. Experiments on real workloads show that Flash combines the high performance of single-process event-driven servers on cached workloads with the high performance of multi-threaded and multi-process servers on out-of-core workloads.
 

Systems support for cluster-based servers

High-performance sites increasingly rely on a hardware platform that consists of a cluster of workstations or personal computers, due to the superior price-performance ratio of this platform. In the most common configuration, a cluster-based server employs a front-end node, which acts as a single point of contact for clients. The front-end distributes incoming requests to a set of back-end nodes, which handle the requests.

We have designed and implemented a novel request distribution policy for cluster front-ends, called locality-aware request distribution (LARD). LARD distributes requests in a manner that simultaneously balances the load among the back-end nodes and dynamically partitions the working set. This policy allows the effective size of the server's main memory cache to approach the sum of the node cache sizes. We have demonstrated the effectiveness of this technique, both via simulation and in a prototype cluster implementation, on real workloads.

LARD is a form of content-based request distribution, where the cluster front-end must establish the client HTTP connection and receive the HTTP request before it can decide which back-end node should handle the request. We have developed a TCP handoff protocol, which allows a cluster front-end to hand off an established TCP connection to another node in an efficient and client transparent manner.  Finally, we have shown how to apply LARD on HTTP/1.1 persistent connection workloads.
 

Current directions

Within the ScalaServer project, we currently turn our attention to resource management in cluster-based Web servers. Providing differentiated quality-of-service (based on client community, type of document or type of service requested) in a cluster-based server requires global resource management not found in current clusters.

In addition, we investigate request distribution, load balancing, and resource management in cluster-based servers in the presence of rich and diverse Web services, including dynamically generated content services with varying and unpredictable resource demands, transactional (i.e., database) services, and secure services (SSL/TLS).
 

Code Releases

• Scalable Content-aware Request Distribution in Cluster-based Network Servers

• Soft timers

• Resource containers and LRP

Publications

Practical, transparent operating system support for superpages. Juan Navarro, Sitaram Iyer, Peter Druschel and Alan Cox. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, Mass., Dec 2002.
slides from conference talk

Anticipatory scheduling: A disk scheduling framework to overcome deceptive idleness in synchronous I/O. Sitaram Iyer, Peter Druschel. In Proceedings of the 18th Symposium on Operating Systems Principles (SOSP), Chateau Lake Louise, Banff, Canada, September 2001.
Project webpage, slides from conference talk.

Cluster Reserves: A Mechanism for Resource Management in Cluster-based Network Servers . Mohit Aron, Peter Druschel and Willy Zwaenepoel. In Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, Santa Clara, CA, June 2000.
slides from the conference talk

Scalable Content-aware Request Distribution in Cluster-based Network Servers. Mohit Aron, Darren Sanders, Peter Druschel and Willy Zwaenepoel. In Proceedings of the USENIX 2000 Annual Technical Conference, San Diego, CA, June 2000. Winner of Best Paper Award.
slides from the conference talk

Soft timers: efficient microsecond software timer support for network processing . Mohit Aron and Peter Druschel. In Proceedings of the 17th Symposium on Operating Systems Principles (SOSP-17), Kiawah Island, SC, December 1999. Award Paper. An extended version also appeared in ACM Transactions on Computer Systems, August 2000.
slides from the conference talk

A scalable and explicit event delivery mechanism for UNIX. Gaurav Banga, Jeff Mogul, Peter Druschel. In the Proceedings of the USENIX 1999 Annual Technical Conference, Monterey, CA, June 1999. Winner of Outstanding Paper Award.

Efficient Support for P-HTTP in Cluster-Based Web Servers . Mohit Aron, Peter Druschel and Willy Zwaenepoel. In Proceedings of the USENIX 1999 Annual Technical Conference, Monterey, CA, June 1999.
slides from the conference talk

Flash: An Efficient and Portable Web Server. Vivek Pai, Peter Druschel and Willy Zwaenepoel. In the Proceedings of the USENIX 1999 Annual Technical Conference, Monterey, CA, June 1999.
slides from the conference talk

IO-Lite: A Unified I/O Buffering and Caching System. Vivek S. Pai, Peter Druschel and Willy Zwaenepoel. In ACM Transactions on Computer Systems, Vol. 18, No. 1, February 2000. A version appeared in the Proceedings of the Third Symposium on Operating Systems Design and Implementation (OSDI'99), New Orleans, LA, February 1999. Best Paper Award.
slides from the conference talk

Resource containers: A new facility for resource management in server systems. Gaurav Banga, Peter Druschel, Jeff Mogul. In the Proceedings of the Third Symposium on Operating System Design and Implementation, New Orleans, LA, February 1999. Winner of Best Student Paper Award.

Locality-aware Request Distribution in Cluster-based Network Servers. Vivek S. Pai, Mohit Aron, Gaurav Banga, Michael Svendsen, Peter Druschel, Willy Zwaenepoel and Erich Nahum. In Proceedings of the Eighth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-VIII), San Jose, California, October 1998.
slides from the conference talk

Measuring the Capacity of a Web Server.  Gaurav Banga, Peter Druschel. In USENIX Symposium on Internet Technologies and Systems (USITS). Monterrey, CA, Dec 1997. Winner of Best Paper and Best Student Paper Awards.

Lazy Receiver Processing (LRP): A Network Subsystem Architecture for Server Systems.  Peter Druschel and Gaurav Banga. In Proceedings of the Second Symposium on Operating Systems Design and Implementation (OSDI'96). Seattle, WA, October 1996.