Operation shipping for mobile file systems

Abstract

This paper addresses a bottleneck problem in mobile file systems: the propagation of updated large files from a weakly-connected client to its servers. It proposes an efficient mechanism called operation shipping or operation-based update propagation. In the new mechanism, the client ships the user operation that updated the large files, rather than the files themselves, across the weak network. (In contrast, existing file systems use value shipping and ship the files.) The user operation is sent to a surrogate client that is strongly connected to the servers. The surrogate replays the user operation, regenerates the files, checks whether they are identical to the originals, and, if so, sends the files to the servers on behalf of the client. Care has been taken such that the new mechanism does not compromise correctness or server scalability. For example, we show how forward error correction (FEC) can restore minor reexecution discrepancies and, thus, make operation shipping work with more applications. Operation shipping can be further classified into two types: application-transparent and application-aware. Their feasibilities and benefits have been demonstrated by the design, implementation, and evaluation of a prototype extension to the Coda File System. In our controlled experiments, operation shipping achieved substantial performance improvements-network traffic reductions from 12 times to nearly 400 times and speedups in the range of 1.4 times to nearly 50 times