In queueing theory, a discipline within the mathematical theory of probability, a fork–join queue is a queue where incoming jobs are split on arrival for service by numerous servers and joined before departure. The model is often used for parallel computations or systems where products need to be obtained simultaneously from different suppliers (in a warehouse or manufacturing setting).:78–80 The key quantity of interest in this model is usually the time taken to service a complete job. The model has been described as a "key model for the performance analysis of parallel and distributed systems." Few analytical results exist for fork–join queues, but various approximations are known.
On arrival at the fork point, a job is split into N sub-jobs which are served by each of the N servers. After service, sub-job wait until all other sub-jobs have also been processed. The sub-jobs are then rejoined and leave the system.
For the fork–join queue to be stable the input rate must be strictly less than sum of the service rates at the service nodes.
The response time (or sojourn time) is the total amount of time a job spends in the system.
Ko and Serfozo give an approximation for the response time distribution when service times are exponentially distributed and jobs arrive either according to a Poisson process or a general distribution.
Average response time
An exact formula for the average response time is only known in the case of two servers (N=2) with exponentially distributed service times (where each server is an M/M/1 queue). In this situation, the response time (total time a job spends in the system) is
- is the utilization
- is the arrival rate of jobs to the system
- is the total service rate across all the nodes.
For general service times (where each node is an M/G/1 queue) Baccelli and Makowski give bounds for the average response time and higher moments of this quantity both in the transient and steady state situations. Kemper and Mandjes show that for some parameters these bounds are not tight and show demonstrate an approximation technique. For heterogeneous fork-join queues (fork-join queues with different service times), Alomari and Menasce propose an approximation based on harmonic numbers that can be extended to cover more general cases such as probabilistic fork, open and closed fork-join queues.
In general the stationary distribution of the number of jobs at each queue is intractable. Flatto considered the case of two servers (N=2) and derived the stationary distribution for the number of jobs at each queue via uniformization techniques. Pinotsi and Zazanis show that a product form solution exists when arrivals are deterministic as the queue lengths are then independent D/M/1 queues.
Heavy traffic/diffusion approximation
When the server is heavily loaded (service rate of the queue is only just larger than arrival rate) the queue length process can be approximated by a reflected Brownian motion which converges to the same stationary distribution as the original queueing process. Under limiting conditions the state space of the synchronisation queues collapses and all queues behave identically.
Join queue distribution
Once jobs are served, the parts are reassembled at the join queue. Nelson and Tantawi published the distribution of the join queue length in the situation where all servers have the same service rate. Heterogeneous service rates and distribution asymptotic analysis are considered by Li and Zhao.
Networks of fork–join queues
An approximate formula can be used to calculate the response time distribution for a network of fork–join queues joined in series (one after the other).
A related model is the split–merge model, for which analytical results exist. Here on arrival a job is split into N sub-tasks which are serviced in parallel. Only when all the tasks ﬁnish servicing and have rejoined can the next job start. This leads to a slower response time on average.
- Kim, C.; Agrawala, A. K. (1989). "Analysis of the fork-join queue". IEEE Transactions on Computers 38 (2): 250. doi:10.1109/12.16501.
- Lebrecht, Abigail; Knottenbelt, William J. (June 2007). "Response Time Approximations in Fork-Join Queues". 23rd Annual UK Performance Engineering Workshop (UKPEW).
- Serfozo, R. (2009). "Markov Chains". "Basics of Applied Stochastic Processes". Probability and Its Applications. doi:10.1007/978-3-540-89332-5_1. ISBN 978-3-540-89331-8.
- Boxma, Onno; Koole, Ger and Liu, Zhen (1996). Queueing-theoretic Solution Methods for Models of Parallel and Distributed Systems (Technical report). CWI. BS-R9425.
- Flatto, L.; Hahn, S. (1984). "Two Parallel Queues Created by Arrivals with Two Demands I". SIAM Journal on Applied Mathematics 44 (5): 1041. doi:10.1137/0144074.
- Wright, Paul E. (1992), Two parallel processors with coupled inputs, Advances in Applied Probability 24: 986–1007, JSTOR 1427722
- Pinotsi, D.; Zazanis, M. A. (2005). "Synchronized queues with deterministic arrivals". Operations Research Letters 33 (6): 560. doi:10.1016/j.orl.2004.12.005.
- Konstantopoulos, Panagiotis; Walrand, Jean (September 1989). "Stationary and Stability of Fork-Join Networks". Journal of Applied Probability 26 (3): 604–614. doi:10.2307/3214417. JSTOR 3214417.
- Lebrecht, A. S.; Dingle, N. J.; Knottenbelt, W. J. (2009). "Modelling Zoned RAID Systems Using Fork-Join Queueing Simulation". "Computer Performance Engineering". Lecture Notes in Computer Science 5652. p. 16. doi:10.1007/978-3-642-02924-0_2. ISBN 978-3-642-02923-3.
- Kemper, B.; Mandjes, M. (2011). "Mean sojourn times in two-queue fork-join systems: Bounds and approximations". OR Spectrum 34 (3): 723. doi:10.1007/s00291-010-0235-y.
- Ko, S. S.; Serfozo, R. F. (2004). "Response times in M/M/s fork-join networks". Advances in Applied Probability 36 (3): 854. doi:10.1239/aap/1093962238.
- Ko, S. S.; Serfozo, R. F. (2008). "Sojourn times in G/M/1 fork‐join networks". Naval Research Logistics 55 (5): 432. doi:10.1002/nav.20294.
- Nelson, R.; Tantawi, A. N. (1988). "Approximate analysis of fork/join synchronization in parallel queues". IEEE Transactions on Computers 37 (6): 739. doi:10.1109/12.2213.
- Varki, Elizabeth; Merchant, Arif; Chen, H. "M/M/1 Fork-join queue with variable sub-tasks".
- Baccelli, François; Makowski, A. (1985), Simple computable bounds for the fork-join queue, National Institute for Research in Computer Science and Control Technical Report, retrieved 2011-07-08
- Alomari, F.; Menasce, D. A. (2013). "Efficient Response Time Approximations for Multiclass Fork and Join Queues in Open and Closed Queuing Networks". IEEE Transactions on Parallel and Distributed Systems: 1. doi:10.1109/TPDS.2013.70.
- Tsimashenka, I.; Knottenbelt, W. J. (2013). "Reduction of Subtask Dispersion in Fork-Join Systems". "Computer Performance Engineering". Lecture Notes in Computer Science 8168. pp. 325–336. doi:10.1007/978-3-642-40725-3_25. ISBN 978-3-642-40724-6.
- Tan, X.; Knessl, C. (1996). "A fork-join queueing model: Diffusion approximation, integral representations and asymptotics". Queueing Systems 22 (3–4): 287. doi:10.1007/BF01149176.
- Varma, Subir (1990). "Heavy and Light Traffic Approximations for Queues with Synchronization Constraints (Ph. D. thesis)". University of Maryland. Retrieved 10 February 2013.
- Atar, R.; Mandelbaum, A.; Zviran, A. (2012). "Control of Fork-Join Networks in heavy traffic". "2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)". p. 823. doi:10.1109/Allerton.2012.6483303. ISBN 978-1-4673-4539-2.
- Li, J.; Zhao, Y. Q. (2010). "On the Probability Distribution of Join Queue Length in a Fork-Join Model". Probability in the Engineering and Informational Sciences 24 (4): 473. doi:10.1017/S0269964810000112.
- Ko, S. S. (2007). "Cycle Times in a Serial Fork-Join Network". "Computational Science and Its Applications – ICCSA 2007". Lecture Notes in Computer Science 4705. pp. 758–766. doi:10.1007/978-3-540-74472-6_62. ISBN 978-3-540-74468-9.
- Harrison, P.; Zertal, S. (2003). "Queueing Models with Maxima of Service Times". "Computer Performance Evaluation. Modelling Techniques and Tools". Lecture Notes in Computer Science 2794. p. 152. doi:10.1007/978-3-540-45232-4_10. ISBN 978-3-540-40814-7.