Outer Billiards is a dynamical system based on a convex shape in the plane. Classically, this system is defined for the Euclidean plane but one can also consider the system in the hyperbolic plane or in other spaces that suitably generalize the plane. Outer billiards differs from a usual dynamical billiard in that it deals with a discrete sequence of moves outside the shape rather than inside of it.
The Outer Billiards Map 
Let P be a convex shape in the plane. Given a point x0 outside P, there is typically a unique point x1 (also outside P) so that the line segment connecting x0 to x1 is tangent to P at its midpoint and a person walking from x0 to x1 would see P on the right. (See Figure.) The map F: x0 -> x1 is called the outer billiards map.
The inverse (or backwards) outer billiards map is also defined, as the map x1 -> x0. One gets the inverse map simply by replacing the word right by the word left in the definition given above. The figure shows the situation in the Euclidean plane, but the definition in the hyperbolic plane is essentially the same.
An outer billiards orbit is the set of all iterations of the point, namely ... x0 <--> x1 <--> x2 <--> x3 ... That is, start at x0 and iteratively apply both the outer billiards map and the backwards outer billiards map. When P is a strictly convex shape, such as an ellipse, every point in the exterior of P has a well defined orbit. When P is a polygon, some points might not have well-defined orbits, on account of the potential ambiguity of choosing the midpoint of the relevant tangent line. Nevertheless, in the polygonal case, almost every point has a well-defined orbit.
- An orbit is called periodic if it eventually repeats.
- An orbit is called aperiodic (or non-periodic) if it is not periodic.
- An orbit is called bounded (or stable) if some bounded region in the plane contains the whole orbit.
- An orbit is called unbounded (or unstable) if it is not bounded.
higher dimensional spaces 
Defining an outer billiards system in a higher dimensional space is beyond the scope of this article. Unlike the case of ordinary billiards, the definition is not straightforward. One natural setting for the map is a complex vector space. In this case, there is a natural choice of line tangent to a convex body at each point. One obtains these tangents by starting with the normals and using the complex structure[disambiguation needed] to rotate 90 degrees. These distinguished tangent lines can be used to define the outer billiards map roughly as above. See S. Tabachnikov's book (cited in the references) for details.
Most people attribute the introduction of outer billiards to Bernhard Neumann in the late 1950s, though it seems that a few people cite an earlier construction in 1945, due to M. Day. Jürgen Moser popularized the system in the 1970s as a toy model for celestial mechanics. This system has been studied classically in the Euclidean plane, and more recently in the hyperbolic plane. One can also consider higher dimensional spaces, though no serious study has yet been made. Bernhard Neumann informally posed the question as to whether or not one can have unbounded orbits in an outer billiards system, and Moser put it in writing in 1973. Sometimes this basic question has been called the Moser-Neumann question. This question, originally posed for shapes in the Euclidean plane and solved only recently, has been a guiding problem in the field.
Moser-Neumann question 
bounded orbits in the Euclidean plane 
In the 70's, Jürgen Moser sketched a proof, based on K.A.M. theory, that outer billiards relative to a 6-times-differentiable shape of positive curvature has all orbits bounded. In 1982, R. Douady gave the full proof of this result. A big advance in the polygonal case came over a period of several years when three teams of authors, Vivaldi-Shaidenko (1987), Kolodziej (1989), and Gutkin-Simanyi (1991), each using different methods, showed that outer billiards relative to a quasirational polygon has all orbits bounded. The notion of quasirational is technical (see references) but it includes the class of regular polygons and convex rational polygons, namely those convex polygons whose vertices have rational coordinates. In the case of rational polygons, all the orbits are periodic. In 1995, Tabachnikov showed that outer billiards for the regular pentagon has some aperiodic orbits, thus clarifying the distinction between the dynamics in the rational and regular cases. In 1996, Boyland showed that outer billiards relative to some shapes can have orbits which accumulate on the shape. In 2005, D. Genin showed that all orbits are bounded when the shape is a trapezoid, thus showing that quasirationality is not a necessary condition for the system to have all orbits bounded. (Not all trapezoids are quasirational.)
unbounded orbits in the Euclidean plane 
In 2007, R. E. Schwartz showed that outer billiards has some unbounded orbits when defined relative to the Penrose Kite, thus answering the original Moser-Neumann question in the affirmative. The Penrose kite is the convex quadrilateral from the kites-and-darts Penrose tilings. Subsequently, Schwartz showed that outer billiards has unbounded orbits when defined relative to any irrational kite. An irrational kite is a quadrilateral with the following property: One of the diagonals of the quadrilateral divides the region into two triangles of equal area and the other diagonal divides the region into two triangles whose areas are not rational multiples of each other. In 2008, Dolgopyat-Fayad showed that outer billiards defined relative to the semidisk has unbounded orbits. The semidisk is the region one gets by cutting a disk in half. The proof of Dolgopyat-Fayad is robust, and also works for regions obtained by cutting a disk nearly in half, when the word nearly is suitably interpreted.
unbounded orbits in the hyperbolic plane 
In 2003, Dogru and Tabachnikov showed that all orbits are unbounded for a certain class of convex polygons in the hyperbolic plane. The authors call such polygons large. (See the reference for the definition.)
Existence of Periodic Orbits 
In ordinary polygonal billiards, the existence of periodic orbits is a major unsolved problem. For instance, it is unknown if every triangular shaped table has a periodic billiard path. More progress has been made for outer billiards, though the situation is far from well-understood. As mentioned above, all the orbits are periodic when the system is defined relative to a convex rational polygon in the Euclidean plane. Moreover, it is a recent theorem of C. Culter (written up by S. Tabachnikov) that outer billiards relative to any convex polygon has periodic orbits—in fact a periodic orbit outside of any given bounded region.
Open Questions 
Outer billiards is a subject still in its beginning phase. Most problems are still unsolved. Here are some open problems in the area.
- Show that outer billiards relative to a regular polygon has almost every orbit periodic. The cases of the equilaterial triangle and the square are trivial, and Tabachnikov answered this for the regular pentagon. These are the only cases known.
- more broadly, characterize the structure of the set of periodic orbits relative to the typical convex polygon.
- understand the structure of periodic orbits relative to simple shapes in the hyperbolic plane, such as small equilateral triangles.
- B. H. Neumann (25 Jan 1959). "Sharing Ham and Eggs". Iota (the Manchester University Mathematics Students' journal).
- J. Moser (1973). Stable and random motions in dynamical systems. Annals of Mathematics Studies 77. Princeton University Press.
- J. Moser (1978). "Is the Solar System Stable?". Mathematical Intelligencer 1 (2): 65–71. doi:10.1007/BF03023062.
- R. Douady (1982). "these de 3-eme cycle". University of Paris 7.
- F. Vivaldi, A. Shaidenko (1987). "Global Stability of a class of discontinuous billiards". Comm. Math. Phys. 110 (4): 625–640. doi:10.1007/BF01205552.
- Kolodziej (1989). "The antibilliard outside a polygon". Bull. Polish Acad. Sci. Math. 34: 163–168.
- E Gutkin and N Simanyi (1991). "Dual polygonal billiard and necklace dynamics". Comm. Math. Phys. 143 (3): 431–450. doi:10.1007/BF02099259.
- S. Tabachnikov (1995). Billiards. SMF Panoramas et Syntheses. ISBN 2-85629-030-2.
- P. Boyland (1996). "Dual billiards, twist maps, and impact oscillators". Nonlinearity 9 (6): 1411–1438. doi:10.1088/0951-7715/9/6/002.
- S. Tabachnikov (2002). "Dual Billiards in the Hyperbolic Plane". Nonlinearity 15 (4): 1051–1072. doi:10.1088/0951-7715/15/4/305.
- F. Dogru, S. Tabachnikov (2003). "On Polygonal Dual Billiards in the Hyperbolic Plane". Regular Chaotic Dynamics 8: 67–82. doi:10.1070/RD2003v008n01ABEH000226.
- D. Genin (2005). "regular and chaotic dynamics of outer billiards". Penn State Ph.D. Thesis.
- R.E. Schwartz (2007). "unbounded orbits for outer billiards". Journal of Modern Dynamics 3.
- R.E. Schwartz (2009). outer billiards on kites. Annals of Math Studies 171. Princeton University Press.
- D. Dolgopyat and B Fayad (2009). "unbounded orbits for semicircular outer billiards". Annales Henri Poincare.
- S. Tabachnikov (2008). "A proof of Culter's theorem on existence of periodic orbits in polygonal outer billiards". Geom. Dedicata.