Geographical distance

Geographical distance or geodetic distance is the distance measured along the surface of the Earth, or the shortest arch length.

View from the Swabian Jura to the Alps

The formulae in this article calculate distances between points which are defined by geographical coordinates in terms of latitude and longitude. This distance is an element in solving the second (inverse) geodetic problem.

Introduction

Calculating the distance between geographical coordinates is based on some level of abstraction; it does not provide an exact distance, which is unattainable if one attempted to account for every irregularity in the surface of the Earth.^[1] Common abstractions for the surface between two geographic points are:

• Flat surface;
• Spherical surface;
• Ellipsoidal surface.

All abstractions above ignore changes in elevation. Calculation of distances which account for changes in elevation relative to the idealized surface are not discussed in this article.

Classification of Formulae based on Approximation

• Tunnel-distance based approximations: Flat surface, Gauss-mid-latitude; ${\displaystyle |\Delta D_{\text{error}}|\propto D^{3}}$
• 0-th-order approximation: Spherical surface; ${\displaystyle |\Delta D_{\text{error}}|\propto D}$
• higher-order approximations based on Ellipsoid: ${\displaystyle f^{1}}$: Andoyer(1932); Andoyer-Lambert(1942), ${\displaystyle f^{2}}$: Andoyer-Lambert-Thomas(1970), ${\displaystyle f^{3}}$: Vincenty(1975), ${\displaystyle f^{6}}$: Kaney(2011); ${\displaystyle \Delta |D_{\text{error}}|\propto D}$ on the hemisphere

The theoretical estimations of error are added in above and ${\displaystyle f}$ is the flattening of the Earth.

Nomenclature

Arc distance, ${\displaystyle D,\,\!}$ is the minimum distance along the surface of sphere/ellipsoid calculated between two points, ${\displaystyle P_{1}\,\!}$ and ${\displaystyle P_{2}\,\!}$. Whereas, the tunnel distance, or chord length, ${\displaystyle D_{\textrm {t}}}$, is measured along Cartesian straight line. The geographical coordinates of the two points, as (latitude, longitude) pairs, are ${\displaystyle (\phi _{1},\lambda _{1})\,\!}$ and ${\displaystyle (\phi _{2},\lambda _{2}),\,\!}$ respectively. Which of the two points is designated as ${\displaystyle P_{1}\,\!}$ is not important for the calculation of distance.

Latitude ${\displaystyle \phi \,\!}$ and longitude ${\displaystyle \lambda \,\!}$ coordinates on maps are usually expressed in degrees. In the given forms of the formulae below, one or more values must be expressed in the specified units to obtain the correct result. Where geographic coordinates are used as the argument of a trigonometric function, the values may be expressed in any angular units compatible with the method used to determine the value of the trigonometric function. Many electronic calculators allow calculations of trigonometric functions in either degrees or radians. The calculator mode must be compatible with the units used for geometric coordinates.

Differences in latitude and longitude are labeled and calculated as follows:

${\displaystyle {\begin{aligned}\Delta \phi &=\phi _{2}-\phi _{1};\\\Delta \lambda &=\lambda _{2}-\lambda _{1}.\end{aligned}}\,\!}$

It is not important whether the result is positive or negative when used in the formulae below.

"Mean latitude" is labeled and calculated as follows:

${\displaystyle \phi _{\mathrm {m} }={\frac {\phi _{1}+\phi _{2}}{2}}.\,\!}$

Unless specified otherwise, the radius of the Earth for the calculations below is:

${\displaystyle R\,\!}$ = 6,371.009 kilometers = 3,958.761 statute miles = 3,440.069 nautical miles.

${\displaystyle D_{\,}\!}$ = Distance between the two points, as measured along the surface of the Earth and in the same units as the value used for radius unless specified otherwise.

Singularities and discontinuity of latitude/longitude

The approximation of sinusoidal functions of ${\displaystyle \Delta \lambda }$, appearing in some flat-surface formulae below, may induce singularity and discontinuity. It may also degrade the accuracy in the case of higher latitude.

Longitude has singularities at the Poles (longitude is undefined) and a discontinuity at the ±180° meridian. Also, planar projections of the circles of constant latitude are highly curved near the Poles. Hence, the above equations for delta latitude/longitude (${\displaystyle \Delta \phi \!}$, ${\displaystyle \Delta \lambda \!}$) and mean latitude (${\displaystyle \phi _{\mathrm {m} }\!}$) may not give the expected answer for positions near the Poles or the ±180° meridian. Consider e.g. the value of ${\displaystyle \Delta \lambda \!}$ ("east displacement") when ${\displaystyle \lambda _{1}\!}$ and ${\displaystyle \lambda _{2}\!}$ are on either side of the ±180° meridian, or the value of ${\displaystyle \phi _{\mathrm {m} }\!}$ ("mean latitude") for the two positions (${\displaystyle \phi _{1}\!}$=89°, ${\displaystyle \lambda _{1}\!}$=45°) and (${\displaystyle \phi _{2}\!}$=89°, ${\displaystyle \lambda _{2}\!}$=−135°).

If a calculation based on latitude/longitude should be valid for all Earth positions, it should be verified that the discontinuity and the Poles are handled correctly. Another solution is to use n-vector instead of latitude/longitude, since this representation does not have discontinuities or singularities.

Flat-surface approximation formulae for very short distance

A planar approximation for the surface of the Earth may be useful over very small distances. It approximates the arc length, ${\displaystyle D}$, to the tunnel distance, ${\displaystyle D_{\textrm {t}}}$, or omits the conversion between arc and chord lengths shown below.

The shortest distance between two points in plane is a Cartesian straight line. The Pythagorean theorem is used to calculate the distance between points in a plane.

Even over short distances, the accuracy of geographic distance calculations which assume a flat Earth depend on the method by which the latitude and longitude coordinates have been projected onto the plane. The projection of latitude and longitude coordinates onto a plane is the realm of cartography.

The formulae presented in this section provide varying degrees of accuracy.

Spherical Earth approximation formulae

This formula takes into account the variation in distance between meridians with latitude:

${\displaystyle {\begin{aligned}D&=R{\sqrt {\left(2\sin {\frac {\Delta \phi }{2}}\,\cos {\frac {\Delta \lambda }{2}}\right)^{2}+\left(2\cos \phi _{\textrm {m}}\sin {\frac {\Delta \lambda }{2}}\right)^{2}}}\\&\approx R{\sqrt {\left(\Delta \phi \,\cos {\frac {\Delta \lambda }{2}}\right)^{2}+\left(2\cos \phi _{\textrm {m}}\sin {\frac {\Delta \lambda }{2}}\right)^{2}}}\ .\end{aligned}}}$

The square root appearing above can be eliminated for such application as ordering locations by distance in a database query.

In the case of medium or low latitude

The above is furthermore simplified by approximating sinusoidal functions of ${\displaystyle {\frac {\Delta \lambda }{2}}}$, justified except for high latitude:

${\displaystyle D=R{\sqrt {(\Delta \phi )^{2}+(\cos(\phi _{\mathrm {m} })\Delta \lambda )^{2}}}}$.

Ellipsoidal Earth approximation formulae

The above formula is extended for ellipsoidal Earth:

${\displaystyle D={\sqrt {\left(M\left(\phi _{\textrm {m}}\right)\Delta \phi \,\cos {\frac {\Delta \lambda }{2}}\right)^{2}+\left(2N\left(\phi _{\textrm {m}}\right)\cos \phi _{\textrm {m}}\sin {\frac {\Delta \lambda }{2}}\right)^{2}}}}$,

where ${\displaystyle M\,\!}$ and ${\displaystyle N\,\!}$ are the meridional and its perpendicular, or "normal", radii of curvature of Earth (See also "Geographic coordinate conversion" for their formulas).

It is derived by the approximation of ${\displaystyle \left(\cos \phi _{\textrm {m}}\sin {\frac {\Delta \lambda }{2}}\Delta \phi \right)^{2}\approx 0}$ in the square root.

In the case of medium or low latitude

The above is furthermore simplified by approximating sinusoidal functions of ${\displaystyle {\frac {\Delta \lambda }{2}}}$, justified except for high latitude as above:^[2][3]

${\displaystyle D={\sqrt {(M(\phi _{\mathrm {m} })\Delta \phi )^{2}+(N(\phi _{\mathrm {m} })\cos \phi _{\mathrm {m} }\Delta \lambda )^{2}}}.}$

FCC's formula

The FCC prescribes the following formulae for distances not exceeding 475 kilometres (295 mi):^[4]

${\displaystyle D={\sqrt {(K_{1}\Delta \phi )^{2}+(K_{2}\Delta \lambda )^{2}}},}$

where

${\displaystyle D\,\!}$ = Distance in kilometers;

${\displaystyle \Delta \phi \,\!}$ and ${\displaystyle \Delta \lambda \,\!}$ are in degrees;

${\displaystyle \phi _{\mathrm {m} }\,\!}$ must be in units compatible with the method used for determining ${\displaystyle \cos \phi _{\mathrm {m} };\,\!}$

${\displaystyle {\begin{aligned}K_{1}&=111.13209-0.56605\cos(2\phi _{\mathrm {m} })+0.00120\cos(4\phi _{\mathrm {m} });\\K_{2}&=111.41513\cos(\phi _{\mathrm {m} })-0.09455\cos(3\phi _{\mathrm {m} })+0.00012\cos(5\phi _{\mathrm {m} }).\end{aligned}}\,\!}$

Where ${\displaystyle K_{1}}$ and ${\displaystyle K_{2}}$ are in units of kilometers per arc degree. They are derived from radii of curvature of Earth as follows:

${\displaystyle K_{1}=M(\phi _{\mathrm {m} }){\frac {\pi }{180}}\,\!}$ = kilometers per arc degree of latitude difference;

${\displaystyle K_{2}=\cos(\phi _{\mathrm {m} })N(\phi _{\mathrm {m} }){\frac {\pi }{180}}\,\!}$ = kilometers per arc degree of longitude difference;

Note that the expressions in the FCC formula are derived from the truncation of the binomial series expansion form of ${\displaystyle M\,\!}$ and ${\displaystyle N\,\!}$, set to the Clarke 1866 reference ellipsoid. For a more computationally efficient implementation of the formula above, multiple applications of cosine can be replaced with a single application and use of recurrence relation for Chebyshev polynomials.

Polar coordinate flat-Earth formula

${\displaystyle D=R{\sqrt {\theta _{1}^{2}\;{\boldsymbol {+}}\;\theta _{2}^{2}\;\mathbf {-} \;2\theta _{1}\theta _{2}\cos(\Delta \lambda )}},}$

where the colatitude values are in radians: ${\displaystyle \theta ={\frac {\pi }{2}}-\phi .}$

For a latitude measured in degrees, the colatitude in radians may be calculated as follows: ${\displaystyle \theta ={\frac {\pi }{180}}(90^{\circ }-\phi ).\,\!}$

Spherical-surface formulae

Main article: Great-circle distance

If one is willing to accept a possible error of 0.5%, one can use formulas of spherical trigonometry on the sphere that best approximates the surface of the Earth.

The shortest distance along the surface of a sphere between two points on the surface is along the great-circle which contains the two points.

The great-circle distance article gives the formula for calculating the shortest arch length ${\displaystyle D}$ on a sphere about the size of the Earth. That article includes an example of the calculation. For example, from tunnel distance ${\displaystyle D_{\textrm {t}}}$,

${\displaystyle D=2R\arcsin {\frac {D_{\textrm {t}}}{2R}}.}$

For short distances (${\displaystyle D\ll R}$),

${\displaystyle D=D_{\textrm {t}}\left(1+{\frac {1}{24}}\left({\frac {D_{\textrm {t}}}{R}}\right)^{2}+\cdots \right).}$

Tunnel distance

A tunnel between points on Earth is defined by a Cartesian line through three-dimensional space between the points of interest. The tunnel distance ${\displaystyle D_{\textrm {t}}=2R\sin {\frac {D}{2R}}}$ is the great-circle chord length and may be calculated as follows for the corresponding unit sphere:

${\displaystyle {\begin{aligned}\Delta {X}&=\cos(\phi _{2})\cos(\lambda _{2})-\cos(\phi _{1})\cos(\lambda _{1});\\\Delta {Y}&=\cos(\phi _{2})\sin(\lambda _{2})-\cos(\phi _{1})\sin(\lambda _{1});\\\Delta {Z}&=\sin(\phi _{2})-\sin(\phi _{1});\\D_{\textrm {t}}&=R{\sqrt {(\Delta {X})^{2}+(\Delta {Y})^{2}+(\Delta {Z})^{2}}}\\&=2R{\sqrt {\sin ^{2}{\frac {\Delta \phi }{2}}+\left(\cos ^{2}{\frac {\Delta \phi }{2}}-\sin ^{2}\phi _{\textrm {m}}\right)\sin ^{2}{\frac {\Delta \lambda }{2}}}}\\&=2R{\sqrt {\left(\sin {\frac {\Delta \lambda }{2}}\cos \phi _{\textrm {m}}\right)^{2}+\left(\cos {\frac {\Delta \lambda }{2}}\sin {\frac {\Delta \phi }{2}}\right)^{2}}}.\end{aligned}}}$

Ellipsoidal-surface formulae

Main article: Geodesics on an ellipsoid

Geodesic on an oblate ellipsoid

An ellipsoid approximates the surface of the Earth much better than a sphere or a flat surface does. The shortest distance along the surface of an ellipsoid between two points on the surface is along the geodesic. Geodesics follow more complicated paths than great circles and in particular, they usually don't return to their starting positions after one circuit of the Earth. This is illustrated in the figure on the right where f is taken to be 1/50 to accentuate the effect. Finding the geodesic between two points on the Earth, the so-called inverse geodetic problem, was the focus of many mathematicians and geodesists over the course of the 18th and 19th centuries with major contributions by Clairaut,^[5] Legendre,^[6] Bessel,^[7] and Helmert English translation of Astron. Nachr. 4, 241–254 (1825). Errata.^[8] Rapp^[9] provides a good summary of this work.

Methods for computing the geodesic distance are widely available in geographical information systems, software libraries, standalone utilities, and online tools. The most widely used algorithm is by Vincenty,^[10] who uses a series which is accurate to third order in the flattening of the ellipsoid, i.e., about 0.5 mm; however, the algorithm fails to converge for points that are nearly antipodal. (For details, see Vincenty's formulae.) This defect is cured in the algorithm given by Karney,^[11] who employs series which are accurate to sixth order in the flattening. This results in an algorithm which is accurate to full double precision and which converges for arbitrary pairs of points on the Earth. This algorithm is implemented in GeographicLib.^[12]

The exact methods above are feasible when carrying out calculations on a computer. They are intended to give millimeter accuracy on lines of any length; one can use simpler formulas if one doesn't need millimeter accuracy, or if one does need millimeter accuracy but the line is short.

The short-line methods have been studied by several researchers. Rapp,^[13] Chap. 6, describes the Puissant method, the Gauss mid-latitude method, and the Bowring method.^[14] Karl Hubeny^[15] got the expanded series of Gauss mid-latitude one represented as the correction to flat-surface one.

Lambert's formula for long lines

Historically, the long-line formulae were derived in the form of expansion series with regard to flattening ${\displaystyle f}$.^[16][17]

Lambert's formulae^[18] use the first-order correction and reduced latitude, ${\displaystyle \beta =\arctan \left((1-f)\tan \phi \right)}$, for better accuracy. They give accuracy on the order of 10 meters over thousands of kilometers.

First convert the latitudes ${\displaystyle \scriptstyle \phi _{1}}$, ${\displaystyle \scriptstyle \phi _{2}}$ of the two points to reduced latitudes ${\displaystyle \scriptstyle \beta _{1}}$, ${\displaystyle \scriptstyle \beta _{2}}$. Then calculate the central angle ${\displaystyle \sigma }$ in radians between two points ${\displaystyle (\beta _{1},\;\lambda _{1})}$ and ${\displaystyle (\beta _{2},\;\lambda _{2})}$ on a sphere using the Great-circle distance method (haversine formula), with longitudes ${\displaystyle \lambda _{1}\;}$ and ${\displaystyle \lambda _{2}\;}$ being the same on the sphere as on the spheroid.

${\displaystyle P={\frac {\beta _{1}+\beta _{2}}{2}}\qquad Q={\frac {\beta _{2}-\beta _{1}}{2}}}$

${\displaystyle X=(\sigma -\sin \sigma ){\frac {\sin ^{2}P\cos ^{2}Q}{\cos ^{2}{\frac {\sigma }{2}}}}\qquad \qquad Y=(\sigma +\sin \sigma ){\frac {\cos ^{2}P\sin ^{2}Q}{\sin ^{2}{\frac {\sigma }{2}}}}}$

${\textstyle D=a{\bigl (}\sigma -{\tfrac {f}{2}}(X+Y){\bigr )}}$,

where ${\displaystyle a}$ is the equatorial radius of the chosen spheroid.

On the GRS 80 spheroid Lambert's formula is off by

0 North 0 West to 40 North 120 West, 12.6 meters

0N 0W to 40N 60W, 6.6 meters

40N 0W to 40N 60W, 0.85 meter

Gauss mid-latitude method for short lines

It has the similar form of the arc length converted from tunnel distance. Detailed formulas are given by Rapp,^[13] §6.4. It is consistent with the above-mentioned flat-surface formulae apparently.

${\displaystyle D=2N\left(\phi _{\textrm {m}}\right)\arcsin {\sqrt {\left(\sin {\frac {\Delta \lambda }{2}}\cos \phi _{\textrm {m}}\right)^{2}+\left(\cos {\frac {\Delta \lambda }{2}}\sin \left({\frac {\Delta \phi }{2}}{\frac {M\left(\phi _{\textrm {m}}\right)}{N\left(\phi _{\textrm {m}}\right)}}\right)\right)^{2}}}.}$

Bowring's method for short lines

Bowring maps the points to a sphere of radius R′, with latitude and longitude represented as φ′ and λ′. Define

${\displaystyle A={\sqrt {1+e'^{2}\cos ^{4}\phi _{1}}},\quad B={\sqrt {1+e'^{2}\cos ^{2}\phi _{1}}},}$

where the second eccentricity squared is

${\displaystyle e'^{2}={\frac {a^{2}-b^{2}}{b^{2}}}={\frac {f(2-f)}{(1-f)^{2}}}.}$

The spherical radius is

${\displaystyle R'={\frac {\sqrt {1+e'^{2}}}{B^{2}}}a.}$

(The Gaussian curvature of the ellipsoid at φ₁ is 1/R′².) The spherical coordinates are given by

${\displaystyle {\begin{aligned}\tan \phi _{1}'&={\frac {\tan \phi _{1}}{B}},\\\Delta \phi '&={\frac {\Delta \phi }{B}}{\biggl [}1+{\frac {3e'^{2}}{4B^{2}}}(\Delta \phi )\sin(2\phi _{1}+{\tfrac {2}{3}}\Delta \phi ){\biggr ]},\\\Delta \lambda '&=A\Delta \lambda ,\end{aligned}}}$

where ${\displaystyle \Delta \phi =\phi _{2}-\phi _{1}}$, ${\displaystyle \Delta \phi '=\phi _{2}'-\phi _{1}'}$, ${\displaystyle \Delta \lambda =\lambda _{2}-\lambda _{1}}$, ${\displaystyle \Delta \lambda '=\lambda _{2}'-\lambda _{1}'}$. The resulting problem on the sphere may be solved using the techniques for great-circle navigation to give approximations for the spheroidal distance and bearing. Detailed formulas are given by Rapp^[13] §6.5, Bowring,^[14] and Karney.^[19]

Altitude correction

The variation in altitude from the topographical or ground level down to the sphere's or ellipsoid's surface, also changes the scale of distance measurements.^[20] The slant distance s (chord length) between two points can be reduced to the arc length on the ellipsoid surface S as:^[21]

${\displaystyle S-s=-0.5(h_{1}+h_{2})s/R-0.5(h_{1}-h_{2})^{2}/s}$

where R is evaluated from Earth's azimuthal radius of curvature and h are ellipsoidal heights are each point. The first term on the right-hand side of the equation accounts for the mean elevation and the second term for the inclination. A further reduction of the above Earth normal section length to the ellipsoidal geodesic length is often negligible.^[21]

See also

• Arc measurement
• Earth radius
• Spherical Earth
• Great-circle distance
• Great-circle navigation
• Ground sample distance
• Vincenty's formulae
• Meridian arc
• Scale (map)

References

1. "The British Cartographic Society > How long is the UK coastline?". Archived from the original on 2012-05-22. Retrieved 2008-12-06.
2. Williams, E. (2013). "Aviation Formulary". Retrieved 2024-06-23.
3. Williams, E. (2002). "Navigation on the spheroidal earth". Retrieved 2023-11-28.
4. "Reference points and distance computations" (PDF). Code of Federal Regulations (Annual Edition). Title 47: Telecommunication. 73 (208). October 1, 2016. Retrieved 8 November 2017.
5. Clairaut, A. C. (1735). "Détermination géometrique de la perpendiculaire à la méridienne tracée par M. Cassini" [Geometrical determination of the perpendicular to the meridian drawn by Jacques Cassini]. Mémoires de l'Académie Royale des Sciences de Paris 1733 (in French): 406–416.
6. Legendre, A. M. (1806). "Analyse des triangles tracées sur la surface d'un sphéroïde" [Analysis of spheroidal triangles]. Mémoires de l'Institut National de France (in French) (1st semester): 130–161.
7. Bessel, F. W. (2010) [1825]. "The calculation of longitude and latitude from geodesic measurements". Astronomische Nachrichten. 331 (8). . Translated by C. F. F. Karney & R. E. Deakin: 852–861. arXiv:0908.1824. Bibcode:2010AN....331..852K. doi:10.1002/asna.201011352. S2CID 118760590.{{cite journal}}: CS1 maint: postscript (link)
8. Helmert, F. R. (1964) [1880]. Mathematical and Physical Theories of Higher Geodesy. Vol. 1. St. Louis: Aeronautical Chart and Information Center.{{cite book}}: CS1 maint: postscript (link) English translation of Die Mathematischen und Physikalischen Theorieen der Höheren Geodäsie, Vol. 1 (Teubner, Leipzig, 1880).
9. Rapp, R. H. (March 1993). Geometric Geodesy, Part II (Technical report). Ohio State University. Retrieved 2011-08-01.
10. Vincenty, T. (April 1975). "Direct and Inverse Solutions of Geodesics on the Ellipsoid with application of nested equations" (PDF). Survey Review. 23 (176): 88–93. doi:10.1179/sre.1975.23.176.88. Retrieved 2009-07-11. Addendum: Survey Review 23 (180): 294 (1976).{{cite journal}}: CS1 maint: postscript (link)
11. Karney, C. F. F. (2013). "Algorithms for geodesics". Journal of Geodesy. 87 (1): 43–55. arXiv:1109.4448. Bibcode:2013JGeod..87...43K. doi:10.1007/s00190-012-0578-z. S2CID 119310141. – (open access). Addenda.
12. Karney, C. F. F. (2013). "GeographicLib". 1.32.
13. Rapp, R, H (1991). Geometric Geodesy, Part I (Report). Ohio Start Univ. hdl:1811/24333.{{cite report}}: CS1 maint: multiple names: authors list (link)
14. Bowring, B. R. (1981). "The direct and inverse problems for short geodesic lines on the ellipsoid". Surveying and Mapping. 41 (2): 135–141.
15. Hubeny, K. (1954). Entwicklung der Gauss'schen Mittelbreitenformeln, Österreichische Zeitschrift für Vermessungswesen.
16. Forsyth, A. R. (1927). Calculus of Variations. Cambridge Univ. Press. ISBN 978-1-107-64083-2. OCLC 250050479..
17. Henri Andoyer: Formule donnant la longueur de la géodésique joignant 2 points de l’ellipsoïde donnés par leurs coordonnées géographiques, Bulletin Géodésique, Volume 34, Number 1, April 1932, pages 77–81, https://doi.org/10.1007%2FBF03030136
18. Lambert, W. D (1942). "The distance between two widely separated points on the surface of the earth". J. Washington Academy of Sciences. 32 (5): 125–130.
19. "GeographicLib: Geodesics on an ellipsoid of revolution". geographiclib.sourceforge.io. Retrieved 2024-08-04.
20. "Archived copy" (PDF). Archived from the original (PDF) on 2014-08-27. Retrieved 2014-08-26.{{cite web}}: CS1 maint: archived copy as title (link)
21. Torge & Müller (2012) Geodesy, De Gruyter, p.249

External links

• An online geodesic calculator (based on GeographicLib).
• An online geodesic bibliography.