emmtrix Wiki - User contributions [en]

tanh Software Implementation

2026-02-04T12:20:11Z

Timo.stripf:

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<big><math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math></big>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==

Localized approximation functions are designed to provide highly accurate and computationally efficient evaluations of <code>tanh(x)</code> within a restricted input range.
They are typically used as building blocks in range-based dispatch strategies, where different approximations are selected depending on the magnitude of the input value.

By limiting the valid input range, these approximations can achieve very low ULP error with simple polynomial or rational expressions, avoiding expensive transcendental functions.

{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NaN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lambert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

The functions listed above are not intended to be used independently over the full input domain.
Instead, they are commonly combined with explicit range checks and sign handling to form a complete <code>tanh</code> implementation.

For example, low-order polynomials are well suited for very small <math>|x|</math>, while rational or higher-order approximations extend the usable range before switching to exponential-based formulations or saturation.

==Recommended Implementation Strategy==
A robust and high-performance <code>tanh(x)</code> implementation typically combines multiple techniques, each optimized for a specific input range. No single formulation provides optimal accuracy and performance across the entire floating-point domain.

A common strategy is based on range-dependent dispatching:
*'''Sign handling'''
: Use the odd symmetry of <code>tanh</code> (<math>\tanh(-x) = -\tanh(x)</math>) to reduce the implementation to non-negative inputs.*'''Very small inputs (<math>|x| \ll 1</math>)'''
: Use a low-order polynomial or Taylor approximation, exploiting <math>\tanh(x) \approx x</math> to avoid cancellation and expensive transcendental functions.
*'''Small to medium inputs'''
: Apply a localized polynomial or rational approximation with bounded ULP error.
*'''Medium to large inputs'''
: Use an <code>expm1</code>-based formulation to maintain numerical stability while avoiding catastrophic cancellation.
*'''Very large inputs'''
: Directly return <code>±1</code>, as <code>tanh(x)</code> saturates rapidly and additional computation provides no meaningful benefit.
This hybrid approach minimizes ULP error, avoids NaN generation, and allows fine-grained control over the accuracy–performance trade-off. It is widely used in optimized math libraries and hardware-accelerated implementations.

== See Also ==

* [[ULP Difference of Float Numbers]]

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:18:52Z

Timo.stripf: /* External Links */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==

Localized approximation functions are designed to provide highly accurate and computationally efficient evaluations of <code>tanh(x)</code> within a restricted input range.
They are typically used as building blocks in range-based dispatch strategies, where different approximations are selected depending on the magnitude of the input value.

By limiting the valid input range, these approximations can achieve very low ULP error with simple polynomial or rational expressions, avoiding expensive transcendental functions.

{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NaN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lambert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

The functions listed above are not intended to be used independently over the full input domain.
Instead, they are commonly combined with explicit range checks and sign handling to form a complete <code>tanh</code> implementation.

For example, low-order polynomials are well suited for very small <math>|x|</math>, while rational or higher-order approximations extend the usable range before switching to exponential-based formulations or saturation.

==Recommended Implementation Strategy==
A robust and high-performance <code>tanh(x)</code> implementation typically combines multiple techniques, each optimized for a specific input range. No single formulation provides optimal accuracy and performance across the entire floating-point domain.

A common strategy is based on range-dependent dispatching:
*'''Sign handling'''
: Use the odd symmetry of <code>tanh</code> (<math>\tanh(-x) = -\tanh(x)</math>) to reduce the implementation to non-negative inputs.*'''Very small inputs (<math>|x| \ll 1</math>)'''
: Use a low-order polynomial or Taylor approximation, exploiting <math>\tanh(x) \approx x</math> to avoid cancellation and expensive transcendental functions.
*'''Small to medium inputs'''
: Apply a localized polynomial or rational approximation with bounded ULP error.
*'''Medium to large inputs'''
: Use an <code>expm1</code>-based formulation to maintain numerical stability while avoiding catastrophic cancellation.
*'''Very large inputs'''
: Directly return <code>±1</code>, as <code>tanh(x)</code> saturates rapidly and additional computation provides no meaningful benefit.
This hybrid approach minimizes ULP error, avoids NaN generation, and allows fine-grained control over the accuracy–performance trade-off. It is widely used in optimized math libraries and hardware-accelerated implementations.

== See Also ==

* [[ULP Difference of Float Numbers]]

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:14:40Z

Timo.stripf: /* Recommended Implementation Strategy */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==

Localized approximation functions are designed to provide highly accurate and computationally efficient evaluations of <code>tanh(x)</code> within a restricted input range.
They are typically used as building blocks in range-based dispatch strategies, where different approximations are selected depending on the magnitude of the input value.

By limiting the valid input range, these approximations can achieve very low ULP error with simple polynomial or rational expressions, avoiding expensive transcendental functions.

{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NaN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lambert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

The functions listed above are not intended to be used independently over the full input domain.
Instead, they are commonly combined with explicit range checks and sign handling to form a complete <code>tanh</code> implementation.

For example, low-order polynomials are well suited for very small <math>|x|</math>, while rational or higher-order approximations extend the usable range before switching to exponential-based formulations or saturation.

==Recommended Implementation Strategy==
A robust and high-performance <code>tanh(x)</code> implementation typically combines multiple techniques, each optimized for a specific input range. No single formulation provides optimal accuracy and performance across the entire floating-point domain.

A common strategy is based on range-dependent dispatching:
*'''Sign handling'''
: Use the odd symmetry of <code>tanh</code> (<math>\tanh(-x) = -\tanh(x)</math>) to reduce the implementation to non-negative inputs.*'''Very small inputs (<math>|x| \ll 1</math>)'''
: Use a low-order polynomial or Taylor approximation, exploiting <math>\tanh(x) \approx x</math> to avoid cancellation and expensive transcendental functions.
*'''Small to medium inputs'''
: Apply a localized polynomial or rational approximation with bounded ULP error.
*'''Medium to large inputs'''
: Use an <code>expm1</code>-based formulation to maintain numerical stability while avoiding catastrophic cancellation.
*'''Very large inputs'''
: Directly return <code>±1</code>, as <code>tanh(x)</code> saturates rapidly and additional computation provides no meaningful benefit.
This hybrid approach minimizes ULP error, avoids NaN generation, and allows fine-grained control over the accuracy–performance trade-off. It is widely used in optimized math libraries and hardware-accelerated implementations.

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:14:06Z

Timo.stripf: /* External Links */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==

Localized approximation functions are designed to provide highly accurate and computationally efficient evaluations of <code>tanh(x)</code> within a restricted input range.
They are typically used as building blocks in range-based dispatch strategies, where different approximations are selected depending on the magnitude of the input value.

By limiting the valid input range, these approximations can achieve very low ULP error with simple polynomial or rational expressions, avoiding expensive transcendental functions.

{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NaN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lambert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

The functions listed above are not intended to be used independently over the full input domain.
Instead, they are commonly combined with explicit range checks and sign handling to form a complete <code>tanh</code> implementation.

For example, low-order polynomials are well suited for very small <math>|x|</math>, while rational or higher-order approximations extend the usable range before switching to exponential-based formulations or saturation.

==Recommended Implementation Strategy==
A robust and high-performance <code>tanh(x)</code> implementation typically combines multiple techniques, each optimized for a specific input range. No single formulation provides optimal accuracy and performance across the entire floating-point domain.

A common strategy is based on range-dependent dispatching:
*'''Sign handling'''
Use the odd symmetry of <code>tanh</code> (<math>\tanh(-x) = -\tanh(x)</math>) to reduce the implementation to non-negative inputs.*'''Very small inputs (<math>|x| \ll 1</math>)'''
Use a low-order polynomial or Taylor approximation, exploiting <math>\tanh(x) \approx x</math> to avoid cancellation and expensive transcendental functions.
*'''Small to medium inputs'''
Apply a localized polynomial or rational approximation with bounded ULP error.
*'''Medium to large inputs'''
Use an <code>expm1</code>-based formulation to maintain numerical stability while avoiding catastrophic cancellation.
*'''Very large inputs'''
Directly return <code>±1</code>, as <code>tanh(x)</code> saturates rapidly and additional computation provides no meaningful benefit.
This hybrid approach minimizes ULP error, avoids NaN generation, and allows fine-grained control over the accuracy–performance trade-off. It is widely used in optimized math libraries and hardware-accelerated implementations.
== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:09:20Z

Timo.stripf: /* Localized Approximation Functions */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==

Localized approximation functions are designed to provide highly accurate and computationally efficient evaluations of <code>tanh(x)</code> within a restricted input range.
They are typically used as building blocks in range-based dispatch strategies, where different approximations are selected depending on the magnitude of the input value.

By limiting the valid input range, these approximations can achieve very low ULP error with simple polynomial or rational expressions, avoiding expensive transcendental functions.

{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NaN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lambert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

The functions listed above are not intended to be used independently over the full input domain.
Instead, they are commonly combined with explicit range checks and sign handling to form a complete <code>tanh</code> implementation.

For example, low-order polynomials are well suited for very small <math>|x|</math>, while rational or higher-order approximations extend the usable range before switching to exponential-based formulations or saturation.

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:08:41Z

Timo.stripf: /* Localized Approximation Functions */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==

Localized approximation functions are designed to provide highly accurate and computationally efficient evaluations of <code>tanh(x)</code> within a restricted input range.
They are typically used as building blocks in range-based dispatch strategies, where different approximations are selected depending on the magnitude of the input value.

By limiting the valid input range, these approximations can achieve very low ULP error with simple polynomial or rational expressions, avoiding expensive transcendental functions.

{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NaN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|Lambert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:05:47Z

Timo.stripf: /* Taylor */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

== Taylor Expansion (Local Approximation) ==

Taylor series approximations of <code>tanh(x)</code> expand the function around <math>x = 0</math> and therefore provide a very accurate approximation only for sufficiently small input magnitudes.

For small <math>|x|</math>, the hyperbolic tangent behaves approximately linear (<math>\tanh(x) \approx x</math>), and low-order Taylor polynomials can achieve excellent accuracy with minimal computational cost. This makes Taylor-based approximations attractive as a fast path in range-reduced implementations.

However, Taylor expansions have a strictly limited radius of convergence. Outside a narrow neighborhood around zero, the approximation error grows extremely rapidly and quickly becomes unusable. As a result, Taylor polynomials are not suitable as standalone implementations of <code>tanh</code>.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but very effective as a local approximation when combined with range-based dispatching.

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:03:01Z

Timo.stripf: /* Taylor */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

==Taylor==

The following graph shows the numerical error of a Taylor series approximation of <code>tanh(x)</code>.
Taylor expansions approximate the function locally around <math>x = 0</math> and are therefore only meaningful for very small input magnitudes.

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

'''Key observation:'''
The Taylor approximation is highly accurate near zero but diverges extremely fast outside its convergence region.
This makes it unsuitable as a standalone <code>tanh</code> implementation, but useful as a local approximation in range-based implementations.

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T12:01:31Z

Timo.stripf: /* exp_v2 */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

The following error graph shows the ULP error of the <code>exp_v2</code> formulation across the full <code>float32</code> input range.
This formulation reduces the number of exponential evaluations to one, but is particularly sensitive to catastrophic cancellation for very small input values and to overflow for large <math>|x|</math>.

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
For <math>|x| \ll 1</math>, the ULP error grows rapidly and reaches extremely large values due to loss of significance.
For sufficiently large <math>x</math>, both numerator and denominator overflow, leading to NaN results.

=== expm1 ===

This graph visualizes the ULP error of the <code>expm1</code>-based formulation.
Using <code>expm1</code> significantly improves numerical accuracy for small input values by avoiding catastrophic cancellation in the expression <math>e^{2x} - 1</math>.

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

'''Key observation:'''
Compared to direct exponential formulations, the <code>expm1</code> approach maintains low ULP error for small and medium values of <math>x</math>.
Overflow is still present for very large inputs, but the overall error behavior is substantially more stable.

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T11:58:42Z

Timo.stripf: /* Numerical Issues and Error Metrics */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

== Numerical Issues and Error Metrics ==

To evaluate and compare different software implementations of <code>tanh</code>, several numerical error metrics are used throughout this article. These metrics make it possible to reason about accuracy, robustness, and failure modes in floating-point arithmetic.

=== ULP Error ===
ULP (Unit in the Last Place) measures the distance between two adjacent representable floating-point numbers. The ULP error of a computed result is defined as the difference, measured in ULPs, between the computed value and the correctly rounded reference value.

An error of <code>ULP ≤ 2</code> is commonly considered acceptable for single-precision transcendental functions, as it indicates that the result is very close to the correctly rounded value.

Large ULP errors typically indicate catastrophic cancellation, loss of significant bits, or severe rounding effects.

=== NaN and Overflow ===
NaN (Not a Number) results occur when undefined floating-point operations are performed, such as <code>inf / inf</code>. In <code>tanh</code> implementations, this often happens when both the numerator and denominator of a formulation overflow to infinity.

Tracking NaN generation is critical, as NaNs can silently propagate through numerical pipelines and invalidate downstream results.

=== Error Graphs ===
The error graphs shown in the following sections visualize ULP error across the full <code>float32</code> input domain. They make it possible to identify:
* regions with catastrophic cancellation,
* input ranges that trigger overflow or NaNs,
* and numerical stability differences between formulations.

These graphs are a key tool for understanding not only the maximum error, but also how errors are distributed across different input magnitudes.

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T11:56:11Z

Timo.stripf: /* Numerical Issues and Error Metrics */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

==Numerical Issues and Error Metrics==

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T11:53:38Z

Timo.stripf:

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.

==Direct Exponential Formulations==

==Numerical Issues and Error Metrics==

{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T11:51:20Z

Timo.stripf: /* Problem Overview */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
The hyperbolic tangent function poses several challenges when implemented in finite-precision floating-point arithmetic. While the mathematical definition is simple, direct translations into software often suffer from numerical instability and performance issues.

A primary challenge is the use of exponential functions. For large input values, <code>exp(x)</code> and <code>exp(-x)</code> can overflow in single-precision floating-point arithmetic, leading to infinities and, in some formulations, NaN results. Avoiding overflow therefore requires careful reformulation or explicit range handling.

For very small input values, catastrophic cancellation becomes the dominant issue. Expressions such as <math>e^x - e^{-x}</math> involve the subtraction of nearly equal numbers, causing a severe loss of significant bits and large ULP errors. In these regions, naive implementations may return zero or highly inaccurate results even though <math>\tanh(x) \approx x</math>.

In addition to accuracy concerns, performance is a critical factor. In many applications, especially machine learning and numerical simulations, <code>tanh</code> is evaluated extremely frequently. This makes it necessary to balance numerical robustness with the cost of transcendental function calls, branch complexity, and instruction-level efficiency.
==Direct Exponential Formulations==
==Numerical Issues and Error Metrics==
== Simple Approximation with Exponential Math Function ==
{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T11:49:38Z

Timo.stripf: /* Simple Approximation with Exponential Math Function */

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Problem Overview ==
==Direct Exponential Formulations==
==Numerical Issues and Error Metrics==
== Simple Approximation with Exponential Math Function ==
{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-04T11:45:10Z

Timo.stripf:

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as <code>tanh</code>, is a mathematical function that maps real numbers to the range <math>(-1, 1)</math>. It is defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

From a software implementation perspective, <code>tanh</code> is a numerically challenging function due to the use of exponential functions, which can easily overflow for large inputs and suffer from catastrophic cancellation for very small values of <math>x</math>. These issues are particularly relevant in single-precision floating-point (<code>float32</code>) implementations, where both accuracy and performance are critical.

In machine learning and deep learning, <code>tanh</code> is widely used as an activation function because it introduces non-linearity and produces outputs centered around zero, which can improve gradient-based optimization. As a result, <code>tanh</code> is often evaluated billions of times during training and inference, making efficient and numerically stable implementations essential.

This article focuses on practical software implementations of <code>tanh</code>, analyzing different computational formulations with respect to floating-point accuracy, ULP error, overflow behavior, and NaN generation. The goal is to provide a foundation for designing robust <code>tanh</code> implementations suitable for performance-critical environments such as numerical libraries, embedded systems, and AI workloads.

== Simple Approximation with Exponential Math Function ==
{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

Main Page

2026-02-04T11:42:20Z

Timo.stripf: /* Other Topics */

__NOTOC__

Welcome to the emmtrix Technologies Wiki. As a company with a deep-rooted passion for compilers, we specialize in source-to-source compilers designed to analyze, optimize and transform your code. This Wiki aims to offer detailed, technical background information that complements the tools and resources available on our official website. Here, you'll find in-depth explanations, usage guidelines, and insights into the engineering behind our specialized software solutions. Whether you're a developer or a technically-inclined enthusiast, this space is designed to deepen your understanding of what makes our tools essential for your projects.

{| border="0" style="padding: 10px"
| style="vertical-align: top; padding: 10px;" |
=== emmtrix Products ===


* [[emmtrix Dependency Analyzer]]
* [[emmtrix Performance Estimator]]
* [[emmtrix Code Vectorizer]]
** [[emmtrix ONNX-to-C Code Generator]]
* [[emmtrix C++ to C Compiler]]
** [https://online-ecpp2c.emmtrix.com emmtrix C++ to C Compiler Online]
* [[emmtrix C to Rust Compiler]]
* emmtrix Parallel Studio
* Generic information
** [[emmtrix Studio Release Notes]]
** [[:Category:emmtrix Studio FAQ|FAQ]]
* Discontinued product
** [[emmtrix Code Generator]]
** [[emmtrix Model Code Generator]]
* Other Solutions
** [[TC4x PPU Coverage Analysis]]
** [[emmtrix Link Stubber]]
| style="vertical-align: top; padding: 10px;" |
=== Supported Architectures ===
* [[TriCore Instruction Set Architecture]]
** [[Infineon AURIX TC2xx]]
** [[Infineon AURIX TC3xx]]
** [[Infineon AURIX TC4x]], including [[Infineon AURIX TC4x Parallel Processing Unit (PPU)|Parallel Processing Unit (PPU)]]
* ... and many more
|-
|
|
|}

=== Compiler Background ===

* [[Loop Transformations]]
* [[:Category:Code Transformation]]
*[[Demystifying C++]]
*[[The alias Attribute|The alias attribute]]
*[[Clang Diagnostics Overview]]
*[[:Category:Clang Diagnostics]]

=== Other Topics ===
* [[C to Z3 Cheat Sheet]]
* [[Eclipse Xcore Cheat Sheet]]
* [[Logical Execution Time (LET)]]
* [[ULP Difference of Float Numbers]]
* [[Numerical Precision in ONNX and AI Inference]]
* [[Automatic C to Rust Translation]]
* [[tanh Software Implementation]]

emmtrix ONNX-to-C Code Generator

2026-02-04T07:08:03Z

Timo.stripf:

'''emmtrix ONNX-to-C Code Generator''' (emx-onnx-cgen) is an emmtrix-developed, open-source AI frontend compiler that translates ONNX models into deterministic, analyzable C code specifically designed for auto-vectorization and embedded target optimization.

The primary goal of emx-onnx-cgen is not to perform any hardware-specific optimizations itself, but to generate high-quality C code that serves as an ideal input for the [[emmtrix Code Vectorizer|emmtrix Vectorizer]] and subsequent backend toolchains targeting embedded architectures.

emx-onnx-cgen is particularly suited for safety-critical, resource-constrained, and bare-metal environments where transparency, determinism, and static analyzability are mandatory.
==Motivation==
Typical AI deployment solutions rely on runtime engines or architecture-specific code generators that:
*hide control flow and memory access patterns,
*rely on dynamic memory allocation,
*limit static analysis and certification,
*restrict the effectiveness of auto-vectorization.
emx-onnx-cgen follows a different approach:

'''ONNX → Clean C → Vectorizer → Target Architecture'''

By lowering AI models into a well-structured and explicit C representation, emx-onnx-cgen enables advanced compiler analyses and vectorization passes that would otherwise be impossible or unreliable.
==Role in the emmtrix Toolchain==
emx-onnx-cgen acts as the '''frontend''' in the emmtrix AI compilation pipeline.
;Pipeline Overview
*'''ONNX Model'''Imported neural network description

*'''emx-onnx-cgen'''Normalization, lowering, and generation of vectorization-friendly C code

*'''emmtrix Vectorizer'''Loop analysis, data-flow analysis, auto-vectorization

*'''Backend / Toolchain'''Architecture-specific code generation (DSP, MCU, SoC)
'''Diagram suggestion:'''<pre>
ONNX Model
|
v
emx-onnx-cgen
|
v
Vectorization-Friendly C Code
|
v
emmtrix Vectorizer
|
v
Optimized Embedded Binary
</pre>
==Design Goals==
emx-onnx-cgen is designed with the following goals:
*Deterministic code generation
*Fully static memory layout (no heap usage)
*Explicit loops and control flow
*Predictable memory access patterns
*Readable and auditable C code
*Maximal compatibility with auto-vectorization tools
*Suitability for embedded and safety-critical systems
==Non-Goals==
emx-onnx-cgen explicitly does '''not''' aim to:
*Act as a runtime inference engine
*Perform hand-written SIMD or target-specific optimizations
*Hide model execution behind opaque APIs
*Support training or backpropagation
==Code Generation Principles==
The generated C code follows strict structural rules to support static analysis and vectorization:
*Simple, canonical loop forms
*Linear array accesses
*No hidden pointer aliasing
*No dynamic dispatch
*No recursion
*No dynamic memory allocation
*Explicit tensor dimensions and strides
These properties allow the emmtrix Vectorizer to reliably detect vectorization opportunities and apply architecture-specific optimizations.
==Example==
===High-Level Operation===
An ONNX operator such as a vector addition is lowered into a simple C loop:<pre>
for (i = 0; i < N; ++i) {
output[i] = input_a[i] + input_b[i];
}
</pre>This structure is intentionally chosen to be:
*easy to analyze,
*free of side effects,
*ideal for SIMD vectorization by downstream tools.
==Embedded Target Suitability==
emx-onnx-cgen is suitable for:
*Bare-metal systems
*RTOS-based systems
*Automotive ECUs
*Industrial controllers
*DSP-based platforms
The generated code depends only on standard C headers and can be integrated into existing embedded build systems without requiring a runtime framework.
==Verification and Determinism==
To ensure correctness, emx-onnx-cgen supports verification against reference ONNX execution.

Key properties:
*Bit-stable code generation
*Reproducible builds
*Deterministic execution behavior
These properties are essential for certification, validation, and long-term maintenance of embedded AI software.
==Related Pages==
*[[emmtrix Code Vectorizer]]
*[[Numerical Precision in ONNX and AI Inference]]
==See Also==
*emx-onnx-cgen GitHub Repository - https://github.com/emmtrix/emx-onnx-cgen/
[[Category:emmtrix Tools]]

emmtrix Performance Estimator

2026-02-04T02:21:41Z

Timo.stripf: /* Estimation Methods */

The [https://www.emmtrix.com/tools/emmtrix-performance-estimator '''emmtrix Performance Estimator (ePE)'''] is a static analysis tool for estimating the execution time of embedded C code. It determines timing characteristics of a software system without executing the code on the target hardware.

Unlike measurement-based approaches, which rely on instrumented code, simulators, or real hardware, ePE performs a purely static analysis. The estimation is based on the program structure, control flow, and target-specific execution models. As a result, performance characteristics can be assessed early in the development process, even when the final hardware platform is not yet available.

The primary goal of ePE is to provide reproducible and analyzable execution time estimates, including minimum, maximum, and average values. These estimates can be evaluated on different levels of granularity, ranging from individual functions up to complete applications.

ePE is designed for embedded software development where timing behavior is a critical system property. Typical application domains include real-time and safety-critical systems, such as automotive ECUs and industrial control units. In particular, ePE is commonly used in projects targeting Infineon microcontroller families, including the '''Infineon's AURIX™ [[Infineon AURIX TC2xx|TC2xx]] / [[Infineon AURIX TC3xx|TC3xx]]/ [[Infineon AURIX TC4x|TC4x]] microcontroller family'''.

The tool supports different estimation techniques with varying levels of accuracy, ranging from source-code-based analysis to assembly-level estimation with microarchitectural modeling. For supported Infineon architectures, this includes processor-specific timing models that reflect relevant architectural features.

In addition, ePE integrates into automated build and verification workflows and can be combined with model-based development environments. This enables continuous performance assessment and early detection of timing regressions during software evolution.
==Problem Statement and Motivation==
In embedded and real-time software systems, execution time is a critical non-functional property. Tasks and functions must meet strict timing constraints in order to guarantee system correctness, schedulability, and safety. Violations of these constraints may result in missed deadlines, degraded system behavior, or safety hazards.

[[File:DefectDetectionandRemediation.webp|524x524px|Cost of defect remediation across the software development life cycle|alt=Cost of defect remediation across the software development life cycle|thumb]]

Traditional performance evaluation techniques are predominantly measurement-based. They rely on executing instrumented software on simulators, hardware-in-the-loop (HIL) setups, or real target hardware. While such approaches can provide accurate measurements, they also introduce several practical limitations:
*Target hardware is often unavailable or not yet representative in early development phases.
*Measurements are influenced by instrumentation overhead and runtime environment effects.
* Results may vary across toolchains, compiler versions, or hardware revisions.
*Performance issues are typically detected late in the software development life cycle.
Early development phases, however, require performance feedback at a time when software architecture, algorithms, and scheduling concepts are still subject to change. At this stage, late detection of execution time problems can lead to costly redesigns, as architectural modifications become increasingly difficult once system integration has progressed.

This general principle is well known from software quality engineering. Defects are typically introduced early during development, but often detected much later, while the effort required to correct them increases significantly over time. Although originally formulated for functional defects, this observation applies analogously to timing- and performance-related issues in embedded systems. Late detection of timing violations frequently requires invasive changes, such as restructuring software components, modifying scheduling strategies, or adapting hardware configurations. In contrast, early performance estimation enables developers to assess timing risks at a point where corrective actions are comparatively inexpensive.

Static execution time estimation addresses this challenge by analyzing software without executing it. By deriving execution frequencies from the program structure and combining them with target-specific execution models, performance characteristics can be estimated deterministically and reproducibly. This allows systematic performance assessment early in the development process and supports continuous evaluation as the software evolves.

The emmtrix Performance Estimator (ePE) is designed to support this approach. It provides a static, analyzable, and reproducible method for estimating execution time, complementing measurement-based techniques used in later development stages and enabling early detection of timing-related risks.

==Fundamental Estimation Model==
The emmtrix Performance Estimator (ePE) is based on a static execution time model that decomposes the overall runtime of a software system into two orthogonal components:
*the '''execution frequency''' of a code block, and
*the '''execution duration''' of a single execution of that block.
For a given code block <math>b</math> (e.g. a basic block, function, or task), the estimated execution time <math>t_{exec}(b)</math> is defined as:

<big><math>
t_{exec}(b) = f(b) \cdot d(b)
</math></big>

where:
*<math>f(b)</math> denotes the execution frequency of block <math>b</math>, i.e. how often the block is executed during one invocation of the surrounding context (task, function, or application),
* <math>d(b)</math> denotes the estimated execution duration of a single execution of block <math>b</math>.
The overall execution time of a larger software entity is obtained by aggregating the execution times of its constituent blocks.
===Execution Frequency===
The execution frequency <math>f(b)</math> is derived statically from the program structure and control flow. ePE analyzes the control flow graph (CFG) of the application and determines execution frequencies based on:
*loop bounds and iteration counts,
*conditional branches,
*call relationships between functions,
*structural properties of the control flow.
Whenever possible, loop bounds and conditions are evaluated using constant folding and static evaluation. This enables deterministic frequency estimates without executing the code. For blocks that are executed conditionally, frequency ranges or bounds may be derived depending on the available information.

Execution frequencies form the structural part of the estimation model and are independent of the target architecture.
===Execution Duration ===
The execution duration <math>d(b)</math> represents the time required to execute block <math>b</math> once on a given target architecture. ePE determines this duration using target-specific execution models.

At the most basic level, the duration is computed by mapping program instructions to abstract timing costs (e.g. processor cycles) and summing these costs over the block. More advanced estimation methods additionally consider:
*compiler optimizations,
*instruction scheduling,
*pipeline effects and microarchitectural behavior.
The level of detail used for duration estimation directly influences the accuracy of the result and depends on the selected estimation technique.
===Aggregation and Granularity===
The estimation model is compositional. Execution time estimates can be aggregated across different levels of granularity, including:
*individual basic blocks,
*functions,
*tasks or runnables,
*complete applications.
This enables performance analysis and comparison at multiple abstraction levels, supporting both detailed investigations and high-level assessments.
===Determinism and Reproducibility===
Since both execution frequencies and execution durations are derived statically, the estimation results produced by ePE are deterministic and reproducible. Given identical source code, configuration, and target model, repeated analyses yield identical results. This property is essential for systematic performance evaluation, regression analysis, and integration into automated build and verification workflows.
=== Estimation Methods ===
In general, the execution time of a task or block can be modelled as:

''texec = execution_frequencyblock* single_durationblock''

The C code based static code analysis derives the execution frequency by analyzing loop boundaries using constant folding. For the duration, each instruction in the code is modeled as the number of cycles of an abstract hardware model of the processor, and then summed.

ePE offers three accuracy levels:
*analysis of C code
*generically compiler-optimized code
*assembly code from the target compiler.
Method 1 yields results with minimum effort while method 3 takes the timing of the processor pipeline into account. All methods offer excellent reliability when tracking the tendency of changes in software runtimes e.g. when used in a continuous integration environment.

==== Static Code Analysis: ====
[[File:Static Code Analysis.jpg|frameless|800x800px]]

==== Processor Pipeline: ====
[[File:ePE Processor Pipeline.png|frameless|950x950px]]

[https://www.emmtrix.com/tools/emmtrix-performance-estimator emmtrix Performance Estimator (ePE)] supports different ways to acquire the duration of the tasks of an application. These methods vary in accuracy and additional software or hardware requirements. Static code analysis provides basic information without the need for hardware or special software and may thus be seamlessly incorporated into early stages of the Software Development Life Cycle (SDLC). More accurate numbers can be collected with interfaces to simulators or the hardware. Depending on the requirements, the methods can be combined as desired.

Addressing performance issues at the onset of the SDLC is not only proactive but also cost-efficient. Early estimation and intervention, as suggested by the below graph, can drastically reduce the complexities and expenses associated with fixing defects post-deployment. Although early estimates may not be as precise as measurements taken during later stages, they serve as a crucial checkpoint to prevent potential timing issues from escalating. This preemptive approach aligns with the principle that the cost to repair defects increases exponentially as they progress through the SDLC, as shown by the sharp rise in cost depicted in the graph. Hence, initial performance estimations are a strategic investment, minimizing the risk of facing a steep climb in remediation costs and efforts at later stages.

== Interested? ==
{{CallToAction|text=Interested in applying this coverage workflow to your own projects?}}
[[Category:emmtrix Tools]]

emmtrix Performance Estimator

2026-02-04T02:03:52Z

Timo.stripf: /* Problem Statement and Motivation */

emmtrix Performance Estimator

2026-02-04T02:00:52Z

Timo.stripf: /* Estimation Methods */

The [https://www.emmtrix.com/tools/emmtrix-performance-estimator '''emmtrix Performance Estimator (ePE)'''] is a static analysis tool for estimating the execution time of embedded C code. It determines timing characteristics of a software system without executing the code on the target hardware.

Unlike measurement-based approaches, which rely on instrumented code, simulators, or real hardware, ePE performs a purely static analysis. The estimation is based on the program structure, control flow, and target-specific execution models. As a result, performance characteristics can be assessed early in the development process, even when the final hardware platform is not yet available.

The primary goal of ePE is to provide reproducible and analyzable execution time estimates, including minimum, maximum, and average values. These estimates can be evaluated on different levels of granularity, ranging from individual functions up to complete applications.

ePE is designed for embedded software development where timing behavior is a critical system property. Typical application domains include real-time and safety-critical systems, such as automotive ECUs and industrial control units. In particular, ePE is commonly used in projects targeting Infineon microcontroller families, including the '''Infineon's AURIX™ [[Infineon AURIX TC2xx|TC2xx]] / [[Infineon AURIX TC3xx|TC3xx]]/ [[Infineon AURIX TC4x|TC4x]] microcontroller family'''.

The tool supports different estimation techniques with varying levels of accuracy, ranging from source-code-based analysis to assembly-level estimation with microarchitectural modeling. For supported Infineon architectures, this includes processor-specific timing models that reflect relevant architectural features.

In addition, ePE integrates into automated build and verification workflows and can be combined with model-based development environments. This enables continuous performance assessment and early detection of timing regressions during software evolution.
==Problem Statement and Motivation==
In embedded and real-time software systems, execution time is a critical non-functional property. Tasks and functions must meet strict timing constraints in order to guarantee system correctness, schedulability, and safety. Violations of these constraints may result in missed deadlines, degraded system behavior, or safety hazards.

Traditional performance evaluation techniques are predominantly measurement-based. They rely on executing instrumented software on simulators, hardware-in-the-loop (HIL) setups, or real target hardware. While such approaches can provide accurate measurements, they also introduce several practical limitations:
*Target hardware is often unavailable or not yet representative in early development phases.
*Measurements are influenced by instrumentation overhead and runtime environment effects.
* Results may vary across toolchains, compiler versions, or hardware revisions.
*Performance issues are typically detected late in the software development life cycle.
Early development phases, however, require performance feedback at a time when software architecture, algorithms, and scheduling concepts are still subject to change. At this stage, late detection of execution time problems can lead to costly redesigns, as architectural modifications become increasingly difficult once system integration has progressed.

This general principle is well known from software quality engineering. Defects are typically introduced early during development, but often detected much later, while the effort required to correct them increases significantly over time. Although originally formulated for functional defects, this observation applies analogously to timing- and performance-related issues in embedded systems.[[File:DefectDetectionandRemediation.webp|524x524px|Cost of defect remediation across the software development life cycle|alt=Cost of defect remediation across the software development life cycle|thumb]]Late detection of timing violations frequently requires invasive changes, such as restructuring software components, modifying scheduling strategies, or adapting hardware configurations. In contrast, early performance estimation enables developers to assess timing risks at a point where corrective actions are comparatively inexpensive.

Static execution time estimation addresses this challenge by analyzing software without executing it. By deriving execution frequencies from the program structure and combining them with target-specific execution models, performance characteristics can be estimated deterministically and reproducibly. This allows systematic performance assessment early in the development process and supports continuous evaluation as the software evolves.

The emmtrix Performance Estimator (ePE) is designed to support this approach. It provides a static, analyzable, and reproducible method for estimating execution time, complementing measurement-based techniques used in later development stages and enabling early detection of timing-related risks.
=== Estimation Methods ===
In general, the execution time of a task or block can be modelled as:

''texec = execution_frequencyblock* single_durationblock''

The C code based static code analysis derives the execution frequency by analyzing loop boundaries using constant folding. For the duration, each instruction in the code is modeled as the number of cycles of an abstract hardware model of the processor, and then summed.

ePE offers three accuracy levels:
*analysis of C code
*generically compiler-optimized code
*assembly code from the target compiler.
Method 1 yields results with minimum effort while method 3 takes the timing of the processor pipeline into account. All methods offer excellent reliability when tracking the tendency of changes in software runtimes e.g. when used in a continuous integration environment.

==== Static Code Analysis: ====
[[File:Static Code Analysis.jpg|frameless|800x800px]]

==== Processor Pipeline: ====
[[File:ePE Processor Pipeline.png|frameless|950x950px]]

[https://www.emmtrix.com/tools/emmtrix-performance-estimator emmtrix Performance Estimator (ePE)] supports different ways to acquire the duration of the tasks of an application. These methods vary in accuracy and additional software or hardware requirements. Static code analysis provides basic information without the need for hardware or special software and may thus be seamlessly incorporated into early stages of the Software Development Life Cycle (SDLC). More accurate numbers can be collected with interfaces to simulators or the hardware. Depending on the requirements, the methods can be combined as desired.

Addressing performance issues at the onset of the SDLC is not only proactive but also cost-efficient. Early estimation and intervention, as suggested by the below graph, can drastically reduce the complexities and expenses associated with fixing defects post-deployment. Although early estimates may not be as precise as measurements taken during later stages, they serve as a crucial checkpoint to prevent potential timing issues from escalating. This preemptive approach aligns with the principle that the cost to repair defects increases exponentially as they progress through the SDLC, as shown by the sharp rise in cost depicted in the graph. Hence, initial performance estimations are a strategic investment, minimizing the risk of facing a steep climb in remediation costs and efforts at later stages.
[[File:DefectDetectionandRemediation.webp|center|frameless|800x800px|Cost of Defect Remediation across the Software Development Life Cycle]]

== Interested? ==
{{CallToAction|text=Interested in applying this coverage workflow to your own projects?}}
[[Category:emmtrix Tools]]

emmtrix Performance Estimator

2026-02-04T01:50:34Z

Timo.stripf:

emmtrix ONNX-to-C Code Generator

2026-02-04T01:19:08Z

Timo.stripf:

'''emmtrix ONNX-to-C Code Generator''' (emx-onnx-cgen) is an emmtrix-developed, open-source AI frontend compiler that translates ONNX models into deterministic, analyzable C code specifically designed for auto-vectorization and embedded target optimization.

The primary goal of emx-onnx-cgen is not to perform aggressive hardware-specific optimizations itself, but to generate high-quality C code that serves as an ideal input for the emmtrix Vectorizer and subsequent backend toolchains targeting embedded architectures.

emx-onnx-cgen is particularly suited for safety-critical, resource-constrained, and bare-metal environments where transparency, determinism, and static analyzability are mandatory.
==Motivation==
Typical AI deployment solutions rely on runtime engines or architecture-specific code generators that:
*hide control flow and memory access patterns,
*rely on dynamic memory allocation,
*limit static analysis and certification,
*restrict the effectiveness of auto-vectorization.
emx-onnx-cgen follows a different approach:

'''ONNX → Clean C → Vectorizer → Target Architecture'''

By lowering AI models into a well-structured and explicit C representation, emx-onnx-cgen enables advanced compiler analyses and vectorization passes that would otherwise be impossible or unreliable.
==Role in the emmtrix Toolchain==
emx-onnx-cgen acts as the '''frontend''' in the emmtrix AI compilation pipeline.
;Pipeline Overview
*'''ONNX Model'''Imported neural network description

*'''emx-onnx-cgen'''Normalization, lowering, and generation of vectorization-friendly C code

*'''emmtrix Vectorizer'''Loop analysis, data-flow analysis, auto-vectorization

*'''Backend / Toolchain'''Architecture-specific code generation (DSP, MCU, SoC)
'''Diagram suggestion:'''<pre>
ONNX Model
|
v
emx-onnx-cgen
|
v
Vectorization-Friendly C Code
|
v
emmtrix Vectorizer
|
v
Optimized Embedded Binary
</pre>
==Design Goals==
emx-onnx-cgen is designed with the following goals:
*Deterministic code generation
*Fully static memory layout (no heap usage)
*Explicit loops and control flow
*Predictable memory access patterns
*Readable and auditable C code
*Maximal compatibility with auto-vectorization tools
*Suitability for embedded and safety-critical systems
==Non-Goals==
emx-onnx-cgen explicitly does '''not''' aim to:
*Act as a runtime inference engine
*Perform hand-written SIMD or target-specific optimizations
*Hide model execution behind opaque APIs
*Support training or backpropagation
==Code Generation Principles==
The generated C code follows strict structural rules to support static analysis and vectorization:
*Simple, canonical loop forms
*Linear array accesses
*No hidden pointer aliasing
*No dynamic dispatch
*No recursion
*No dynamic memory allocation
*Explicit tensor dimensions and strides
These properties allow the emmtrix Vectorizer to reliably detect vectorization opportunities and apply architecture-specific optimizations.
==Example==
===High-Level Operation===
An ONNX operator such as a vector addition is lowered into a simple C loop:<pre>
for (i = 0; i < N; ++i) {
output[i] = input_a[i] + input_b[i];
}
</pre>This structure is intentionally chosen to be:
*easy to analyze,
*free of side effects,
*ideal for SIMD vectorization by downstream tools.
==Embedded Target Suitability==
emx-onnx-cgen is suitable for:
*Bare-metal systems
*RTOS-based systems
*Automotive ECUs
*Industrial controllers
*DSP-based platforms
The generated code depends only on standard C headers and can be integrated into existing embedded build systems without requiring a runtime framework.
==Verification and Determinism==
To ensure correctness, emx-onnx-cgen supports verification against reference ONNX execution.

Key properties:
*Bit-stable code generation
*Reproducible builds
*Deterministic execution behavior
These properties are essential for certification, validation, and long-term maintenance of embedded AI software.
==Related Pages==
*[[emmtrix Code Vectorizer]]
*[[Numerical Precision in ONNX and AI Inference]]
==See Also==
*emx-onnx-cgen GitHub Repository - https://github.com/emmtrix/emx-onnx-cgen/
[[Category:emmtrix Tools]]

emmtrix ONNX-to-C Code Generator

2026-02-04T01:17:50Z

Timo.stripf: Created page with "'''emmtrix ONNX-to-C Code Generator (emx-onnx-cgen)''' is an emmtrix-developed '''AI frontend compiler''' that translates ONNX models into '''deterministic, analyzable C code''' specifically designed for '''auto-vectorization and embedded target optimization'''. The primary goal of emx-onnx-cgen is not to perform aggressive hardware-specific optimizations itself, but to generate '''high-quality C code''' that serves as an ideal input for the emmtrix '''Vectorizer''' and..."

'''emmtrix ONNX-to-C Code Generator (emx-onnx-cgen)''' is an emmtrix-developed '''AI frontend compiler''' that translates ONNX models into '''deterministic, analyzable C code''' specifically designed for '''auto-vectorization and embedded target optimization'''.

The primary goal of emx-onnx-cgen is not to perform aggressive hardware-specific optimizations itself, but to generate '''high-quality C code''' that serves as an ideal input for the emmtrix '''Vectorizer''' and subsequent backend toolchains targeting embedded architectures.

emx-onnx-cgen is particularly suited for safety-critical, resource-constrained, and bare-metal environments where transparency, determinism, and static analyzability are mandatory.
==Motivation==
Typical AI deployment solutions rely on runtime engines or architecture-specific code generators that:
*hide control flow and memory access patterns,
*rely on dynamic memory allocation,
*limit static analysis and certification,
*restrict the effectiveness of auto-vectorization.
emx-onnx-cgen follows a different approach:

'''ONNX → Clean C → Vectorizer → Target Architecture'''

By lowering AI models into a well-structured and explicit C representation, emx-onnx-cgen enables advanced compiler analyses and vectorization passes that would otherwise be impossible or unreliable.
==Role in the emmtrix Toolchain==
emx-onnx-cgen acts as the '''frontend''' in the emmtrix AI compilation pipeline.
;Pipeline Overview
*'''ONNX Model'''Imported neural network description

*'''emx-onnx-cgen'''Normalization, lowering, and generation of vectorization-friendly C code

*'''emmtrix Vectorizer'''Loop analysis, data-flow analysis, auto-vectorization

*'''Backend / Toolchain'''Architecture-specific code generation (DSP, MCU, SoC)
'''Diagram suggestion:'''<pre>
ONNX Model
|
v
emx-onnx-cgen
|
v
Vectorization-Friendly C Code
|
v
emmtrix Vectorizer
|
v
Optimized Embedded Binary
</pre>
==Design Goals==
emx-onnx-cgen is designed with the following goals:
*Deterministic code generation
*Fully static memory layout (no heap usage)
*Explicit loops and control flow
*Predictable memory access patterns
*Readable and auditable C code
*Maximal compatibility with auto-vectorization tools
*Suitability for embedded and safety-critical systems
==Non-Goals==
emx-onnx-cgen explicitly does '''not''' aim to:
*Act as a runtime inference engine
*Perform hand-written SIMD or target-specific optimizations
*Hide model execution behind opaque APIs
*Support training or backpropagation
==Code Generation Principles==
The generated C code follows strict structural rules to support static analysis and vectorization:
*Simple, canonical loop forms
*Linear array accesses
*No hidden pointer aliasing
*No dynamic dispatch
*No recursion
*No dynamic memory allocation
*Explicit tensor dimensions and strides
These properties allow the emmtrix Vectorizer to reliably detect vectorization opportunities and apply architecture-specific optimizations.
==Example==
===High-Level Operation===
An ONNX operator such as a vector addition is lowered into a simple C loop:<pre>
for (i = 0; i < N; ++i) {
output[i] = input_a[i] + input_b[i];
}
</pre>This structure is intentionally chosen to be:
*easy to analyze,
*free of side effects,
*ideal for SIMD vectorization by downstream tools.
==Embedded Target Suitability==
emx-onnx-cgen is suitable for:
*Bare-metal systems
*RTOS-based systems
*Automotive ECUs
*Industrial controllers
*DSP-based platforms
The generated code depends only on standard C headers and can be integrated into existing embedded build systems without requiring a runtime framework.
==Verification and Determinism==
To ensure correctness, emx-onnx-cgen supports verification against reference ONNX execution.

Key properties:
*Bit-stable code generation
*Reproducible builds
*Deterministic execution behavior
These properties are essential for certification, validation, and long-term maintenance of embedded AI software.
==Related Pages==
*[[emmtrix Code Vectorizer]]
*[[Numerical Precision in ONNX and AI Inference]]
==See Also==
*emx-onnx-cgen GitHub Repository - https://github.com/emmtrix/emx-onnx-cgen/
[[Category:emmtrix Tools]]

Main Page

2026-02-04T01:12:29Z

Timo.stripf:

emmtrix Code Vectorizer

2026-02-03T09:51:27Z

Timo.stripf:

emmtrix Code Vectorizer is a C source-to-source compiler that vectorizes C code for various SIMD architectures like [[Infineon AURIX TC4x|Infineon AURIX TC4x PPU]], x86 AVX, ARM NEON, or RISC-V V extensions. It follows an semi-automatic approach, where the user can guide the vectorization process from an Eclipse-based GUI or by using pragmas in the C code. The vectorized C code is always visible to the user, which makes it easy to follow the transformation and to understand the generated code. The generated code can be compiled with the vendor’s compiler and either run on the target hardware or in a simulator to get performance feedback and verify the correctness of the transformation.

=== Multi-Precision Vector Math Library ===
The emmtrix Code Vectorizer incorporates a multi-precision vector math library that provides vectorized implementations of common C mathematical functions like sinf, cosf, expf, logf, powf, sqrtf, tanhf and many more. The library provides multiple versions of each function to provide the best trade-off between accuracy and performance. During vectorization, the vector math library is used to replace scalar math functions with vectorized versions. By specifying the accuracy requirements in [[ULP Difference of Float Numbers|ULPs (units in the last place)]], the user can control which version of the function is used.
{| class="wikitable"
|-
! Function
!Info
! Accuracy<ref>Accuracy of hardware version are given from the data sheet. A ULP 0.0 means that the implementation is exact. Otherwise, the highest know ULP is provided. Math functions that are based on non-perfect hardware implementation use a pessimistic software implementation for ULP evaluation. A more accurate ULP evaluation could be provided on request.</ref> [ULP]
! Latency on AURIX™ TC4x [cycles]
! Throughput on AURIX™ TC4x [cycles]
|-
|fabsf
|
|0.0
|
|
|-
|fminf
|
|0.0
|
|
|-
|fmaxf
|
|0.0
|
|
|-
|roundf
|
|0.0
|11
|5
|-
|ceilf
|
|0.0
|6
|4
|-
|floorf
|
|0.0
|6
|4
|-
|truncf
|
|0.0
|6
|4
|-
| sqrtf
|Hardware version
| 0.5
|16
|7
|-
| expf
|Hardware version
| 1.0
|11
|6
|-
|exp2f
|Hardware version
|1.0
|14
|6
|-
| log2f
|Hardware version
| 1.0
|14
|6
|-
|logf
|Based on log2f ULP 1.0 hardware version
|1.93
|
|
|-
|log10f
|Based on log2f ULP 1.0 hardware version
|2.50
|
|
|-
| sinf
|Hardware version
| 1.0
|14
|6
|-
| cosf
|Hardware version
| 1.0
|14
|6
|-
|tanf
|Based on sinf/cosf ULP 1.0 hardware version
|3.42
|
|
|-
| rowspan="4" | tanhf
| rowspan="4" |Based on expf ULP 1.0 hardware version
|7.08
|41
|18
|-
|4.23
|41
|19
|-
|2.74
|41
|23
|-
|1.91
|41
|27
|-
|hypotf
|Based on sqrt ULP 0.5 hardware vesrion
|1.995
|
|
|-
|remainderf
|
|
|
|
|-
|fmodf
|
|
|
|
|-
|fdimf
|
|
|
|
|}

[[Category:emmtrix Tools]]
<references />

== Interested? ==
{{CallToAction|text=Interested in applying this coverage workflow to your own projects?}}
[[Category:Numerical Precision]]

tanh Software Implementation

2026-02-03T09:51:12Z

Timo.stripf:

[[File:Hyperbolic Tangent.svg|thumb]]
The hyperbolic tangent function, commonly referred to as tanh, is a mathematical function that maps real numbers to the range (−1,1). Defined as:

:<math>\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}</math>

it is a smooth, continuous, and symmetric function centered at the origin. The tanh function is widely used in mathematics, physics, and engineering, particularly in areas involving hyperbolic geometry and signal processing.

In machine learning and deep learning, tanh is a popular activation function because it introduces non-linearity and maps inputs to a centered range around zero, helping to balance gradients during optimization. Its output approaches 1 for large positive inputs and −1 for large negative inputs, with steep transitions around x=0.

== Simple Approximation with Exponential Math Function ==
{| class="wikitable"
|+
!Id
!Formula
!Highest ULP Error
!ULP <= 2.0
!NaN
|-
|exp_v1
|<math>\frac{e^x - e^{-x}}{e^x + e^{-x}}</math>
|x < 0: 8.55638e+08 at -2.98023e-08 with 0x0p+0 instead of -0x1p-25

x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1p-25
| -88.7246 < x < -7.14197
7.14197 < x < 88.7246
| -inf < x < -88.7246
88.7246 < x < inf
|-
|exp_v2
|<math>\frac{e^{2x} - 1}{e^{2x} + 1}</math>
|x < 0: 8.47249e+08 at -1.49012e-08 with 0x0p+0 instead of -0x1p-26
x > 0: 8.55638e+08 at 2.98023e-08 with 0x0p+0 instead of 0x1.fffffep-26
| -inf < x < -0.547821
0.254822 < x < 8.31787

8.38379 < x < 44.3623
|44.3623 < x < inf
|-
|exp_v3
|<math>1 - \frac{2}{e^{2x} + 1}</math>
|x < 0: 8.59832e+08 at -4.47035e-08 with 0x0p+0 instead of -0x1.8p-25
x > 0: 8.68221e+08 at 8.9407e-08 with 0x0p+0 instead of 0x1.7ffffcp-24
| -inf < x < -6.26233
0.346382 < x < inf
| -
|-
|expm1
|<math>\frac{expm1(2x)}{expm1(2x) + 2}</math>
|x < 0: 2.49585 at -3.95627 with -0x1.ffa00ap-1 instead of -0x1.ffa00ep-1
x > 0: 2.41657 at 0.0155837 with 0x1.fe9b66p-7 instead of 0x1.fe9b62p-7
| -inf < x < -5.57227
-0.42981 < x < 0.00777948

0.125351 < x < 44.3623
|44.3623 < x < inf
|}

The first equation is the definition of the hyperbolic tangent function. This equation requires two exponentials with different values. Since exponentials are expensive to compute, we can use the second equation to reduce the number of exponentials to one.

The second equation has the drawback that both the numerator and the denominator can become infinity for large values of x which results in NaN. That happens for <math>|x| > \log(FLT\_MAX)/2 \approx 44</math>.

The third equation solves this problem but has a slightly higher error for <math>x \approx 2^{-25}</math>. In the following graph you can see the error chart for equation 3. The error charts for the other two equations are very similar. All three equations have an error of up to 16 mio ULPs for <math>x < 2^{-5}</math> which means that up to 24 bits could be wrong.

=== exp_v2 ===

{{#widget:FloatErrorGraph|id=tanhf_exp_v2
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",5.95888e-08,0.999734],["-2^2",8.86696e-08,1.48763],["-2^1",8.91609e-08,1.49587],["-2^0",8.73743e-08,1.4659],["-2^-1",8.65382e-08,2.13222],["-2^-2",6.18481e-08,2.61428],["-2^-3",3.88071e-08,3.08594],["-2^-4",2.6234e-08,4.23594],["-2^-5",2.02799e-08,8.19726],["-2^-6",1.75183e-08,12.4495],["-2^-7",1.61937e-08,17.3878],["-2^-8",1.5566e-08,33.4278],["-2^-9",1.51688e-08,65.1495],["-2^-10",1.50281e-08,129.09],["-2^-11",1.49613e-08,257.033],["-2^-12",1.49296e-08,512.977],["-2^-13",1.49174e-08,1025.12],["-2^-14",1.4909e-08,2049.08],["-2^-15",1.49049e-08,4097.02],["-2^-16",1.4903e-08,8193],["-2^-17",1.49021e-08,16385],["-2^-18",1.48862e-08,32735],["-2^-19",1.48973e-08,65519],["-2^-20",1.49001e-08,131063],["-2^-21",1.49009e-08,262139],["-2^-22",1.49011e-08,524285],["-2^-23",1.49011e-08,1.04857e+06],["-2^-24",1.49011e-08,2.09715e+06],["-2^-25",1.49012e-08,4.1943e+06],["-2^-26",1.49007e-08,8.38835e+06],["-2^-27",1.49012e-08,8.47249e+08],["-2^-28",7.45058e-09,8.38861e+08],["-2^-29",3.72529e-09,8.30472e+08],["-2^-30",1.86265e-09,8.22084e+08],["-2^-31",9.31323e-10,8.13695e+08],["-2^-32",4.65661e-10,8.05306e+08],["-2^-33",2.32831e-10,7.96918e+08],["-2^-34",1.16415e-10,7.88529e+08],["-2^-35",5.82077e-11,7.80141e+08],["-2^-36",2.91038e-11,7.71752e+08],["-2^-37",1.45519e-11,7.63363e+08],["-2^-38",7.27596e-12,7.54975e+08],["-2^-39",3.63798e-12,7.46586e+08],["-2^-40",1.81899e-12,7.38198e+08],["-2^-41",9.09495e-13,7.29809e+08],["-2^-42",4.54747e-13,7.2142e+08],["-2^-43",2.27374e-13,7.13032e+08],["-2^-44",1.13687e-13,7.04643e+08],["-2^-45",5.68434e-14,6.96254e+08],["-2^-46",2.84217e-14,6.87866e+08],["-2^-47",1.42109e-14,6.79477e+08],["-2^-48",7.10543e-15,6.71089e+08],["-2^-49",3.55271e-15,6.627e+08],["-2^-50",1.77636e-15,6.54311e+08],["-2^-51",8.88178e-16,6.45923e+08],["-2^-52",4.44089e-16,6.37534e+08],["-2^-53",2.22045e-16,6.29146e+08],["-2^-54",1.11022e-16,6.20757e+08],["-2^-55",5.55111e-17,6.12368e+08],["-2^-56",2.77556e-17,6.0398e+08],["-2^-57",1.38778e-17,5.95591e+08],["-2^-58",6.93889e-18,5.87203e+08],["-2^-59",3.46945e-18,5.78814e+08],["-2^-60",1.73472e-18,5.70425e+08],["-2^-61",8.67362e-19,5.62037e+08],["-2^-62",4.33681e-19,5.53648e+08],["-2^-63",2.1684e-19,5.4526e+08],["-2^-64",1.0842e-19,5.36871e+08],["-2^-65",5.42101e-20,5.28482e+08],["-2^-66",2.71051e-20,5.20094e+08],["-2^-67",1.35525e-20,5.11705e+08],["-2^-68",6.77626e-21,5.03316e+08],["-2^-69",3.38813e-21,4.94928e+08],["-2^-70",1.69407e-21,4.86539e+08],["-2^-71",8.47033e-22,4.78151e+08],["-2^-72",4.23516e-22,4.69762e+08],["-2^-73",2.11758e-22,4.61373e+08],["-2^-74",1.05879e-22,4.52985e+08],["-2^-75",5.29396e-23,4.44596e+08],["-2^-76",2.64698e-23,4.36208e+08],["-2^-77",1.32349e-23,4.27819e+08],["-2^-78",6.61744e-24,4.1943e+08],["-2^-79",3.30872e-24,4.11042e+08],["-2^-80",1.65436e-24,4.02653e+08],["-2^-81",8.27181e-25,3.94265e+08],["-2^-82",4.1359e-25,3.85876e+08],["-2^-83",2.06795e-25,3.77487e+08],["-2^-84",1.03398e-25,3.69099e+08],["-2^-85",5.16988e-26,3.6071e+08],["-2^-86",2.58494e-26,3.52322e+08],["-2^-87",1.29247e-26,3.43933e+08],["-2^-88",6.46235e-27,3.35544e+08],["-2^-89",3.23117e-27,3.27156e+08],["-2^-90",1.61559e-27,3.18767e+08],["-2^-91",8.07794e-28,3.10378e+08],["-2^-92",4.03897e-28,3.0199e+08],["-2^-93",2.01948e-28,2.93601e+08],["-2^-94",1.00974e-28,2.85213e+08],["-2^-95",5.04871e-29,2.76824e+08],["-2^-96",2.52435e-29,2.68435e+08],["-2^-97",1.26218e-29,2.60047e+08],["-2^-98",6.31089e-30,2.51658e+08],["-2^-99",3.15544e-30,2.4327e+08],["-2^-100",1.57772e-30,2.34881e+08],["-2^-101",7.88861e-31,2.26492e+08],["-2^-102",3.9443e-31,2.18104e+08],["-2^-103",1.97215e-31,2.09715e+08],["-2^-104",9.86076e-32,2.01327e+08],["-2^-105",4.93038e-32,1.92938e+08],["-2^-106",2.46519e-32,1.84549e+08],["-2^-107",1.2326e-32,1.76161e+08],["-2^-108",6.16298e-33,1.67772e+08],["-2^-109",3.08149e-33,1.59384e+08],["-2^-110",1.54074e-33,1.50995e+08],["-2^-111",7.70372e-34,1.42606e+08],["-2^-112",3.85186e-34,1.34218e+08],["-2^-113",1.92593e-34,1.25829e+08],["-2^-114",9.62965e-35,1.17441e+08],["-2^-115",4.81482e-35,1.09052e+08],["-2^-116",2.40741e-35,1.00663e+08],["-2^-117",1.20371e-35,9.22747e+07],["-2^-118",6.01853e-36,8.38861e+07],["-2^-119",3.00927e-36,7.54975e+07],["-2^-120",1.50463e-36,6.71089e+07],["-2^-121",7.52316e-37,5.87203e+07],["-2^-122",3.76158e-37,5.03316e+07],["-2^-123",1.88079e-37,4.1943e+07],["-2^-124",9.40395e-38,3.35544e+07],["-2^-125",4.70198e-38,2.51658e+07],["-2^-126",2.35099e-38,1.67772e+07],["-SN",1.17546e-38,8.38835e+06],["0",0,1.17549e-38],["+SN",1.17546e-38,8.38835e+06],["+2^-126",2.35095e-38,1.6777e+07],["+2^-125",4.70191e-38,2.51656e+07],["+2^-124",9.40381e-38,3.35542e+07],["+2^-123",1.88076e-37,4.19428e+07],["+2^-122",3.76152e-37,5.03314e+07],["+2^-121",7.52305e-37,5.872e+07],["+2^-120",1.50461e-36,6.71086e+07],["+2^-119",3.00922e-36,7.54972e+07],["+2^-118",6.01844e-36,8.38858e+07],["+2^-117",1.20369e-35,9.22744e+07],["+2^-116",2.40738e-35,1.00663e+08],["+2^-115",4.81475e-35,1.09052e+08],["+2^-114",9.6295e-35,1.1744e+08],["+2^-113",1.9259e-34,1.25829e+08],["+2^-112",3.8518e-34,1.34217e+08],["+2^-111",7.7036e-34,1.42606e+08],["+2^-110",1.54072e-33,1.50995e+08],["+2^-109",3.08144e-33,1.59383e+08],["+2^-108",6.16288e-33,1.67772e+08],["+2^-107",1.23258e-32,1.76161e+08],["+2^-106",2.46515e-32,1.84549e+08],["+2^-105",4.93031e-32,1.92938e+08],["+2^-104",9.86061e-32,2.01326e+08],["+2^-103",1.97212e-31,2.09715e+08],["+2^-102",3.94424e-31,2.18104e+08],["+2^-101",7.88849e-31,2.26492e+08],["+2^-100",1.5777e-30,2.34881e+08],["+2^-99",3.1554e-30,2.43269e+08],["+2^-98",6.31079e-30,2.51658e+08],["+2^-97",1.26216e-29,2.60047e+08],["+2^-96",2.52432e-29,2.68435e+08],["+2^-95",5.04863e-29,2.76824e+08],["+2^-94",1.00973e-28,2.85212e+08],["+2^-93",2.01945e-28,2.93601e+08],["+2^-92",4.03891e-28,3.0199e+08],["+2^-91",8.07781e-28,3.10378e+08],["+2^-90",1.61556e-27,3.18767e+08],["+2^-89",3.23112e-27,3.27155e+08],["+2^-88",6.46225e-27,3.35544e+08],["+2^-87",1.29245e-26,3.43933e+08],["+2^-86",2.5849e-26,3.52321e+08],["+2^-85",5.1698e-26,3.6071e+08],["+2^-84",1.03396e-25,3.69098e+08],["+2^-83",2.06792e-25,3.77487e+08],["+2^-82",4.13584e-25,3.85876e+08],["+2^-81",8.27168e-25,3.94264e+08],["+2^-80",1.65434e-24,4.02653e+08],["+2^-79",3.30867e-24,4.11042e+08],["+2^-78",6.61734e-24,4.1943e+08],["+2^-77",1.32347e-23,4.27819e+08],["+2^-76",2.64694e-23,4.36207e+08],["+2^-75",5.29388e-23,4.44596e+08],["+2^-74",1.05878e-22,4.52985e+08],["+2^-73",2.11755e-22,4.61373e+08],["+2^-72",4.2351e-22,4.69762e+08],["+2^-71",8.4702e-22,4.7815e+08],["+2^-70",1.69404e-21,4.86539e+08],["+2^-69",3.38808e-21,4.94928e+08],["+2^-68",6.77616e-21,5.03316e+08],["+2^-67",1.35523e-20,5.11705e+08],["+2^-66",2.71046e-20,5.20093e+08],["+2^-65",5.42093e-20,5.28482e+08],["+2^-64",1.08419e-19,5.36871e+08],["+2^-63",2.16837e-19,5.45259e+08],["+2^-62",4.33674e-19,5.53648e+08],["+2^-61",8.67349e-19,5.62036e+08],["+2^-60",1.7347e-18,5.70425e+08],["+2^-59",3.46939e-18,5.78814e+08],["+2^-58",6.93879e-18,5.87202e+08],["+2^-57",1.38776e-17,5.95591e+08],["+2^-56",2.77552e-17,6.0398e+08],["+2^-55",5.55103e-17,6.12368e+08],["+2^-54",1.11021e-16,6.20757e+08],["+2^-53",2.22041e-16,6.29145e+08],["+2^-52",4.44082e-16,6.37534e+08],["+2^-51",8.88165e-16,6.45923e+08],["+2^-50",1.77633e-15,6.54311e+08],["+2^-49",3.55266e-15,6.627e+08],["+2^-48",7.10532e-15,6.71088e+08],["+2^-47",1.42106e-14,6.79477e+08],["+2^-46",2.84213e-14,6.87866e+08],["+2^-45",5.68426e-14,6.96254e+08],["+2^-44",1.13685e-13,7.04643e+08],["+2^-43",2.2737e-13,7.13031e+08],["+2^-42",4.5474e-13,7.2142e+08],["+2^-41",9.09481e-13,7.29809e+08],["+2^-40",1.81896e-12,7.38197e+08],["+2^-39",3.63792e-12,7.46586e+08],["+2^-38",7.27585e-12,7.54974e+08],["+2^-37",1.45517e-11,7.63363e+08],["+2^-36",2.91034e-11,7.71752e+08],["+2^-35",5.82068e-11,7.8014e+08],["+2^-34",1.16414e-10,7.88529e+08],["+2^-33",2.32827e-10,7.96918e+08],["+2^-32",4.65654e-10,8.05306e+08],["+2^-31",9.31308e-10,8.13695e+08],["+2^-30",1.86262e-09,8.22083e+08],["+2^-29",3.72523e-09,8.30472e+08],["+2^-28",7.45047e-09,8.38861e+08],["+2^-27",1.49009e-08,8.47249e+08],["+2^-26",2.98019e-08,8.55638e+08],["+2^-25",2.98023e-08,8.38861e+06],["+2^-24",2.98023e-08,4.1943e+06],["+2^-23",2.98023e-08,2.09715e+06],["+2^-22",2.98023e-08,1.04857e+06],["+2^-21",2.98021e-08,524284],["+2^-20",2.98014e-08,262136],["+2^-19",2.97987e-08,131056],["+2^-18",2.97878e-08,65504],["+2^-17",2.98014e-08,32767],["+2^-16",2.98005e-08,16384],["+2^-15",2.98023e-08,8192],["+2^-14",2.98025e-08,4351.99],["+2^-13",2.98044e-08,2815.96],["+2^-12",2.98091e-08,1024.23],["+2^-11",2.9812e-08,512.167],["+2^-10",2.98154e-08,256.113],["+2^-9",2.98591e-08,128.244],["+2^-8",3.00204e-08,85.3328],["+2^-7",3.01947e-08,43.675],["+2^-6",3.05713e-08,30.45],["+2^-5",3.14319e-08,8.43743],["+2^-4",3.31291e-08,7.77058],["+2^-3",3.58597e-08,3.99659],["+2^-2",4.14639e-08,2.29246],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",1.33955e-07,2.2474],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}
=== expm1 ===

{{#widget:FloatErrorGraph|id=tanhf_expm1
|data=[ ["-2^127",0,1.17549e-38],["-2^126",0,1.17549e-38],["-2^125",0,1.17549e-38],["-2^124",0,1.17549e-38],["-2^123",0,1.17549e-38],["-2^122",0,1.17549e-38],["-2^121",0,1.17549e-38],["-2^120",0,1.17549e-38],["-2^119",0,1.17549e-38],["-2^118",0,1.17549e-38],["-2^117",0,1.17549e-38],["-2^116",0,1.17549e-38],["-2^115",0,1.17549e-38],["-2^114",0,1.17549e-38],["-2^113",0,1.17549e-38],["-2^112",0,1.17549e-38],["-2^111",0,1.17549e-38],["-2^110",0,1.17549e-38],["-2^109",0,1.17549e-38],["-2^108",0,1.17549e-38],["-2^107",0,1.17549e-38],["-2^106",0,1.17549e-38],["-2^105",0,1.17549e-38],["-2^104",0,1.17549e-38],["-2^103",0,1.17549e-38],["-2^102",0,1.17549e-38],["-2^101",0,1.17549e-38],["-2^100",0,1.17549e-38],["-2^99",0,1.17549e-38],["-2^98",0,1.17549e-38],["-2^97",0,1.17549e-38],["-2^96",0,1.17549e-38],["-2^95",0,1.17549e-38],["-2^94",0,1.17549e-38],["-2^93",0,1.17549e-38],["-2^92",0,1.17549e-38],["-2^91",0,1.17549e-38],["-2^90",0,1.17549e-38],["-2^89",0,1.17549e-38],["-2^88",0,1.17549e-38],["-2^87",0,1.17549e-38],["-2^86",0,1.17549e-38],["-2^85",0,1.17549e-38],["-2^84",0,1.17549e-38],["-2^83",0,1.17549e-38],["-2^82",0,1.17549e-38],["-2^81",0,1.17549e-38],["-2^80",0,1.17549e-38],["-2^79",0,1.17549e-38],["-2^78",0,1.17549e-38],["-2^77",0,1.17549e-38],["-2^76",0,1.17549e-38],["-2^75",0,1.17549e-38],["-2^74",0,1.17549e-38],["-2^73",0,1.17549e-38],["-2^72",0,1.17549e-38],["-2^71",0,1.17549e-38],["-2^70",0,1.17549e-38],["-2^69",0,1.17549e-38],["-2^68",0,1.17549e-38],["-2^67",0,1.17549e-38],["-2^66",0,1.17549e-38],["-2^65",0,1.17549e-38],["-2^64",0,1.17549e-38],["-2^63",0,1.17549e-38],["-2^62",0,1.17549e-38],["-2^61",0,1.17549e-38],["-2^60",0,1.17549e-38],["-2^59",0,1.17549e-38],["-2^58",0,1.17549e-38],["-2^57",0,1.17549e-38],["-2^56",0,1.17549e-38],["-2^55",0,1.17549e-38],["-2^54",0,1.17549e-38],["-2^53",0,1.17549e-38],["-2^52",0,1.17549e-38],["-2^51",0,1.17549e-38],["-2^50",0,1.17549e-38],["-2^49",0,1.17549e-38],["-2^48",0,1.17549e-38],["-2^47",0,1.17549e-38],["-2^46",0,1.17549e-38],["-2^45",0,1.17549e-38],["-2^44",0,1.17549e-38],["-2^43",0,1.17549e-38],["-2^42",0,1.17549e-38],["-2^41",0,1.17549e-38],["-2^40",0,1.17549e-38],["-2^39",0,1.17549e-38],["-2^38",0,1.17549e-38],["-2^37",0,1.17549e-38],["-2^36",0,1.17549e-38],["-2^35",0,1.17549e-38],["-2^34",0,1.17549e-38],["-2^33",0,1.17549e-38],["-2^32",0,1.17549e-38],["-2^31",0,1.17549e-38],["-2^30",0,1.17549e-38],["-2^29",0,1.17549e-38],["-2^28",0,1.17549e-38],["-2^27",0,1.17549e-38],["-2^26",0,1.17549e-38],["-2^25",0,1.17549e-38],["-2^24",0,1.17549e-38],["-2^23",0,1.17549e-38],["-2^22",0,1.17549e-38],["-2^21",0,1.17549e-38],["-2^20",0,1.17549e-38],["-2^19",0,1.17549e-38],["-2^18",0,1.17549e-38],["-2^17",0,1.17549e-38],["-2^16",0,1.17549e-38],["-2^15",0,1.17549e-38],["-2^14",0,1.17549e-38],["-2^13",0,1.17549e-38],["-2^12",0,1.17549e-38],["-2^11",0,1.17549e-38],["-2^10",0,1.17549e-38],["-2^9",0,1.17549e-38],["-2^8",0,1.17549e-38],["-2^7",0,1.17549e-38],["-2^6",0,1.17549e-38],["-2^5",0,1.17549e-38],["-2^4",2.53131e-14,4.24683e-07],["-2^3",1.19138e-07,1.9988],["-2^2",1.46791e-07,2.46274],["-2^1",1.47723e-07,2.47839],["-2^0",1.40758e-07,2.36153],["-2^-1",1.13422e-07,2.23445],["-2^-2",6.38012e-08,2.14081],["-2^-3",2.69538e-08,1.80884],["-2^-4",1.21308e-08,1.62817],["-2^-5",5.59535e-09,1.50199],["-2^-6",2.69177e-09,1.44513],["-2^-7",1.31289e-09,1.4097],["-2^-8",6.49688e-10,1.3952],["-2^-9",3.32043e-10,1.42611],["-2^-10",1.66275e-10,1.4283],["-2^-11",8.08595e-11,1.38916],["-2^-12",3.8939e-11,1.33794],["-2^-13",1.9401e-11,1.33323],["-2^-14",7.88212e-12,1.08331],["-2^-15",3.71372e-12,1.02082],["-2^-16",1.82845e-12,1.0052],["-2^-17",9.10676e-13,1.0013],["-2^-18",4.54895e-13,1.00032],["-2^-19",2.27392e-13,1.00008],["-2^-20",1.13689e-13,1.00002],["-2^-21",5.68437e-14,1],["-2^-22",2.84217e-14,1],["-2^-23",1.42109e-14,1],["-2^-24",7.10543e-15,1],["-2^-25",3.55271e-15,1],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,1.17549e-38],["-2^-56",0,1.17549e-38],["-2^-57",0,1.17549e-38],["-2^-58",0,1.17549e-38],["-2^-59",0,1.17549e-38],["-2^-60",0,1.17549e-38],["-2^-61",0,1.17549e-38],["-2^-62",0,1.17549e-38],["-2^-63",0,1.17549e-38],["-2^-64",0,1.17549e-38],["-2^-65",0,1.17549e-38],["-2^-66",0,1.17549e-38],["-2^-67",0,1.17549e-38],["-2^-68",0,1.17549e-38],["-2^-69",0,1.17549e-38],["-2^-70",0,1.17549e-38],["-2^-71",0,1.17549e-38],["-2^-72",0,1.17549e-38],["-2^-73",0,1.17549e-38],["-2^-74",0,1.17549e-38],["-2^-75",0,1.17549e-38],["-2^-76",0,1.17549e-38],["-2^-77",0,1.17549e-38],["-2^-78",0,1.17549e-38],["-2^-79",0,1.17549e-38],["-2^-80",0,1.17549e-38],["-2^-81",0,1.17549e-38],["-2^-82",0,1.17549e-38],["-2^-83",0,1.17549e-38],["-2^-84",0,1.17549e-38],["-2^-85",0,1.17549e-38],["-2^-86",0,1.17549e-38],["-2^-87",0,1.17549e-38],["-2^-88",0,1.17549e-38],["-2^-89",0,1.17549e-38],["-2^-90",0,1.17549e-38],["-2^-91",0,1.17549e-38],["-2^-92",0,1.17549e-38],["-2^-93",0,1.17549e-38],["-2^-94",0,1.17549e-38],["-2^-95",0,1.17549e-38],["-2^-96",0,1.17549e-38],["-2^-97",0,1.17549e-38],["-2^-98",0,1.17549e-38],["-2^-99",0,1.17549e-38],["-2^-100",0,1.17549e-38],["-2^-101",0,1.17549e-38],["-2^-102",0,1.17549e-38],["-2^-103",0,1.17549e-38],["-2^-104",0,1.17549e-38],["-2^-105",0,1.17549e-38],["-2^-106",0,1.17549e-38],["-2^-107",0,1.17549e-38],["-2^-108",0,1.17549e-38],["-2^-109",0,1.17549e-38],["-2^-110",0,1.17549e-38],["-2^-111",0,1.17549e-38],["-2^-112",0,1.17549e-38],["-2^-113",0,1.17549e-38],["-2^-114",0,1.17549e-38],["-2^-115",0,1.17549e-38],["-2^-116",0,1.17549e-38],["-2^-117",0,1.17549e-38],["-2^-118",0,1.17549e-38],["-2^-119",0,1.17549e-38],["-2^-120",0,1.17549e-38],["-2^-121",0,1.17549e-38],["-2^-122",0,1.17549e-38],["-2^-123",0,1.17549e-38],["-2^-124",0,1.17549e-38],["-2^-125",0,1.17549e-38],["-2^-126",0,1.17549e-38],["-SN",0,1.17549e-38],["0",0,1.17549e-38],["+SN",0,1.17549e-38],["+2^-126",0,1.17549e-38],["+2^-125",0,1.17549e-38],["+2^-124",0,1.17549e-38],["+2^-123",0,1.17549e-38],["+2^-122",0,1.17549e-38],["+2^-121",0,1.17549e-38],["+2^-120",0,1.17549e-38],["+2^-119",0,1.17549e-38],["+2^-118",0,1.17549e-38],["+2^-117",0,1.17549e-38],["+2^-116",0,1.17549e-38],["+2^-115",0,1.17549e-38],["+2^-114",0,1.17549e-38],["+2^-113",0,1.17549e-38],["+2^-112",0,1.17549e-38],["+2^-111",0,1.17549e-38],["+2^-110",0,1.17549e-38],["+2^-109",0,1.17549e-38],["+2^-108",0,1.17549e-38],["+2^-107",0,1.17549e-38],["+2^-106",0,1.17549e-38],["+2^-105",0,1.17549e-38],["+2^-104",0,1.17549e-38],["+2^-103",0,1.17549e-38],["+2^-102",0,1.17549e-38],["+2^-101",0,1.17549e-38],["+2^-100",0,1.17549e-38],["+2^-99",0,1.17549e-38],["+2^-98",0,1.17549e-38],["+2^-97",0,1.17549e-38],["+2^-96",0,1.17549e-38],["+2^-95",0,1.17549e-38],["+2^-94",0,1.17549e-38],["+2^-93",0,1.17549e-38],["+2^-92",0,1.17549e-38],["+2^-91",0,1.17549e-38],["+2^-90",0,1.17549e-38],["+2^-89",0,1.17549e-38],["+2^-88",0,1.17549e-38],["+2^-87",0,1.17549e-38],["+2^-86",0,1.17549e-38],["+2^-85",0,1.17549e-38],["+2^-84",0,1.17549e-38],["+2^-83",0,1.17549e-38],["+2^-82",0,1.17549e-38],["+2^-81",0,1.17549e-38],["+2^-80",0,1.17549e-38],["+2^-79",0,1.17549e-38],["+2^-78",0,1.17549e-38],["+2^-77",0,1.17549e-38],["+2^-76",0,1.17549e-38],["+2^-75",0,1.17549e-38],["+2^-74",0,1.17549e-38],["+2^-73",0,1.17549e-38],["+2^-72",0,1.17549e-38],["+2^-71",0,1.17549e-38],["+2^-70",0,1.17549e-38],["+2^-69",0,1.17549e-38],["+2^-68",0,1.17549e-38],["+2^-67",0,1.17549e-38],["+2^-66",0,1.17549e-38],["+2^-65",0,1.17549e-38],["+2^-64",0,1.17549e-38],["+2^-63",0,1.17549e-38],["+2^-62",0,1.17549e-38],["+2^-61",0,1.17549e-38],["+2^-60",0,1.17549e-38],["+2^-59",0,1.17549e-38],["+2^-58",0,1.17549e-38],["+2^-57",0,1.17549e-38],["+2^-56",0,1.17549e-38],["+2^-55",0,1.17549e-38],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",0,1.17549e-38],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",0,1.17549e-38],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",0,1.17549e-38],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",3.55271e-15,1],["+2^-24",7.10543e-15,1],["+2^-23",1.42109e-14,1],["+2^-22",2.84217e-14,1],["+2^-21",5.68437e-14,1],["+2^-20",1.13689e-13,1.00002],["+2^-19",2.27391e-13,1.00008],["+2^-18",4.54892e-13,1.00032],["+2^-17",9.10665e-13,1.00129],["+2^-16",1.82841e-12,1.00518],["+2^-15",3.71353e-12,1.02077],["+2^-14",1.39798e-11,1.92136],["+2^-13",2.63239e-11,1.80896],["+2^-12",5.37712e-11,1.84757],["+2^-11",1.09932e-10,1.88861],["+2^-10",2.24787e-10,1.93091],["+2^-9",4.3452e-10,1.86625],["+2^-8",1.01783e-09,2.18576],["+2^-7",2.10562e-09,2.26089],["+2^-6",4.03954e-09,2.16871],["+2^-5",8.15746e-09,2.18975],["+2^-4",1.5545e-08,2.08641],["+2^-3",2.62153e-08,2.05864],["+2^-2",4.09295e-08,1.73804],["+2^-1",8.02409e-08,1.34622],["+2^0",8.66201e-08,1.45324],["+2^1",8.8654e-08,1.48737],["+2^2",6.7976e-08,1.14045],["+2^3",7.44117e-08,1.24842],["+2^4",2.53131e-14,4.24683e-07],["+2^5",0,null],["+2^6",0,null],["+2^7",0,null],["+2^8",0,null],["+2^9",0,null],["+2^10",0,null],["+2^11",0,null],["+2^12",0,null],["+2^13",0,null],["+2^14",0,null],["+2^15",0,null],["+2^16",0,null],["+2^17",0,null],["+2^18",0,null],["+2^19",0,null],["+2^20",0,null],["+2^21",0,null],["+2^22",0,null],["+2^23",0,null],["+2^24",0,null],["+2^25",0,null],["+2^26",0,null],["+2^27",0,null],["+2^28",0,null],["+2^29",0,null],["+2^30",0,null],["+2^31",0,null],["+2^32",0,null],["+2^33",0,null],["+2^34",0,null],["+2^35",0,null],["+2^36",0,null],["+2^37",0,null],["+2^38",0,null],["+2^39",0,null],["+2^40",0,null],["+2^41",0,null],["+2^42",0,null],["+2^43",0,null],["+2^44",0,null],["+2^45",0,null],["+2^46",0,null],["+2^47",0,null],["+2^48",0,null],["+2^49",0,null],["+2^50",0,null],["+2^51",0,null],["+2^52",0,null],["+2^53",0,null],["+2^54",0,null],["+2^55",0,null],["+2^56",0,null],["+2^57",0,null],["+2^58",0,null],["+2^59",0,null],["+2^60",0,null],["+2^61",0,null],["+2^62",0,null],["+2^63",0,null],["+2^64",0,null],["+2^65",0,null],["+2^66",0,null],["+2^67",0,null],["+2^68",0,null],["+2^69",0,null],["+2^70",0,null],["+2^71",0,null],["+2^72",0,null],["+2^73",0,null],["+2^74",0,null],["+2^75",0,null],["+2^76",0,null],["+2^77",0,null],["+2^78",0,null],["+2^79",0,null],["+2^80",0,null],["+2^81",0,null],["+2^82",0,null],["+2^83",0,null],["+2^84",0,null],["+2^85",0,null],["+2^86",0,null],["+2^87",0,null],["+2^88",0,null],["+2^89",0,null],["+2^90",0,null],["+2^91",0,null],["+2^92",0,null],["+2^93",0,null],["+2^94",0,null],["+2^95",0,null],["+2^96",0,null],["+2^97",0,null],["+2^98",0,null],["+2^99",0,null],["+2^100",0,null],["+2^101",0,null],["+2^102",0,null],["+2^103",0,null],["+2^104",0,null],["+2^105",0,null],["+2^106",0,null],["+2^107",0,null],["+2^108",0,null],["+2^109",0,null],["+2^110",0,null],["+2^111",0,null],["+2^112",0,null],["+2^113",0,null],["+2^114",0,null],["+2^115",0,null],["+2^116",0,null],["+2^117",0,null],["+2^118",0,null],["+2^119",0,null],["+2^120",0,null],["+2^121",0,null],["+2^122",0,null],["+2^123",0,null],["+2^124",0,null],["+2^125",0,null],["+2^126",0,null],["+2^127",0,null] ]
}}

==Taylor==

{{#widget:FloatErrorGraph|id=taylor
|data=[ ["-2^127",0,0],["-2^126",0,0],["-2^125",0,0],["-2^124",0,0],["-2^123",0,0],["-2^122",0,0],["-2^121",0,0],["-2^120",0,0],["-2^119",0,0],["-2^118",0,0],["-2^117",0,0],["-2^116",0,0],["-2^115",0,0],["-2^114",0,0],["-2^113",0,0],["-2^112",0,0],["-2^111",0,0],["-2^110",0,0],["-2^109",0,0],["-2^108",0,0],["-2^107",0,0],["-2^106",0,0],["-2^105",0,0],["-2^104",0,0],["-2^103",0,0],["-2^102",0,0],["-2^101",0,0],["-2^100",0,0],["-2^99",0,0],["-2^98",0,0],["-2^97",0,0],["-2^96",0,0],["-2^95",0,0],["-2^94",0,0],["-2^93",0,0],["-2^92",0,0],["-2^91",0,0],["-2^90",0,0],["-2^89",0,0],["-2^88",0,0],["-2^87",0,0],["-2^86",0,0],["-2^85",0,0],["-2^84",0,0],["-2^83",0,0],["-2^82",0,0],["-2^81",0,0],["-2^80",0,0],["-2^79",0,0],["-2^78",0,0],["-2^77",0,0],["-2^76",0,0],["-2^75",0,0],["-2^74",0,0],["-2^73",0,0],["-2^72",0,0],["-2^71",0,0],["-2^70",0,0],["-2^69",0,0],["-2^68",0,0],["-2^67",0,0],["-2^66",0,0],["-2^65",0,0],["-2^64",0,0],["-2^63",1.84467e+19,3.09485e+26],["-2^62",9.22336e+18,1.54742e+26],["-2^61",4.61168e+18,7.73712e+25],["-2^60",2.30584e+18,3.86856e+25],["-2^59",1.15292e+18,1.93428e+25],["-2^58",5.7646e+17,9.6714e+24],["-2^57",2.8823e+17,4.8357e+24],["-2^56",1.44115e+17,2.41785e+24],["-2^55",7.20575e+16,1.20892e+24],["-2^54",3.60288e+16,6.04462e+23],["-2^53",1.80144e+16,3.02231e+23],["-2^52",9.00719e+15,1.51116e+23],["-2^51",4.5036e+15,7.55578e+22],["-2^50",2.2518e+15,3.77789e+22],["-2^49",1.1259e+15,1.88894e+22],["-2^48",5.62949e+14,9.44472e+21],["-2^47",2.81475e+14,4.72236e+21],["-2^46",1.40737e+14,2.36118e+21],["-2^45",7.03687e+13,1.18059e+21],["-2^44",3.51843e+13,5.90295e+20],["-2^43",1.75922e+13,2.95148e+20],["-2^42",8.79608e+12,1.47574e+20],["-2^41",4.39804e+12,7.37869e+19],["-2^40",2.19902e+12,3.68935e+19],["-2^39",1.09951e+12,1.84467e+19],["-2^38",5.49755e+11,9.22336e+18],["-2^37",2.74878e+11,4.61168e+18],["-2^36",1.37439e+11,2.30584e+18],["-2^35",6.87194e+10,1.15292e+18],["-2^34",3.43597e+10,5.7646e+17],["-2^33",1.71799e+10,2.8823e+17],["-2^32",8.58993e+09,1.44115e+17],["-2^31",4.29085e+09,7.19885e+16],["-2^30",2.14542e+09,3.59943e+16],["-2^29",1.07271e+09,1.79971e+16],["-2^28",5.36356e+08,8.99856e+15],["-2^27",2.68178e+08,4.49928e+15],["-2^26",1.34089e+08,2.24964e+15],["-2^25",6.70445e+07,1.12482e+15],["-2^24",3.35223e+07,5.6241e+14],["-2^23",1.67611e+07,2.81205e+14],["-2^22",8.38056e+06,1.40603e+14],["-2^21",4.19028e+06,7.03013e+13],["-2^20",2.09514e+06,3.51506e+13],["-2^19",1.04757e+06,1.75753e+13],["-2^18",523784,8.78764e+12],["-2^17",261892,4.39381e+12],["-2^16",130945,2.1969e+12],["-2^15",65472.2,1.09844e+12],["-2^14",32735.6,5.49212e+11],["-2^13",16367.3,2.74598e+11],["-2^12",8183.15,1.3729e+11],["-2^11",4091.07,6.86368e+10],["-2^10",2045.04,3.431e+10],["-2^9",1022.02,1.71466e+10],["-2^8",510.509,8.56492e+09],["-2^7",254.754,4.27406e+09],["-2^6",126.876,2.12862e+09],["-2^5",62.9354,1.05588e+09],["-2^4",30.9629,5.19471e+08],["-2^3",14.9718,2.51185e+08],["-2^2",6.96672,1.16882e+08],["-2^1",2.94633,4.94312e+07],["-2^0",0.940715,1.57826e+07],["-2^-1",0.127635,2.14135e+06],["-2^-2",0.00318735,106950],["-2^-3",3.11475e-05,2090.28],["-2^-4",2.57737e-07,34.5929],["-2^-5",3.89213e-09,1.04478],["-2^-6",9.51472e-10,0.510818],["-2^-7",4.66402e-10,0.500795],["-2^-8",2.32917e-10,0.500185],["-2^-9",1.16423e-10,0.500033],["-2^-10",5.82082e-11,0.500004],["-2^-11",2.91037e-11,0.499998],["-2^-12",1.45519e-11,0.5],["-2^-13",4.85062e-12,0.333332],["-2^-14",6.06328e-13,0.0833331],["-2^-15",7.5791e-14,0.0208333],["-2^-16",9.47387e-15,0.00520832],["-2^-17",1.18424e-15,0.00130208],["-2^-18",1.48029e-16,0.00032552],["-2^-19",1.85039e-17,8.13808e-05],["-2^-20",2.31304e-18,2.03457e-05],["-2^-21",2.89156e-19,5.08688e-06],["-2^-22",3.61577e-20,1.27219e-06],["-2^-23",4.52633e-21,3.18512e-07],["-2^-24",5.691e-22,8.00937e-08],["-2^-25",7.27919e-23,2.04891e-08],["-2^-26",9.92617e-24,5.58794e-09],["-2^-27",1.65436e-24,1.86265e-09],["-2^-28",8.27181e-25,1.86265e-09],["-2^-29",4.1359e-25,1.86265e-09],["-2^-30",2.06795e-25,1.86265e-09],["-2^-31",1.03398e-25,1.86265e-09],["-2^-32",5.16988e-26,1.86265e-09],["-2^-33",2.58494e-26,1.86265e-09],["-2^-34",1.29247e-26,1.86265e-09],["-2^-35",6.46235e-27,1.86265e-09],["-2^-36",3.23117e-27,1.86265e-09],["-2^-37",1.61559e-27,1.86265e-09],["-2^-38",8.07794e-28,1.86265e-09],["-2^-39",4.03897e-28,1.86265e-09],["-2^-40",2.01948e-28,1.86265e-09],["-2^-41",1.00974e-28,1.86265e-09],["-2^-42",5.04871e-29,1.86265e-09],["-2^-43",2.52435e-29,1.86265e-09],["-2^-44",1.26218e-29,1.86265e-09],["-2^-45",6.31089e-30,1.86265e-09],["-2^-46",3.15544e-30,1.86265e-09],["-2^-47",1.57772e-30,1.86265e-09],["-2^-48",7.88861e-31,1.86265e-09],["-2^-49",3.9443e-31,1.86265e-09],["-2^-50",1.97215e-31,1.86265e-09],["-2^-51",9.86076e-32,1.86265e-09],["-2^-52",4.93038e-32,1.86265e-09],["-2^-53",2.46519e-32,1.86265e-09],["-2^-54",1.2326e-32,1.86265e-09],["-2^-55",0,0],["-2^-56",0,0],["-2^-57",0,0],["-2^-58",0,0],["-2^-59",0,0],["-2^-60",0,0],["-2^-61",0,0],["-2^-62",0,0],["-2^-63",0,0],["-2^-64",0,0],["-2^-65",0,0],["-2^-66",0,0],["-2^-67",0,0],["-2^-68",0,0],["-2^-69",0,0],["-2^-70",0,0],["-2^-71",0,0],["-2^-72",0,0],["-2^-73",0,0],["-2^-74",0,0],["-2^-75",0,0],["-2^-76",0,0],["-2^-77",0,0],["-2^-78",0,0],["-2^-79",0,0],["-2^-80",0,0],["-2^-81",0,0],["-2^-82",0,0],["-2^-83",0,0],["-2^-84",0,0],["-2^-85",0,0],["-2^-86",0,0],["-2^-87",0,0],["-2^-88",0,0],["-2^-89",0,0],["-2^-90",0,0],["-2^-91",0,0],["-2^-92",0,0],["-2^-93",0,0],["-2^-94",0,0],["-2^-95",0,0],["-2^-96",0,0],["-2^-97",0,0],["-2^-98",0,0],["-2^-99",0,0],["-2^-100",0,0],["-2^-101",0,0],["-2^-102",0,0],["-2^-103",0,0],["-2^-104",0,0],["-2^-105",0,0],["-2^-106",0,0],["-2^-107",0,0],["-2^-108",0,0],["-2^-109",0,0],["-2^-110",0,0],["-2^-111",0,0],["-2^-112",0,0],["-2^-113",0,0],["-2^-114",0,0],["-2^-115",0,0],["-2^-116",0,0],["-2^-117",0,0],["-2^-118",0,0],["-2^-119",0,0],["-2^-120",0,0],["-2^-121",0,0],["-2^-122",0,0],["-2^-123",0,0],["-2^-124",0,0],["-2^-125",0,0],["-2^-126",0,0],["-SN",0,0],["0",0,0],["+SN",0,0],["+2^-126",0,0],["+2^-125",0,0],["+2^-124",0,0],["+2^-123",0,0],["+2^-122",0,0],["+2^-121",0,0],["+2^-120",0,0],["+2^-119",0,0],["+2^-118",0,0],["+2^-117",0,0],["+2^-116",0,0],["+2^-115",0,0],["+2^-114",0,0],["+2^-113",0,0],["+2^-112",0,0],["+2^-111",0,0],["+2^-110",0,0],["+2^-109",0,0],["+2^-108",0,0],["+2^-107",0,0],["+2^-106",0,0],["+2^-105",0,0],["+2^-104",0,0],["+2^-103",0,0],["+2^-102",0,0],["+2^-101",0,0],["+2^-100",0,0],["+2^-99",0,0],["+2^-98",0,0],["+2^-97",0,0],["+2^-96",0,0],["+2^-95",0,0],["+2^-94",0,0],["+2^-93",0,0],["+2^-92",0,0],["+2^-91",0,0],["+2^-90",0,0],["+2^-89",0,0],["+2^-88",0,0],["+2^-87",0,0],["+2^-86",0,0],["+2^-85",0,0],["+2^-84",0,0],["+2^-83",0,0],["+2^-82",0,0],["+2^-81",0,0],["+2^-80",0,0],["+2^-79",0,0],["+2^-78",0,0],["+2^-77",0,0],["+2^-76",0,0],["+2^-75",0,0],["+2^-74",0,0],["+2^-73",0,0],["+2^-72",0,0],["+2^-71",0,0],["+2^-70",0,0],["+2^-69",0,0],["+2^-68",0,0],["+2^-67",0,0],["+2^-66",0,0],["+2^-65",0,0],["+2^-64",0,0],["+2^-63",0,0],["+2^-62",0,0],["+2^-61",0,0],["+2^-60",0,0],["+2^-59",0,0],["+2^-58",0,0],["+2^-57",0,0],["+2^-56",0,0],["+2^-55",0,0],["+2^-54",1.2326e-32,1.86265e-09],["+2^-53",2.46519e-32,1.86265e-09],["+2^-52",4.93038e-32,1.86265e-09],["+2^-51",9.86076e-32,1.86265e-09],["+2^-50",1.97215e-31,1.86265e-09],["+2^-49",3.9443e-31,1.86265e-09],["+2^-48",7.88861e-31,1.86265e-09],["+2^-47",1.57772e-30,1.86265e-09],["+2^-46",3.15544e-30,1.86265e-09],["+2^-45",6.31089e-30,1.86265e-09],["+2^-44",1.26218e-29,1.86265e-09],["+2^-43",2.52435e-29,1.86265e-09],["+2^-42",5.04871e-29,1.86265e-09],["+2^-41",1.00974e-28,1.86265e-09],["+2^-40",2.01948e-28,1.86265e-09],["+2^-39",4.03897e-28,1.86265e-09],["+2^-38",8.07794e-28,1.86265e-09],["+2^-37",1.61559e-27,1.86265e-09],["+2^-36",3.23117e-27,1.86265e-09],["+2^-35",6.46235e-27,1.86265e-09],["+2^-34",1.29247e-26,1.86265e-09],["+2^-33",2.58494e-26,1.86265e-09],["+2^-32",5.16988e-26,1.86265e-09],["+2^-31",1.03398e-25,1.86265e-09],["+2^-30",2.06795e-25,1.86265e-09],["+2^-29",4.1359e-25,1.86265e-09],["+2^-28",8.27181e-25,1.86265e-09],["+2^-27",1.65436e-24,1.86265e-09],["+2^-26",9.92617e-24,5.58794e-09],["+2^-25",7.27919e-23,2.04891e-08],["+2^-24",5.691e-22,8.00937e-08],["+2^-23",4.52633e-21,3.18512e-07],["+2^-22",3.61577e-20,1.27219e-06],["+2^-21",2.89156e-19,5.08688e-06],["+2^-20",2.31304e-18,2.03457e-05],["+2^-19",1.85039e-17,8.13808e-05],["+2^-18",1.48029e-16,0.00032552],["+2^-17",1.18424e-15,0.00130208],["+2^-16",9.47387e-15,0.00520832],["+2^-15",7.5791e-14,0.0208333],["+2^-14",6.06328e-13,0.0833331],["+2^-13",4.85062e-12,0.333332],["+2^-12",1.45519e-11,0.5],["+2^-11",2.91037e-11,0.499998],["+2^-10",5.82082e-11,0.500004],["+2^-9",1.16423e-10,0.500033],["+2^-8",2.32917e-10,0.500185],["+2^-7",4.66402e-10,0.500795],["+2^-6",9.51472e-10,0.510818],["+2^-5",3.89213e-09,1.04478],["+2^-4",2.57737e-07,34.5929],["+2^-3",3.11475e-05,2090.28],["+2^-2",0.00318735,106950],["+2^-1",0.127635,2.14135e+06],["+2^0",0.940715,1.57826e+07],["+2^1",2.94633,4.94312e+07],["+2^2",6.96672,1.16882e+08],["+2^3",14.9718,1.33874e+08],["+2^4",30.9629,2.59736e+08],["+2^5",62.9354,5.27941e+08],["+2^6",126.876,1.06431e+09],["+2^7",254.754,2.13703e+09],["+2^8",510.509,4.28246e+09],["+2^9",1022.02,8.57331e+09],["+2^10",2045.04,1.7155e+10],["+2^11",4091.07,3.43184e+10],["+2^12",8183.15,6.86452e+10],["+2^13",16367.3,1.37299e+11],["+2^14",32735.6,2.74606e+11],["+2^15",65472.2,5.4922e+11],["+2^16",130945,1.09845e+12],["+2^17",261892,2.19691e+12],["+2^18",523784,4.39382e+12],["+2^19",1.04757e+06,8.78765e+12],["+2^20",2.09514e+06,1.75753e+13],["+2^21",4.19028e+06,3.51506e+13],["+2^22",8.38056e+06,7.03013e+13],["+2^23",1.67611e+07,1.40603e+14],["+2^24",3.35223e+07,2.81205e+14],["+2^25",6.70445e+07,5.6241e+14],["+2^26",1.34089e+08,1.12482e+15],["+2^27",2.68178e+08,2.24964e+15],["+2^28",5.36356e+08,4.49928e+15],["+2^29",1.07271e+09,8.99856e+15],["+2^30",2.14542e+09,1.79971e+16],["+2^31",4.29085e+09,3.59943e+16],["+2^32",8.58993e+09,7.20575e+16],["+2^33",1.71799e+10,1.44115e+17],["+2^34",3.43597e+10,2.8823e+17],["+2^35",6.87194e+10,5.7646e+17],["+2^36",1.37439e+11,1.15292e+18],["+2^37",2.74878e+11,2.30584e+18],["+2^38",5.49755e+11,4.61168e+18],["+2^39",1.09951e+12,9.22336e+18],["+2^40",2.19902e+12,1.84467e+19],["+2^41",4.39804e+12,3.68935e+19],["+2^42",8.79608e+12,7.37869e+19],["+2^43",1.75922e+13,1.47574e+20],["+2^44",3.51843e+13,2.95148e+20],["+2^45",7.03687e+13,5.90295e+20],["+2^46",1.40737e+14,1.18059e+21],["+2^47",2.81475e+14,2.36118e+21],["+2^48",5.62949e+14,4.72236e+21],["+2^49",1.1259e+15,9.44472e+21],["+2^50",2.2518e+15,1.88894e+22],["+2^51",4.5036e+15,3.77789e+22],["+2^52",9.00719e+15,7.55578e+22],["+2^53",1.80144e+16,1.51116e+23],["+2^54",3.60288e+16,3.02231e+23],["+2^55",7.20575e+16,6.04462e+23],["+2^56",1.44115e+17,1.20892e+24],["+2^57",2.8823e+17,2.41785e+24],["+2^58",5.7646e+17,4.8357e+24],["+2^59",1.15292e+18,9.6714e+24],["+2^60",2.30584e+18,1.93428e+25],["+2^61",4.61168e+18,3.86856e+25],["+2^62",9.22336e+18,7.73712e+25],["+2^63",1.84467e+19,1.54742e+26],["+2^64",0,0],["+2^65",0,0],["+2^66",0,0],["+2^67",0,0],["+2^68",0,0],["+2^69",0,0],["+2^70",0,0],["+2^71",0,0],["+2^72",0,0],["+2^73",0,0],["+2^74",0,0],["+2^75",0,0],["+2^76",0,0],["+2^77",0,0],["+2^78",0,0],["+2^79",0,0],["+2^80",0,0],["+2^81",0,0],["+2^82",0,0],["+2^83",0,0],["+2^84",0,0],["+2^85",0,0],["+2^86",0,0],["+2^87",0,0],["+2^88",0,0],["+2^89",0,0],["+2^90",0,0],["+2^91",0,0],["+2^92",0,0],["+2^93",0,0],["+2^94",0,0],["+2^95",0,0],["+2^96",0,0],["+2^97",0,0],["+2^98",0,0],["+2^99",0,0],["+2^100",0,0],["+2^101",0,0],["+2^102",0,0],["+2^103",0,0],["+2^104",0,0],["+2^105",0,0],["+2^106",0,0],["+2^107",0,0],["+2^108",0,0],["+2^109",0,0],["+2^110",0,0],["+2^111",0,0],["+2^112",0,0],["+2^113",0,0],["+2^114",0,0],["+2^115",0,0],["+2^116",0,0],["+2^117",0,0],["+2^118",0,0],["+2^119",0,0],["+2^120",0,0],["+2^121",0,0],["+2^122",0,0],["+2^123",0,0],["+2^124",0,0],["+2^125",0,0],["+2^126",0,0],["+2^127",0,0] ]
}}

== Localized Approximation Functions ==
{| class="wikitable"
!Id
!Formula
!ULP <= 2.0
!NAN
|-
|signum
|<math>\sgn{x}</math>
| -inf < x < -8.31763
8.31763 < x < inf
| -
|-
|poly1
|<math>x</math>
| -0.000704229 < x < 0.000704229
| -
|-
|poly3
|<math>x - \frac{1}{3} x^3</math>
| -0.0291781 < x < 0.0291781
| -
|-
|lampert7
|<math>\frac{x \cdot (135135 + x^2 \cdot (17325 + x^2 \cdot (378 + x^2)))}{135135 + x^2 \cdot (62370 + x^2 \cdot (3150 + 28 \cdot x^2))}
</math>
| -0.000947416 < x < 0.000947207
| -inf < x < -1.51629e+06
1.51629e+06 < x < inf
|-
|pade
|<math>x + x \cdot \frac{x^2 \cdot \left( p_1 \cdot x^2 + p_0 \right)}{\left( x^2 + q_1 \right) \cdot x^2 + q_0}

</math> <math>\begin{align}
p_0 &= -0.2059432032 \\
p_1 &= -0.0009577527 \\
q_0 &= 0.6178299136 \\
q_1 &= 0.25
\end{align}
</math>
| -0.0016973 < x < 0.0016973
| -inf < x < -6.14896e+18
6.14896e+18 < x < inf
|}

== External Links ==

* https://yaikhom.com/2020-04-28-localised-approximation-of-hyperbolic-tangents.html#mjx-eqn-eqn7-th%20degree%20lambert%20approximant
*https://github.com/llvm/llvm-project/blob/main/libc/src/math/generic/tanhf.cpp
*https://forums.developer.nvidia.com/t/hardware-accelerated-tanh-on-turing/173291
[[Category:Numerical Precision]]

Category:Numerical Precision

2026-02-03T09:50:52Z

Timo.stripf: Timo.stripf moved page Category:Math Function Accuracy to Category:Numerical Precision without leaving a redirect

[[Category:Categories]]

Finding Highest ULP of Math Functions

2026-02-03T09:50:27Z

Timo.stripf:

This article discusses methods for measuring and finding the maximum Unit in the Last Place (ULP) error in floating-point math functions. It covers both single-precision (binary32) and double-precision (binary64) IEEE 754 formats, highlighting the impracticality of exhaustive search for large input spaces and presenting a universal search algorithm that focuses on identifying inputs likely to produce large errors.

== Background and Motivation ==

In the IEEE 754 single-precision format (binary32), there are <math>2^{32} - 2^{24}</math> finite representable values (excluding <math>\pm\infty</math> and NaN). For a function with one input parameter, it is still feasible to exhaustively test all possible values. However, difficulties arise when:

* '''Single-precision bivariate functions''': The input space could be as large as <math>2^{64}</math> pairs of 32-bit floats, making an exhaustive search prohibitively expensive with academic resources.
* '''Double-precision (binary64) functions''': The input space grows even larger (<math>2^{64}</math> possible 64-bit patterns for a univariate function, and <math>2^{128}</math> for a bivariate function), making a full enumeration infeasible.

Although a full enumeration for some single-precision univariate functions can be practical, bivariate or double-precision functions present exponentially larger input spaces. Exhaustive testing quickly becomes impossible with standard computational resources. Hence, a “black-box” approach is beneficial:

* It avoids manual analysis of library code.
* It does not require highly specialized tests for each function or library.
* It can detect errors such as incorrect argument reduction for large inputs (e.g., near 2^{1024} for sin or cos) without manually coding edge-case checks.

== Motivation for a Universal Algorithm ==

Exhaustive search for single-precision univariate functions is sometimes practical. However, for higher-dimensional or double-precision functions, the sheer number of inputs makes exhaustive testing nearly impossible with standard computational resources.

A “black-box” approach to testing:
* Avoids manual analysis of library code.
* Eliminates the need for highly specialized tests for each function or library.
* Allows the detection of errors such as incorrect argument reduction for large inputs (e.g., near <math>2^{1024}</math> for <math>\sin</math> or <math>\cos</math>) without manually coding those edge-case checks.

== Universal Search Algorithm ==

This section outlines the subdivision-based search algorithm for univariate double-precision functions, though the same approach can be adapted to:
* Any IEEE 754 format, provided there is a corresponding integer type of the same bit width.
* Bivariate or higher-dimensional functions (with appropriate modifications).

=== Algorithm Overview ===

Suppose we have a univariate function <math> f(x) </math> in double precision. Each possible input <math> x </math> can be associated with a 64-bit integer, thanks to the binary64 format. Let:

* <code>to_double(uint64_t)</code> be a function that converts a 64-bit unsigned integer to its corresponding double-precision floating-point number.
* <code>check(double x)</code> be a function that checks f at position x and returns the ULP error
* [a, b] be an inclusive 64-bit integer range representing a subrange of possible inputs.
* t be a threshold indicating when to switch from sampling to exhaustive checking.

The algorithm proceeds as follows:

# '''Base Case (Exhaustive Check)''': If the interval size <math>(b - a) < t</math>, then test '''all''' integers i in [a, b].
#*For each i in [a, b]
#**convert i to double: <code>x = to_double(i)</code>
#**check f at x: <code>err = check(x)</code>
#**and keep track of the largest observed error.
#'''Recursive Subdivision''': If the interval size <math>(b - a) \geq t</math>
#*Subdivide [a, b] into two (almost equal) sub-intervals.
#*In each subinterval, randomly sample t values.
#*Convert each sampled integer to double and compute the ULP error as above.
#*Identify the subrange producing the largest observed error.
#*Recurse on that subrange.
This subdivision continues until the interval width becomes less than the total number of function evaluations already performed. At that point, the algorithm can afford an exhaustive search on the remaining values without excessively prolonging the total runtime.

=== Practical Optimization: “Top-K” Subintervals===

A more refined strategy, suggested by Eric Schneider, modifies the subdivision step to keep multiple promising subintervals at each level instead of focusing on just one:

*Maintain a list of, for example, 20 subintervals (instead of one) with the largest observed errors.
*Subdivide each selected subinterval, yielding 40 (or 80 for bivariate) new subintervals in total.
* Retain the 20 most “promising” intervals out of these newly generated ones.
*Continue until the intervals are small enough to be exhaustively checked or until resources are exhausted.

===Additional Strategies for Selecting Subintervals===

Several heuristics can be used to choose the “best” subintervals at each recursion step:

# '''Maximal ULP error''': Keep the subinterval with the highest single-sample error.
# '''Maximal average ULP error''': Keep the subinterval whose sampled points give the highest average ULP error (ignoring cases that produce NaN or <math>\pm\infty</math>).
# '''Statistical estimate''': Model the error distribution on each subinterval and estimate the maximum ULP error for the number of points in that interval.

In practice, the first strategy often finds large errors most quickly, while the second and third strategies can occasionally discover extreme corner cases. A useful approach on multi-core systems is to run strategy 2 and 3 each on one core and strategy 1 on the remaining cores.
===Extension to Bivariate Functions===
For bivariate functions, there are two approaches to adapt the algorithm. Suppose we have a bivariate function <math> f(x, y) </math> in single precision and the <code>to_single(uint32_t>)</code> function that converts a 32-bit unsigned integer to its corresponding single-precision float.

* We can use two 32-bit integers i1 and i2 to represent each input. The integers are then converted to single-precision floats using <code>to_single</code>. The intervals become rectangular intervals and subdivide of the rectangular intervals create four sub-intervals.
* Alternatively, we can use a single 64-bit integer to represent the input. The intervals are still one-dimensional. The 64-bit integer is split into two 32-bit using odd and even bits before converting them to single-precision floats. The algorithm then proceeds as before.

==Conclusions==

Finding the highest ULP error for floating-point math functions is an essential part of ensuring numerical accuracy. While exhaustive methods are still possible for some single-precision use cases, more complex scenarios require search-based or sampling-based algorithms. The general-purpose algorithm outlined here:

*Does not rely on inspecting the internal implementations of math libraries.
*Does not require function-specific logic.
*Subdivides large input spaces, refining searches where large errors are found.

The approach yields a practical, largely automated tool for pinpointing problematic inputs—even in extreme domains—offering valuable insights into the accuracy and robustness of floating-point math libraries.
[[Category:Numerical Precision]]

Numerical Precision in ONNX and AI Inference

2026-02-03T09:46:20Z

Timo.stripf:

ULP Difference of Float Numbers

2026-02-03T09:19:30Z

Timo.stripf:

The '''Unit in the Last Place (ULP)''' is the smallest difference between two adjacent floating-point numbers at a specific precision. It measures the granularity of floating-point representations and is essential for understanding rounding errors and numerical stability. For example, a ULP near 1.0 is typically smaller than a ULP near 10.0, as the step size depends on the exponent of the number.

'''ULP differences''' are often used to evaluate the accuracy of floating-point operations. When comparing two floating-point numbers, a difference of 1 ULP indicates that the numbers are as close as they can be while still being distinct. Smaller ULP differences imply better precision and numerical accuracy. When implementing floating-point operations or algorithms (e.g., the tanhf function), the smallest possible ULP difference is 0.5 indicating that the exact result is correctly rounded to the nearest floating-point number.

==Calculating ULP==
To calculate the ULP of a floating-point number, you can use the following C function:

<syntaxhighlight lang="c">
#include <math.h>

float flt_ulp_size(float x) {
x = fabs(x);

return nextafterf(x, INFINITY) - x;
}
</syntaxhighlight>

This function calculates the ULP size of a given floating-point number <code>x</code>. It first takes the absolute value of <code>x</code> using <code>fabs(x)</code>, then uses <code>nextafterf(x, INFINITY)</code> to find the next representable float value after <code>x</code> in the direction of positive infinity. The difference between these two values gives the ULP size for <code>x</code>.
{| class="wikitable"
|+ULP Example Values 32-bit Float
!Floating-Point Number !! ULP (Macro) !! ULP (Float) !! ULP (Float Hex Representation)
!Description
|-
|1.0 || FLT_EPSILON || 1.1920929e-07 || 0x1.0p-23
|Smallest step near 1.0
|-
|0.0 || FLT_TRUE_MIN || 1.4012985e-45 || 0x1.0p-149
|Smallest positive float
|-
|0.5 || FLT_EPSILON/2 || 5.9604645e-08 || 0x1.0p-24
|Half the step size near 1.0
|-
|10.0 || FLT_EPSILON*8 || 9.536743e-07 || 0x1.0p-20
|Larger step size near 10.0
|}

==Calculating ULP Difference==

=== Simple ULP Difference ===

It is relative hard to correctly calculate the ULP difference between two floating-point numbers. The following C function is a simple way to calculate the ULP difference between a reference value and a given value:

<syntaxhighlight lang="c">
float flt_ulp_diff(float ref, float val) {
return fabs(ref - val) / flt_ulp_size(ref);
}
</syntaxhighlight>

This function simple divides the absolute difference by the ULP. The problem with this approach is which ULP to use. You can use the ULP of the reference value, approximated value or maximum/minimum/average or both values. In the given implementation the ULP of the reference value is used following the idea that the value of an potentially inaccurate approximation should not influence the precision for comparison. However, whatever ULP you choose, it will not be perfect. E.g. in the provided function <code>flt_ulp_diff(0.9999, 2.0)</code> will be around factor 2 greater than <code>flt_ulp_diff(1.0, 2.0)</code> which is not very intuitive.

=== Correct ULP Difference ===

To correctly calculate the ULP difference between two floating-point numbers, you can use the following C function:

<syntaxhighlight lang="c">
uint32_t ulp_intdiff_float(float f1, float f2) {
if (signbit(f1) != signbit(f2)) {
return ulp_intdiff_float(0.0f, fabs(f1)) + ulp_intdiff_float(0.0f, fabs(f2)) + 1;
}

uint32_t i1 = *(uint32_t*)&f1;
uint32_t i2 = *(uint32_t*)&f2;

return i1 > i2 ? i1 - i2 : i2 - i1;
}
</syntaxhighlight>

This function calculates the integer difference in ULP between two floating-point numbers <code>f1</code> and <code>f2</code>. The trick is to reinterpret the floating-point numbers as 32-bit unsigned integers and then calculate the difference between them. The floating-point IEEE 754 representation is designed in such a way that the integer difference between two floating-point numbers (with the same sign) is the ULP difference between them.

To also consider floating-point numbers with different signs, the function calculates the ULP difference towards zero for both numbers and adds the differences together. Additionally, it adds 1 to the result that ULP difference between negative and positive zero (<code>ulp_intdiff_float(-0.0f, 0.0f)</code>) is 1. Adding 1 to the result is not necessary if you do not care about the difference between negative and positive zero.

This implementation satisfies both the properties of triangular additivity (<code>ulp_intdiff_float(x, y) + ulp_intdiff_float(y, z) == ulp_intdiff_float(x, z) for x < y < z</code>) and commutativity (<code>ulp_intdiff_float(x, y) == ulp_intdiff_float(y, x)</code>) of ULP differences, ensuring that the ULP difference between any two floating-point numbers is consistent and order-independent when summed across intermediates.

=== Examples ===
{| class="wikitable"
|+ Example Results for `ulp_intdiff_float` Function
! Input 1 !! Input 2 !! ULP Difference !! Explanation
|-
| 1.0 || 1.0 + FLT_EPSILON || 1 || Adjacent floating-point values
|-
| 1.0 || 1.0 - FLT_EPSILON || 2 || One step away in the opposite direction
|-
| FLT_TRUE_MIN || 0.0 || 1 || Smallest positive float to zero
|-
| -FLT_TRUE_MIN || FLT_TRUE_MIN || 3 || Crossing zero, including sign change
|-
| -1.0 || -1.0 - FLT_EPSILON || 1 || Adjacent floating-point values on the negative side
|-
| -1.0 || -1.0 + FLT_EPSILON || 2 || One step away in the opposite direction on the negative side
|-
| -0.0 || 0.0 || 1 || Difference between signed zero values
|-
| 0.0 || 0.0 || 0 || No difference between identical values
|-
| FLT_MAX || INFINITY || 1 || Transition from the largest finite float to infinity
|-
| -FLT_MAX || -INFINITY || 1 || Transition from the largest negative finite float to negative infinity
|-
| 1.0 || 0.5 || 8388608 || Number of representable floats between 1.0 and 0.5 (2^23)
|-
| 1.0 || 2.0 || 8388608 || Number of representable floats between 1.0 and 2.0 (2^23)
|}

==Calculating Rational ULP Differences==

When comparing a reference value provided as a <code>double</code> and an approximated value as a <code>float</code>, rational ULP differences can provide a finer granularity of comparison. The function below implements this computation by combining integer ULP differences with a fractional component derived from the difference between the <code>double</code> reference value and its <code>float</code> representation.

The following function calculates rational ULP differences:

<syntaxhighlight lang="c">
double ulp_diff(double ref, float b) {
// Handle NaN cases: returns 0 if both are NaN, INFINITY otherwise
if (isnan(b) || isnan(ref)) {
return isnan(b) == isnan(ref) ? 0.0 : INFINITY;
}

// Convert reference value to float for integer ULP difference calculation
float fref = (float)ref;

// Calculate integer ULP difference between the float reference and test value
double iulp = (double) ulp_intdiff_float(fref, b);

// Ensure positive values for further calculations
ref = fabs(ref);
fref = fabsf(fref);
b = fabsf(b);

// Calculate the ULP size of the rounded-down reference value
float ulpRef = flt_ulp_size(double_to_float_rounddown(ref));

// Calculate the fractional difference in ULPs
double diff = (ref - (double)fref);
double fulp = diff / ulpRef;
if (b > ref)
fulp = -fulp;

// Return the sum of integer ULPs and fractional ULPs
return iulp + fulp;
}
</syntaxhighlight>

The function first handles special cases like NaN values, then calculates the integer ULP difference between the float-converted reference value and the approximated value. It then computes the fractional ULP difference based on the difference between the original double reference value and its float representation. We can use the method from the simple ULP difference to calculate the fractional ULP difference. That is working here because the ULP size for the fractional part is clearly defined by the location of the reference value.

One problem is the rounding when converting the double to float. E.g. 1.0 - FLT_EPSILON/4 require the ULP size of the range [0.5, 1.0) but would be converted to 1.0 in a normal double to float conversion. This is why the function uses <code>double_to_float_rounddown</code> to convert the double to float. This function converts the double to float and rounds down to the next smaller float value:

<syntaxhighlight lang="c">
float double_to_float_rounddown(double value) {
float float_value = (float)value;

if (float_value > value) {
return nextafterf(float_value, -INFINITY);
}

return float_value;
}
</syntaxhighlight>

=== Examples ===

The following table provides examples of the results obtained using the <code>ulp_diff</code> function:

{| class="wikitable"
|+ Example Results for <code>ulp_diff</code> Function
! Reference Value !! Test Value !! ULP Difference !!Explanation
|-
|1.0 + FLT_EPSILON/2||1.0||0.5||The test value is halfway between two floats.
|-
|1.0 + FLT_EPSILON/4|| 1.0 ||0.25||The test value is a quarter ULP away.
|-
| 1.0 - FLT_EPSILON/2|| 1.0|| 1.0||The test value is 1 ULP lower.
|-
| 1.0 - FLT_EPSILON/4||1.0||0.5||The test value is halfway to the next float.
|-
|1.0 + FLT_EPSILON/2||1.0 + FLT_EPSILON*10|| 9.5||Integer difference with fractional part.
|-
|1.0 + FLT_EPSILON/4||1.0 + FLT_EPSILON*10||9.75||Includes both integer and fractional differences.
|-
|1.0 - FLT_EPSILON/2||1.0 - FLT_EPSILON*10||19.0||The difference spans multiple ULPs.
|-
|1.0 - FLT_EPSILON/4||1.0 - FLT_EPSILON*10||19.5||Combined fractional and integer ULP difference.
|-
|0.0 + FLT_TRUE_MIN/100||0.0||0.01||A very small fractional ULP difference.
|}

== Accuracy of Mathematical Functions in Math Libraries ==
The following table gives an overview of the ULP errors in widely used math libraries for single-precision floating-point math functions. The ULP error is the largest error in ULP between the exact result and the result of the function. The provided ULP error is exact for math functions with one parameter and the largest known error for two-parameter functions like atan2.
{| class="wikitable sortable"
|+Largest (known) error in ULP for single precision math functions<ref name=":0" />
!library

version
!GNU libc

2.40
!IML

2024.0.2
!AMD

4.2
!Newlib

4.4.0
!OpenLibm

0.8.3
!Musl

1.2.5
!Apple

14.5
!LLVM

18.1.8
!MSVC

2022
!FreeBSD

14.1
!ArmPL

24.04
!CUDA

12.2.1
!ROCm

5.7.0
|-
|acos
|0.899
|0.528
|0.897
|0.899
|0.918
|0.918
|0.634
|'''0.500'''
|0.669
|0.918
|1.32
|1.34
|1.47
|-
|acosh
|2.01
|0.501
|0.504
|2.01
|2.01
|2.01
|0.502
|'''0.500'''
|2.89
|2.01
|2.79
|2.18
|0.564
|-
|asin
|0.898
|0.528
|0.781
|0.926
|0.743
|0.743
|0.634
|'''0.500'''
|0.861
|0.743
|2.41
|1.36
|2.54
|-
|asinh
|1.78
|0.527
|0.518
|1.78
|1.78
|1.78
|0.515
|'''0.500'''
|1.99
|1.78
|3.57
|1.78
|0.573
|-
|atan
|0.853
|0.541
|0.501
|0.853
|0.853
|0.853
|0.722
|'''0.500'''
|0.501
|0.853
|2.88
|1.21
|2.10
|-
|atanh
|1.73
|0.507
|0.547
|1.73
|1.73
|1.73
|0.511
|'''0.500'''
|2.35
|1.73
|3.09
|3.16
|0.574
|-
|cbrt
|0.969
|0.520
|0.548
|3.56
|'''0.500'''
|'''0.500'''
|'''0.500'''
|
|1.83
|'''0.500'''
|1.53
|1.17
|1.14
|-
|cos
|0.561
|0.548
|0.729
|2.91
|0.501
|0.501
|0.846
|'''0.500'''
|0.530
|0.501
|0.561
|1.52
|1.61
|-
|cosh
|1.89
|0.506
|1.03
|2.51
|1.36
|1.03
|0.579
|'''0.500'''
|'''0.500'''
|1.36
|1.89
|2.34
|0.567
|-
|erf
|0.968
|0.780
|0.531
|0.968
|0.943
|0.968
|0.501
|'''0.500'''
|3.99
|0.890
|1.93
|1.04
|1.51
|-
|erfc
|3.13
|0.934
|
|63.9
|3.17
|3.13
|'''0.750'''
|
|6.66
|3.18
|1.64
|4.49
|3.33
|-
|exp
|0.502
|0.506
|0.501
|0.911
|0.911
|0.502
|0.514
|'''0.500'''
|0.501
|0.911
|0.502
|1.94
|1.00
|-
|exp10
|0.502
|0.507
|0.501
|1.06
|
|3.88
|0.514
|'''0.500'''
|
|
|
|2.07
|1.00
|-
|exp2
|0.502
|0.519
|0.501
|1.02
|0.501
|0.502
|0.514
|'''0.500'''
|2.14
|0.501
|0.502
|2.39
|0.871
|-
|expm1
|0.813
|0.544
|0.536
|0.813
|0.813
|0.813
|0.687
|'''0.500'''
|3.02
|0.813
|1.51
|1.45
|1.45
|-
|j0
|9.00
|'''0.678'''
|
|6.18e6
|3.66e6
|3.66e6
|
|
|
|3.66e6
|
|3.78e10
|7.60e7
|-
|j1
|9.00
|'''1.69'''
|
|1.68e7
|2.25e6
|2.25e6
|
|
|
|2.25e6
|
|7.48e9
|7.53e7
|-
|lgamma
|6.78
|0.510
|
|7.50e6
|7.50e6
|7.50e6
|'''0.501'''
|
|2.92e5
|7.50e6
|
|1.35e7
|7.50e6
|-
|log
|0.818
|0.519
|0.577
|0.888
|0.888
|0.818
|0.511
|'''0.500'''
|0.562
|0.888
|0.818
|0.865
|1.89
|-
|log10
|2.07
|0.516
|1.40
|2.10
|0.832
|0.832
|0.502
|'''0.500'''
|0.626
|0.832
|0.82
|2.09
|1.71
|-
|log1p
|1.30
|0.525
|0.501
|1.30
|0.839
|0.835
|0.513
|'''0.500'''
|1.44
|0.839
|2.02
|0.887
|0.579
|-
|log2
|0.752
|0.508
|0.766
|1.65
|0.865
|0.752
|0.502
|'''0.500'''
|2.04
|0.865
|0.752
|0.919
|1.00
|-
|sin
|0.561
|0.546
|0.530
|1.37
|0.501
|0.501
|0.846
|'''0.500'''
|0.530
|0.501
|0.561
|1.50
|1.61
|-
|sinh
|1.89
|0.538
|'''0.500'''
|2.51
|1.83
|1.83
|0.579
|'''0.500'''
|0.501
|1.83
|2.26
|2.94
|0.922
|-
|sqrt
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|'''0.500'''
|
|'''0.500'''
|'''0.500'''
|-
|tan
|1.48
|0.520
|0.509
|3.48
|0.800
|0.800
|0.746
|'''0.500'''
|0.502
|0.800
|3.30
|3.10
|2.33
|-
|tanh
|2.19
|0.514
|'''0.500'''
|2.19
|2.19
|2.19
|0.817
|'''0.500'''
|1.27
|2.19
|2.59
|1.82
|1.41
|-
|tgamma
|7.91
|0.510
|
|239
|'''0.501'''
|'''0.501'''
|'''0.501'''
|
|3.58e5
|'''0.501'''
|
|4.34
|1.68e7
|-
|y0
|8.98
|'''3.40'''
|
|4.84e6
|4.84e6
|4.84e6
|
|
|
|4.84e6
|
|2.36e10
|7.53e7
|-
|y1
|9.00
|'''2.07'''
|
|6.18e6
|4.17e6
|3.66e6
|
|
|
|4.17e6
|
|4.96e10
|9.35e7
|-
|atan2
|1.52
|'''0.550'''
|0.584
|1.52
|1.55
|1.55
|0.722
|
|0.584
|1.55
|2.93
|2.18
|2.01
|-
|atan2pi
|
|'''0.841'''
|
|
|
|
|
|
|
|
|
|
|
|-
|compound
|
|'''0.501'''
|
|
|
|
|
|
|
|
|
|
|
|-
|hypot
|0.501
|0.501
|0.501
|1.21
|1.21
|0.927
|0.501
|'''0.500'''
|0.501
|1.21
|
|1.03
|1.57
|-
|pow
|0.817
|0.515
|1.56
|169.
|0.970
|0.817
|0.515
|'''0.500'''
|0.568
|0.970
|0.817
|2.60
|1.40
|}

== Further Reading ==

* [[wikipedia:Unit in the last place|Unit in the last place]] article on Wikipedia
* [https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=bc07e8d52dbbac5bef9551bbc111b95d82dc4766 On the definition of ulp (x)] <ref>Jean-Michel Muller. On the definition of ulp(x). [Research Report] RR-5504, LIP RR-2005-09, INRIA, LIP. 2005, pp.16. [https://inria.hal.science/inria-00070503 ⟨inria-00070503⟩]</ref>
* [https://members.loria.fr/PZimmermann/papers/accuracy.pdf Accuracy of Mathematical Functions in Single, Double, Double Extended, and Quadruple Precision] <ref name=":0">Brian Gladman, Vincenzo Innocente, John Mather, Paul Zimmermann. Accuracy of Mathematical Functions in Single, Double, Double Extended, and Quadruple Precision. 2024. [https://inria.hal.science/hal-03141101 ⟨hal-03141101v7⟩]</ref>
* [https://www.gnu.org/software/libc/manual/html_node/Errors-in-Math-Functions.html Official glibc documentation about their math function errors]

== References ==
<references />
[[Category:Numerical Precision]]

Numerical Precision in ONNX and AI Inference

2026-02-03T09:16:13Z

Timo.stripf:

Automatic C to Rust Translation

2026-02-03T08:00:59Z

Timo.stripf:

The systems programming community has long grappled with security flaws from memory-unsafe languages like C. Rust offers memory and thread safety guarantees by enforcing strict ownership and lifetime rules at compile time. Migrating legacy C codebases to Rust could eliminate entire classes of vulnerabilities. However, manual rewrites of large C codebases in Rust are labor-intensive and error-prone. This has spurred development of '''automatic C-to-Rust translation''' tools. The goal is to automate conversion of C into Rust code that preserves the original program’s behavior while leveraging Rust’s safety features. In fact, the importance of this challenge is highlighted by DARPA’s ''Translating All C to Rust (TRACTOR)'' program, which explicitly aims to fully automate conversion of C into high-quality, idiomatic Rust code with memory safety guarantees. Achieving this is difficult due to fundamental differences between C and Rust (e.g. manual memory management vs. Rust’s ownership model, unrestricted pointers vs. Rust’s borrow checker). Nonetheless, several tools and research projects are actively tackling automatic C-to-Rust translation using a mix of compiler techniques and, more recently, AI. This article surveys the state-of-the-art tools – focusing on those actively maintained – and compares their approaches, features, fidelity, safety guarantees, and limitations. <ref name="ref1"/>

== Challenges in Translating C to Rust ==
Automatically translating C into Rust is non-trivial because the languages have different paradigms and safety models. '''Pointers and memory management''' are central: C allows arbitrary pointer arithmetic, unchecked array accesses, and manual <code>malloc/free</code> control, whereas Rust requires structured borrowing, prohibits data races, and enforces bounds-checking for safe references. A direct transliteration of C pointers into Rust will typically use Rust’s ''raw pointers'' and <code>unsafe</code> blocks, which forgo Rust’s safety checks. This preserves semantics but yields Rust code that is as unsafe as the original C. The challenge is to infer higher-level safe abstractions (like Rust slices, references, or smart pointers) from low-level C code. Other tricky features include '''C’s unions''', which have no direct safe equivalent in Rust (Rust’s unions are unsafe and lack tagging) and '''setjmp/longjmp''' or goto-based control flow, which do not map cleanly to Rust’s structured control constructs. Because C allows patterns that violate Rust’s safety rules (like aliasing a mutable buffer in multiple places), any automated tool must either (a) leave such code in <code>unsafe</code> blocks, or (b) apply complex analyses or transformations to restructure the code for safety.

Another concern is handling '''undefined behavior (UB)''' in C. C code that invokes UB (like overflowing a signed integer, dereferencing an invalid pointer, or reading uninitialized memory) has no well-defined semantics. A straightforward translation may “inherit” these issues. For example, translating a potentially overflowing C arithmetic operation to Rust might use Rust’s normal arithmetic operator, which will panic on overflow in debug builds (a detectable failure) instead of exhibiting arbitrary behavior. In general, tools either assume the input C is well-defined (no UB), or they insert checks/conversions for certain UB cases, accepting that the Rust output may safely abort or diverge from C’s unpredictable behavior. These challenges require a combination of parsing technology, AST analysis, dataflow/alias analysis, and sometimes dynamic or heuristic methods to produce correct and safe Rust code. <ref name="ref2"/><ref name="ref3"/>

== Semantics-Preserving Transpilers ==
Early and foundational work on C-to-Rust translation has focused on '''semantic preservation''' – generating Rust code that is as close as possible to the original C in behavior and structure. These tools prioritize correctness and completeness of translation, emitting Rust that may not be idiomatic but can be compiled and run to mirror the C program’s output (assuming no UB). Because Rust is a safer language, such translations typically mark the unsafe portions explicitly, resulting in a Rust program that compiles and runs, but largely bypasses Rust’s safety checks by using <code>unsafe</code> constructs.

=== C2Rust ===
'''C2Rust''' is one of the most prominent C-to-Rust transpilers and is actively maintained (primarily by Immunant and Galois). It parses C source using '''Clang''' (supporting C99) and then programmatically generates corresponding Rust code. The design goal is to translate ''most C code into semantically equivalent Rust'' without manual intervention. C2Rust leverages Clang’s type information to produce Rust with matching types and function signatures, ensuring that the translated modules can link with code compiled from the original C (useful for incremental migration). Arbitrary control flow (including <code>goto</code> and complex loops) is handled by using Emscripten’s '''Relooper''' algorithm to structure Rust <code>if/while</code> constructs that emulate the original flow. For example, computed <code>goto</code> or irreducible loops are transformed into state-machine like Rust code with explicit <code>break</code>/<code>continue</code> logic, since Rust has no direct <code>goto</code>. <ref name="ref4"/><ref name="ref5"/>

C2Rust’s output is intentionally close to the C source. Pointers in C become raw pointers in Rust (<code>''mut T</code> or <code>''const T</code>), pointer arithmetic is translated using <code>.offset()</code> or integer address calculations, and manual memory management calls (e.g. <code>malloc</code>, <code>free</code>) remain as calls to libc equivalents in Rust. The emphasis is on '''fidelity''': the Rust code will behave the same as the C code, down to low-level details, provided the C program had defined behavior. As a result, the initial translated Rust is often littered with <code>unsafe</code> blocks and can look “unidiomatic” or overly verbose. For instance, array indexing in C might be translated to pointer arithmetic in Rust (<code>*p.offset(i as isize)</code> rather than a safe slice index) if the translator cannot prove it’s safe to use Rust slices. Indeed, early users noted that C2Rust’s output “just converts C to rather painful Rust” that mirrors pointer manipulations. This is a conscious design choice to avoid changing program semantics. C2Rust does not automatically convert C’s '''structs''' or pointer-heavy data structures into richer Rust types; they remain as struct definitions and pointer fields in Rust, preserving memory layout so that external binary interfaces (ABI) remain compatible. <ref name="ref2"/><ref name="ref6"/>

From an '''implementation''' standpoint, C2Rust originally built Rust ASTs using <code>librustc</code> internals, but it has since been refactored to use the stable <code>syn</code> library for Rust code generation. This decoupling from Rust compiler internals has made C2Rust easier to maintain and install. The tool operates via a command-line that takes a '''compile commands database''' (e.g. from CMake or Bear) and processes each C file into a corresponding Rust file. It supports transpiling large projects module-by-module. Not every C construct is handled (for example, C99 variable-length arrays and certain GCC extensions may not be fully supported), but known unsupported features are explicitly warned or skipped. C2Rust assumes the input is portable C99; it runs the C preprocessor and thus the translation is platform-specific (it will embed definitions as seen on the host platform, which means the Rust output might be specific to the host OS/architecture). <ref name="ref5"/><ref name="ref7"/>

'''Undefined behavior handling:''' C2Rust largely carries over the original logic into Rust <code>unsafe</code> code without adding safety checks, so memory errors in C would still be possible in the Rust output (now confined to <code>unsafe</code> blocks). For certain UB cases that Rust ''cannot'' represent directly, C2Rust has to make a choice. For instance, C allows reading uninitialized values (UB), whereas Rust forbids using an uninitialized variable. C2Rust’s strategy in such cases is typically to initialize variables to a default when necessary or use constructs like <code>std::mem::MaybeUninit</code> to represent them, to ensure the Rust compiles. For integer overflow, as noted, it doesn’t insert any special wrapping: it relies on Rust’s standard overflow semantics (two’s complement wrap in release builds, panic in debug). If the C code relies on overflow behavior, this could introduce a divergence (but since relying on signed overflow is UB in C, C2Rust chooses a reasonable Rust default rather than trying to exactly mimic the undefined). Overall, C2Rust’s priority is that the Rust program '''compiles and reproduces the C program’s functionality''' for all well-defined cases. Memory safety is not improved in this first-stage translation – the output Rust is essentially an “unsafe Rust” codebase. To help evolve this into idiomatic safe Rust, C2Rust was designed with subsequent refactoring in mind. The project provides (or in recent versions, plans to provide) additional tools to gradually refactor the transpiled code into more idiomatic and safe Rust by applying automated transformations. For example, there were <code>c2rust-refactor</code> tools for doing things like renaming, or converting certain pointer patterns to references, though an “exciting new approach” to automate deeper unsafe-to-safe rewrites is under development. <ref name="ref2"/><ref name="ref4"/><ref name="ref7"/>

=== Corrode ===
'''Corrode''' is another early C-to-Rust translator, developed by Jamey Sharp in 2016-2017, written in Haskell. It uses the ''language-c'' Haskell library as a C parser and similarly attempts to emit equivalent Rust code. Corrode was an inspiration for C2Rust – in fact, C2Rust’s authors acknowledge borrowing ideas from Corrode’s design. The overarching philosophy of Corrode is to '''preserve the original program’s behavior, API, and structure as much as possible'''. Like C2Rust, it outputs Rust code intended to act as a drop-in replacement for the C, not changing data representations or logic. For example, Corrode will translate a C global array or pointer arithmetic in a way that keeps the same memory layout and computations in Rust. Its maintainers explicitly note that if the input program is free of undefined and implementation-defined behavior, the Rust output should behave ''exactly'' the same. In cases of C undefined behavior, Corrode makes a best-effort guess at a sensible Rust translation. A concrete example given in Corrode’s documentation is how it handles '''overflowing arithmetic''': in C, signed overflow is UB, but Corrode translates it to Rust’s <code>+</code> operator on integers, which will yield panics in debug mode (or two’s-complement wrapping in release mode). This choice means the Rust program might crash in debug builds when the C would have overflowed silently, but it avoids introducing silent wrong results—favoring a predictable failure over undefined behavior. <ref name="ref2"/><ref name="ref5"/>

Corrode’s output, much like C2Rust’s, is initially ''unsafe Rust''. It uses raw pointers and <code>unsafe</code> for operations that Rust cannot verify. However, Corrode did make some attempts at slightly more idiomatic constructs in simple cases. For instance, Corrode can recognize certain C loop patterns. A common pattern is a <code>for</code> loop incrementing an index; Corrode will translate a C <code>for(int i=0; i<i_max; ++i)</code> into a Rust <code>for i in 0..i_max</code> range loop. This is possible when the loop fits a clear Rust iterator pattern. Such transformations improve readability without altering semantics. Another example is converting some uses of C’s <code>NULL</code> to Rust’s <code>Option</code> type when context allows (though this is limited). Despite these niceties, Corrode’s general output still contains many unsafe pointers. Like C2Rust, it doesn’t magically introduce Rust’s ownership tracking where none existed in C. Its focus is a correct translation that ''compiles'', leaving deeper safety improvements to be done later (likely by a human). Corrode also strives to maintain C ABI compatibility, meaning one could replace one C file of a project with Corrode’s Rust output and link it with the remaining C. This constrained Corrode to avoid changing data layouts or function signatures. <ref name="ref6"/>

It should be noted that as of late 2010s, Corrode is '''no longer actively developed'''. Its author announced in 2018 that he would deprecate Corrode in favor of C2Rust, acknowledging that C2Rust’s more extensive engineering effort made it a better foundation going forward. Corrode’s last updates were around 2017, and while the repository is still available, it may not be compatible with the latest Rust compilers or handle newer C features. Nonetheless, Corrode is historically important as a pioneering tool, and its design principles (preserve behavior, keep code maintainable, allow gradual adoption) have influenced subsequent projects. <ref name="ref8"/>

== Safety-Oriented Translation Tools ==
While direct transpilers like C2Rust produce a Rust version of the C code, they do not immediately yield the ''safe'', idiomatic Rust that human developers would write. Recognizing this, a number of tools and research efforts have focused on **translating C to ''safer'' Rust**, going beyond rote syntax conversion to attempt enforcement of Rust’s safety guarantees. These tools typically start from code that is equivalent to the C (often leveraging output from C2Rust or a similar baseline), and then apply analyses or transformations to refactor <code>unsafe</code> constructs into safe ones (like replacing raw pointers with references, or introducing Rust types that eliminate unchecked operations). This process is challenging because it requires reasoning about aliasing and lifetimes in the C code. Below, we discuss some representative safety-oriented tools that are under active development or study.

=== Laertes (Ownership Inference Post-Translation) ===
'''Laertes''' is a tool introduced by Emre et al. (OOPSLA 2021) that aims to '''lift C2Rust-generated code into safer Rust''' by inferring ownership and lifetimes for pointers. The input to Laertes is essentially the output of C2Rust (or an equivalent unsafe Rust translation). Laertes then tries to automatically convert as many raw pointers as possible into Rust references (<code>&</code> or <code>&mut</code>) which the Rust borrow checker can verify. The core algorithm is an '''iterative, compiler-guided inference''': Laertes optimistically assumes all pointers can be safely transformed into references and assigns them provisional lifetime parameters. It then attempts to compile the program. Where the Rust compiler throws errors (due to violations of borrowing rules or type mismatches), Laertes analyzes the error and “backs off” that assumption for the offending pointers. For example, if two pointers were actually aliasing in a way that Rust’s rules disallow for references, the compiler would complain, and Laertes would revert those pointers to raw pointers (unsafe) in the next iteration. Under the hood, Laertes uses a combination of pointer analyses (Steensgaard’s inclusion-based analysis and Andersen’s subset-based points-to analysis) to propagate constraints and determine which pointers must remain unsafe. It effectively treats the Rust compiler’s borrow checker as an oracle: any pointer that can be converted to a reference without compiler errors is assumed to be safe to do so. <ref name="ref9"/>

Laertes can automatically introduce lifetime annotations in function signatures and struct definitions so that converted references have proper lifetimes. It also distinguishes owning pointers vs. borrowing pointers as part of its inference (e.g., a pointer that is only ever freed in one place might be turned into a Rust <code>Box</code> or an owned value). However, Laertes is '''conservative''' in what it attempts. It completely ignores any pointer involved in patterns that inherently require unsafe code, such as pointer arithmetic, union use, or casting to/from integers. Only pointers that are “unsafe solely because C lacks explicit lifetime/ownership” are candidates. In their study, this turned out to be a relatively small subset – about 11% of raw pointers on average could be handled by Laertes’s approach. The rest still needed to stay as raw pointers due to other complicating factors (like global variables, complex aliasing, etc.). In other words, Laertes could make a portion of the C2Rust output safe, but most of the code remained untouched. Subsequent research (Emre et al., OOPSLA 2023) examined the limits of this approach, finding that even if one pre-processes the code to remove other causes of unsafety, inferring correct lifetimes and ownership for all pointers is extremely difficult. Nonetheless, Laertes demonstrated that an automated tool can ''safely convert a non-trivial fraction of raw pointers to references'', eliminating those memory-unsafe pieces from the code. The advantage of Laertes’ method is that it '''guarantees preservation of behavior''' (it only changes pointers where the Rust compiler confirms no borrowing rule is violated) – so any pointer it converts to a reference is, by construction, not involved in unsafe aliasing. This means the translated program remains functionally equivalent to the original C (assuming the original had no UB) but now enjoys some enforced safety. The limitation is simply that many pointers cannot be transformed with this approach, so manual effort or more powerful analyses are needed for those. <ref name="ref9"/>

=== Crown (Static Ownership Analysis) ===
'''Crown''' (Zhang et al., 2023) is another tool that automates conversion of C pointers to Rust references, but it takes a different approach from Laertes. Instead of relying on trial-and-error with the compiler, Crown uses a more '''static, ownership-oriented analysis''' to determine which pointers can be safely replaced by references. According to its description, Crown performs a whole-program ownership inference that can handle a larger set of pointers than Laertes, which was restricted by needing a clean compile for each guess. Essentially, Crown analyzes the C code (or the unsafe Rust) to identify data that have a single owner or no concurrent aliases, allowing those to become <code>Box</code> (owned heap allocations) or <code>&mut</code> (unique mutable references) in Rust. By tracking the flow of pointers through assignments and function calls, it can sometimes determine, for example, that a pointer is only ever used in one context and is not aliased elsewhere, qualifying it for safe transformation. The Crown paper notes that it “employs ownership analysis to facilitate the replacement of a larger number of pointers compared to Laertes”. In contrast to Laertes’s iterative feedback loop, Crown’s approach is more direct: it computes a model of which pointers are exclusive, which are shared read-only, etc., and then rewrites the code accordingly in one go. <ref name="ref10"/>

Crown is able to convert not only simple function-local pointers but also more complex cases, thanks to its analysis of how pointers are passed and returned (ensuring the callee doesn’t stash a reference in a global or somewhere escaping the caller’s scope, for instance). Any pointer conversions Crown performs are vetted by compiling the resulting program – the output must still pass Rust’s borrow checker, meaning Crown’s static analysis must be sound. If Crown is too optimistic, the Rust code won’t compile, so presumably the implementation either avoids unsound replacements or would need to catch errors and adjust (the details in literature suggest Crown was successful on their benchmarks). In addition, Crown addresses some issues outside the scope of Laertes. For example, ''Concrat'' (Hong and Ryu, 2023) is a complementary tool cited alongside Crown that specifically targets replacing certain C library calls with safer Rust equivalents (such as replacing C’s <code>pthread_mutex</code> and lock functions with Rust’s <code>std::sync::Mutex</code> API). This indicates the broader effort to not only transform pointers but also other unsafe patterns. Crown and related techniques are currently research prototypes and are not as plug-and-play as C2Rust. They often require the code to be translated by C2Rust first, then a separate phase to apply the safe refactor. Nevertheless, Crown represents the direction of using deeper program analyses to push the boundary of what can be automatically made safe, moving closer to the TRACTOR program’s goal of human-quality Rust output. By using static ownership reasoning, Crown can handle cases that would stump a purely local or trial-based approach – for instance, inferring that a struct and all its pointers are encapsulated in one module and can be given lifetimes that make the whole module safe. <ref name="ref10"/>

=== CRustS (Semantics-Relaxed Source Rewriting) ===
'''CRustS''' (Ling et al., 2022) takes yet another approach: it applies a series of '''source-to-source transformation rules''' to the C2Rust output (unsafe Rust) with the aim of significantly increasing the proportion of code that is safe, even if it means slightly '''relaxing the requirement of full semantic preservation'''. The philosophy here is that ''if'' we allow minor changes to program behavior (preferably in cases that don’t affect intended functionality), we can achieve much safer Rust code automatically. CRustS is implemented using TXL (a source transformation language) and defines 220 rewrite rules that pattern-match on Rust code and transform it. Of these, 198 rules are strictly semantics-preserving (similar to what Corrode or Laertes would do – only reorganizing code without changing its meaning), and 22 are ''semantics-approximating''. The approximating transformations deliberately make trade-offs that might alter corner-case behavior but remove <code>unsafe</code> constructs. For example, one approximating rule might convert a raw pointer used for array access into a safe slice, filling in a default value for out-of-bounds indices rather than exactly mimicking a buffer overrun (which is UB anyway). Another might replace an unsafe union with an enum that covers common cases but might not preserve exact bit-level behavior for unsupported union variants. The idea is to eliminate as many unsafety causes as possible, then let the Rust compiler’s checks and tests ensure the program still behaves correctly for expected inputs. <ref name="ref11"/>

CRustS reports impressive improvements in safety: applying these rules to C2Rust-translated code yielded a '''significantly higher ratio of safe code''', even reaching function-level safe code ratios comparable to idiomatic Rust projects. In comparison to Laertes, which could only handle a small subset of pointers, CRustS claims a much higher conversion rate (their paper notes that on Laertes’s own benchmarks, CRustS achieved a higher safe-pointer ratio). The cost, of course, is that some transformations are not guaranteed semantics-preserving in all cases. For instance, CRustS might assume that certain global variables can be made immutable or that certain pointer casts are never meant to alias incompatible types – assumptions that a human would validate but the tool can only guess. Because of this, CRustS is presented as a “demo” or research prototype – it’s a proof that by bending the rules, one can get far more safe code automatically. In practice, one would likely use CRustS’s output as a starting point and then run test suites or formal verification to ensure nothing critical broke. The CRustS pipeline is essentially: run C2Rust to get an unsafe Rust version, then run CRustS (which is a separate tool) to refactor that code. Installation instructions show it builds on top of C2Rust’s output. This two-phase approach underscores that CRustS doesn’t parse C directly; it relies on Rust code input. This is reasonable since TXL rules are applied to the Rust abstract syntax tree or source. <ref name="ref11"/>

'''Examples of CRustS transformations''' (deduced from its description and related work) likely include: converting pointer arithmetic into indexed slice accesses when possible (with bounds checks), turning C-style string manipulation with <code>char''</code> into usage of Rust <code>String</code> or <code>&str</code> where lengths are known, and simplifying macro-expanded code. It may also introduce safe wrappers: e.g., if a C function returns a pointer that must be freed by the caller, CRustS could change it to return a <code>Box<T></code> (owned pointer) in Rust, thus transferring ownership and making the deallocation deterministic (this might be a semantics-preserving change if done carefully, or semantics-relaxed if the C code sometimes leaked or reused that pointer in unusual ways). The end result of CRustS is a Rust codebase that has far fewer <code>unsafe</code> blocks. Its authors measured that a majority of functions became '''completely safe Rust''' after transformation (95% of functions in their sample had no unsafe code inside, up from a much smaller percentage originally). This is a huge step towards the goal of automatic full migration. However, a caveat noted by other researchers is that CRustS’s safety might be ''shallow* – e.g., making each function body safe doesn’t guarantee the entire program is free of logical memory errors if some unsafe operations were hidden behind foreign calls or relaxed rules. In any case, CRustS is a valuable approach exploring the frontier between strict semantic equivalence and practical safety gains. <ref name="ref9"/>

=== Other Notable Efforts ===
Beyond the tools above, there are other specialized efforts. ''Concrat'' (2023) was mentioned earlier – it focuses on replacing C concurrency primitives with Rust’s standard library equivalents using dataflow analysis. This addresses a niche (but important) aspect: translating C’s pthreads usage into Rust’s thread and mutex types safely. Another area of focus has been C’s '''union types'''. One research work titled “To Tag or Not to Tag” (2023) examines how to automatically translate C unions into Rust in a safer way by introducing tagged enums or on-demand initialization of union fields. Unions are tricky because a C union allows reinterpretation of memory in incompatible ways. Tools like C2Rust leave unions as Rust <code>union</code> (which still requires unsafe usage when accessing), but experimental transforms can wrap union accesses in safe enums or generate accessors that maintain a tag to track the active union variant. These kinds of targeted solutions could eventually be composed into larger translation pipelines. <ref name="ref10"/>

== LLM-Assisted Translation Approaches ==
In recent years, the emergence of powerful '''Large Language Models (LLMs)''' for code (such as OpenAI’s Codex/GPT or similar) has opened a new frontier for automatic code translation, including C to Rust. LLMs can be prompted with C code and asked to produce Rust code, and often they will generate more ''idiomatic'' Rust by drawing on high-level patterns rather than doing a one-to-one translation. For example, an AI might translate a C buffer + length pair into a Rust slice, or use high-level library functions in Rust to replace low-level C loops. This is promising in terms of producing human-like Rust code, but LLM-based translation has its own challenges. Chief among them: the LLM might produce code that '''looks''' plausible but is semantically incorrect or fails to compile. Unlike a deterministic compiler-based tool, an LLM might miss subtle aspects of the C semantics, especially for larger functions or when complex pointer manipulation is involved. Additionally, LLMs have context length limits, making it hard to directly feed large codebases.

To harness the strengths of both worlds, some projects combine traditional analysis with LLMs – a ''neuro-symbolic'' approach. '''C2SaferRust''' (Nitin et al., 2023) is one such approach that uses C2Rust ''and then'' uses an LLM to improve the safety of the code. In C2SaferRust’s pipeline, first C2Rust generates the baseline unsafe Rust. Then the code is automatically sliced into small, independent chunks that an LLM (GPT-4 in their case) can handle. Each chunk (for example, a single function or a group of related functions) is given to the LLM with instructions to produce a safer Rust version of that code. The LLM can thereby suggest, say, using a Rust <code>Vec<T></code> instead of a raw array pointer + length, or using <code>Option</code> for possibly-null returns, etc., effectively performing a context-aware refactoring. After getting the LLM’s suggestions for all parts, C2SaferRust reassembles the program and runs its test suite to verify correctness. This last step is crucial – it compensates for the fact that the LLM might introduce mistakes. If a test fails, the process could be iterated or the problematic section flagged for manual review. On a benchmark of real C projects, C2SaferRust was able to automatically reduce the number of raw pointer usages by up to 38% and overall <code>unsafe</code> code by up to 28%, ''while all tests still passed''. This demonstrates measurable progress: the resulting code is closer to idiomatic Rust (significantly fewer unsafe blocks), and it still behaves correctly on the tested functionality. C2SaferRust also outperformed some prior techniques like Laertes on those benchmarks, showing the benefit of letting an LLM propose creative refactorings that pure static tools might not attempt. <ref name="ref12"/>

Other LLM-based tools include '''Flourine''' and '''Vert''' (both discussed in a 2024 study). Flourine uses a ''test-driven repair'' strategy: it generates two versions of Rust code for a given C input – one via a safe-but-idiomatic guess from an LLM, and one via a more literal translation (e.g., through an intermediate WebAssembly representation to ensure correctness) – then it fuzz-tests the outputs to find behavioral differences and tries to automatically fix any discrepancies. The idea is to combine the reliability of a baseline translation with the elegance of an LLM translation. Vert, on the other hand, emphasizes breaking the input program into ''vertically sliced components'' that an LLM can handle piecewise, attempting to ensure that dependencies are managed when merging the LLM-generated parts. Despite these sophisticated strategies, the results so far show that '''LLM-based translation still struggles with large or complex code'''. One report noted that less than 20% of C programs over 150 lines could be satisfactorily translated by an LLM-based method without manual intervention. The necessity to split programs into small chunks is a major hurdle – if a function <code>f</code> calls <code>g</code>, the LLM ideally needs to see both to produce consistent Rust, otherwise it might generate conflicting versions of <code>g</code> when given different contexts. Indeed, experiments with Flourine and Vert found that when functions were translated independently, it led to inconsistencies that had to be reconciled, and translating whole programs in one prompt often exceeded LLM capacity or produced non-compiling output. <ref name="ref3"/>

That said, LLMs have shown they can handle many ''syntactic'' or even somewhat semantic conversions gracefully on smaller snippets. They excel at producing idiomatic constructs: for example, using Rust’s <code>Iterator</code> APIs instead of C-style loops, or leveraging pattern matching and <code>Result</code> types for error handling. These high-level refactorings are something that pure static tools won’t do unless explicitly programmed for each pattern. As LLMs improve and as techniques like guided prompting, automated verification, and iterative refinement advance, we can expect LLM-assisted translation to become more reliable. It’s likely that the future of automatic C to Rust translation will involve a hybrid: using static analysis to break down and formally understand the source, and using LLMs or other AI to suggest the more complex transformations that require semantic understanding or creative restructuring, all under the validation of tests or formal checks. <ref name="ref1"/>

== Comparative Analysis of Tools ==
The landscape of C-to-Rust translators can be compared along several axes: '''fidelity vs. idiomaticity''', '''safety guarantees''', '''automation scope''', and '''maturity'''.

* '''Translation Fidelity:''' Tools like ''C2Rust'' and ''Corrode'' prioritize fidelity. They retain the exact behavior and even the structure of the C code in the generated Rust (making only minimal adjustments needed for Rust syntax). This means they have very high success in translating arbitrary C code (including low-level tricks) into ''compilable'' Rust. However, the Rust they produce is often verbose and non-idiomatic – essentially a C program expressed in Rust syntax. At the other end, LLM-based approaches and aggressive refactoring tools sacrifice some fidelity for readability and safety. For instance, an AI might refactor a C loop into a Rust iterator, which is more idiomatic but could conceivably alter performance characteristics or behavior in edge cases. Tools like Laertes and Crown try to walk the line: they keep the overall structure from C2Rust, but on the subset of code they can handle, they introduce Rust references (which is a semantic change – e.g., if in C two aliased pointers were written through, translating both to <code>&mut</code> would be incorrect and is avoided). These tools ensure fidelity by ''only'' making changes that Rust’s compiler or analyses can guarantee are safe (thus presumed semantically valid). <ref name="ref2"/><ref name="ref6"/>

* '''Safety and Memory Guarantees:''' The primary benefit of translating C to Rust is potential memory safety, but not all tools achieve this equally. C2Rust’s direct output provides '''no additional memory safety''' beyond the original C – it compiles under <code>unsafe</code> and the programmer must manually inspect or refactor to get safety. Corrode is similar in this regard. In contrast, Laertes and Crown explicitly aim to ''increase'' memory safety by converting pointers to safe references wherever possible. The result is a partially safe program: some parts are protected by Rust’s compile-time checks (no data races or use-after-frees on those portions), while other parts remain <code>unsafe</code>. CRustS pushes further, attempting to maximize the safe portion by even altering some semantics if necessary. Its output can be predominantly safe Rust code (with runtime checks like bounds checking in place of what were raw pointer operations) – a big step in guarantees, although a developer may need to verify that the semantics deviations are acceptable. LLM-based translations often ''target'' completely safe Rust (no <code>unsafe</code> at all), because they try to use high-level Rust features. When they succeed, the code has all of Rust’s guarantees (e.g., any out-of-bounds access would be caught as a panic, not a memory violation). However, the caveat is that if an LLM is unsure how to handle some low-level operation, it might omit necessary functionality or mis-handle it, leading to an incomplete translation that might not even compile. In a controlled setting with tests, the hope is to get the best of both: safe Rust that passes all the same tests as the C. Notably, when Rust translations enforce safety, they inherently change how '''formerly undefined behaviors''' manifest – rather than silent memory corruption, you might get a Rust panic or error for things like buffer overflow or null dereference. This is usually considered a desirable outcome (failing fast instead of continuing with corruption). <ref name="ref2"/><ref name="ref3"/><ref name="ref9"/><ref name="ref10"/><ref name="ref11"/>

* '''Idiomatic Quality and Maintainability:''' From a maintainability standpoint, the closer the output is to how a human would have written it, the better. On this front, basic transpilers score low – they tend to produce code that, while correct, is hard for Rust programmers to work with or modify. Variables might have awkward names or extra casts, and everything is wrapped in unsafe. Tools that do even modest refactoring help a lot: e.g., Corrode’s loop translation or C2Rust’s upcoming refactoring successor can turn clunky patterns into cleaner Rust forms. The most idiomatic results currently come from '''AI-based translations''' or very high-level refactor rules. For example, an LLM might recognize a C idiom for error handling (set an integer status and jump to cleanup) and translate it into Rust using the <code>?</code> operator and <code>Result</code> type – a very idiomatic construct. Such transformations are beyond the current static tools’ pattern matching (which tends to be local). RustMap (2025) is a research prototype that attempts project-scale migration by analyzing the C code for patterns that can be globally replaced by Rust equivalents, such as turning C macros into Rust <code>const</code> or functions, or identifying groups of functions that operate on a data structure and converting that whole data structure into a Rust struct with methods. This yields more idiomatic designs rather than line-by-line translation. Across the board, there is a trade-off: the more idiomatic the Rust, the more the tool must '''understand the code’s intent''', which is hard to do automatically. Thus, the most idiomatic translations may require AI and carry a risk of misinterpretation. <ref name="ref6"/><ref name="ref13"/>

* '''Tool Maturity and Ecosystem:''' C2Rust is fairly mature and has been used on real-world projects; it is open-source (BSD-licensed), with an active community (over 4k stars on GitHub) and recent releases (as of 2025). It supports integration into build systems via compile command databases, making it practical for large codebases. Corrode, conversely, is discontinued, and while one can still use it on small code, it may not handle newer C or Rust versions well. The safety-oriented tools like Laertes and CRustS are research prototypes – they have published artifacts or code (Laertes’s artifact and dataset were released for evaluation, and CRustS is available as a Cargo package for demo purposes), but they are not one-click solutions and may require expertise to apply. We can consider them as early explorations likely to be incorporated into future pipelines. LLM-based tools currently are mostly in academic or experimental stages; they are not broadly available as off-the-shelf products, partly because of the cost and complexity (LLM APIs, managing prompts, etc.). However, one can already experiment with LLMs via services (for instance, there are online converters using GPT-4 that attempt C to Rust conversion, though their reliability varies). The DARPA TRACTOR program is likely to spawn more integrated tools in the next few years that combine these techniques. <ref name="ref5"/><ref name="ref11"/><ref name="ref14"/>

In summary, '''no single tool yet completely automates a perfect translation of C to fully safe and idiomatic Rust for large codebases'''. C2Rust provides a strong foundation with complete and correct (but unsafe) translations. On top of that foundation, tools like Laertes, Crown, and CRustS layer increasingly powerful analyses to convert chunks to safe code. Meanwhile, AI-driven approaches offer glimpses of near-human-like translations, excelling in small scopes but struggling to scale. The current best practice for migrating C to Rust might be to use a transpiler (like C2Rust) to do the heavy lifting of rote translation, and then incrementally apply automated refactors (where possible) and manual improvements for critical sections – essentially a human-in-the-loop approach. The ongoing research and development aim to reduce the amount of human effort needed by making the automated steps smarter and more comprehensive.

== Handling of Undefined Behavior and Edge Cases ==
A critical aspect of translation is how tools deal with '''undefined behavior (UB)''' and low-level C quirks. As mentioned, most transpilers assume that the C code is ''reasonably well-behaved''; they don’t attempt to make an incorrect C program safer or correct. If the C code has latent bugs (like buffer overflows), the translated Rust (with <code>unsafe</code> code) will have the same bug. However, when translation tools attempt to produce ''safe'' Rust, they inherently must address UB because safe Rust cannot express certain dangerous operations without a check. One straightforward example is '''buffer overflows''': C might let you increment a pointer past the end of an array (UB if dereferenced). If a tool converts that array and pointer into a Rust slice, any out-of-bounds access will cause a runtime panic in Rust. Thus, the ''behavior'' diverges: the C might have continued (possibly corrupting memory or misbehaving later), whereas the Rust will immediately abort on a bounds check failure. Researchers categorize this as an expected difference – the Rust translation “eliminates potentially unsafe behavior” and replaces it with a controlled failure. In a sense, the Rust version is ''safer by design'': it won’t blindly continue after an out-of-bounds, thereby avoiding exploitation, though it might not match an ill-defined C outcome (there is no meaningful “equivalent” to a wild memory write in safe Rust except termination). <ref name="ref3"/>

Similarly, for '''null pointers''': C can have a null pointer that gets dereferenced (UB). A safe Rust translation might use <code>Option<&T></code> and explicitly handle the null case (e.g., return an error or panic). This changes the control flow (the Rust might error out where the C would crash unpredictably). Tools like Laertes and Crown, when converting pointers to references, must ensure that those pointers were never null – otherwise the Rust code would introduce a new crash. They likely conservatively leave potentially-null pointers as raw <code>''mut T</code> (still unsafe) unless they can prove null cannot happen. CRustS’s approach of semantics relaxation might, for instance, initialize pointers that are observed to be null to a dummy object to avoid crashes – but that could mask a bug. The trade-off is complex: do we preserve a bug or guard against it? Most automatic tools lean toward ''guarding*, on the principle that avoiding UB is the goal (especially since the exact UB behavior can’t be preserved anyway).

'''Integer behaviors''' are another subtle area. C’s integer types can overflow silently (except when using compiler sanitizers). Rust’s safe arithmetic will panic on overflow in debug builds and wrap in release. Neither is “the same” as C’s UB (which could do anything, but on typical two’s complement hardware effectively wraps). Tools generally choose Rust’s native arithmetic, which means a translation is effectively giving defined semantics to what was undefined. Empirical studies using fuzzing have shown that Rust translations often exhibit different behavior from C when it comes to things like I/O or arithmetic edge cases, precisely because Rust checks things C doesn’t. For example, if a C program reads an invalid UTF-8 byte sequence into a <code>char''</code> and later prints it, in C this might just output garbled text or pass through bytes. A direct Rust translation using <code>std::string::String</code> might refuse to encode those bytes (Rust <code>String</code> requires UTF-8), causing a runtime error or loss of data. One study noted differences in how tools handle I/O encoding – some Rust translations assume UTF-8 and thus diverge when given non-UTF-8 input. These are not ''memory safety* issues per se, but they show how higher-level assumptions can creep in. <ref name="ref3"/>

'''Setjmp/longjmp''' and other non-local jumps are typically not supported by Rust at all. C2Rust and others usually cannot translate a <code>longjmp</code> in any straightforward way; they might either reject such code or transform it into a Rust <code>panic!</code>/<code>catch_unwind</code> (which is not exactly the same, and is unsafe FFI with C). Similarly, inline assembly in C is often left as inline assembly in Rust (which is unstable and <code>unsafe</code>), or the tool will bail out if it can’t handle it.

In summary, automatic translators try to stay correct on defined behavior and ''usually'' either skip or embed as unsafe anything truly undefined. When aiming for safety, they effectively turn some UB into Rust runtime errors or compiler errors. This is generally acceptable since one goal is to use Rust’s safety to catch those issues. The key is that when evaluating these tools, one should not expect bit-for-bit identical behavior in scenarios that were undefined in C; rather, we expect that if the C program was intended to be correct, the Rust will be correct on the same inputs. When UB is unavoidable, the Rust will likely either refuse to compile (requiring human attention) or will contain safety abstractions that prevent the UB (possibly changing the program’s failure mode). Tools under active development are increasingly looking to integrate '''dynamic analysis''' (like running test cases or fuzzing) as part of the translation workflow to detect behavioral deviations caused by these issues and correct them. <ref name="ref3"/>

== Conclusion and Future Outlook ==
Automatic C-to-Rust translation has evolved from simple transpilers that emit line-by-line unsafe Rust into a multi-faceted field combining compiler theory, program analysis, and machine learning. Currently, if one needs to port a C project to Rust, a typical pipeline might involve using C2Rust to get an initial Rust codebase that is functionally correct but rife with <code>unsafe</code>, then employing a mix of automated refactoring tools (like the experimental Laertes or CRustS) and manual rewrites to improve safety and idiomatic style. This incremental approach is necessary because fully automated translation ''at the level of a skilled human programmer'' is still an open problem. The active research (funded by initiatives like DARPA’s TRACTOR) indicates optimism that the gap can be closed. Future translators will likely integrate the static and dynamic techniques: for example, a tool could perform sophisticated alias analysis to convert most pointers to safe references (like Crown), use an LLM to rewrite low-level C patterns into high-level Rust (like using iterators or Rust libraries), and then validate the result with exhaustive testing or model checking. Each of those components is being prototyped today. <ref name="ref1"/>

One can imagine a near-future tool that, given a C codebase, produces a Rust crate where memory safety issues are largely fixed, perhaps with annotations or reports for the developer for any remaining <code>unsafe</code> bits that could not be resolved automatically. The developer can then focus their effort only on the truly tricky parts (for example, a complex bit-casting union or an exotic macro). In essence, the goal is to handle the “mundane 90%” of the translation automatically and safely, and leave the 10% of corner cases for human expertise – or further learning by the tool as these corner patterns repeat across projects.

It’s also worth noting that the presence of a correct automatic translation tool could influence how people write C (knowing it will be machine-converted) or how they write Rust (perhaps in a way that is easier to generate from C). Such co-evolution has precedent (as seen with automatic refactoring tools influencing coding standards).

In conclusion, automatic C to Rust translation has made significant strides: '''C2Rust''' provides a reliable base for transpiling C99 code to Rust, '''Corrode''' demonstrated the feasibility and influenced later work, and newer tools like '''Laertes''', '''Crown''', and '''CRustS''' show that a sizable portion of <code>unsafe</code> can be eliminated through clever analysis. Meanwhile, '''Flourine''', '''Vert''', and '''C2SaferRust''' illustrate the power of AI and fuzzing to make translated code more idiomatic and safe. The field is moving quickly, and collaborations between academia and industry (such as the open-source efforts on C2Rust and the research from multiple universities) are bringing us closer to the day where legacy C code can be automatically “uplifted” into safe Rust. That promises not only to reduce bugs but also to extend the life of critical software by modernizing it for today’s secure development standards. Each tool today contributes a piece of the puzzle – from preserving exact semantics to inferring lifetimes or suggesting Rust-specific idioms – and combined, they are paving the way toward fully automated, trustworthy translation from C to Rust. <ref name="ref3"/><ref name="ref4"/><ref name="ref5"/><ref name="ref9"/><ref name="ref11"/><ref name="ref12"/>

'''References:''' The content above cites information from official tool documentation, academic papers, and reputable sources, including the C2Rust project site, the Corrode GitHub README, research papers on Laertes and related techniques, the CRustS demonstration summary, and studies on LLM-based translation effectiveness. These references provide further details for readers interested in the technical specifics of each tool and approach. <ref name="ref2"/><ref name="ref3"/><ref name="ref4"/><ref name="ref9"/><ref name="ref10"/><ref name="ref11"/>

== References ==

<references>
<ref name="ref1">Translating All C to Rust — darpa.mil — https://www.darpa.mil/program/translating-all-c-to-rust</ref>
<ref name="ref2">GitHub - jameysharp/corrode: C to Rust translator — github.com — https://github.com/jameysharp/corrode</ref>
<ref name="ref3">https://www.ndss-symposium.org/wp-content/uploads/2025-1407-paper.pdf — ndss-symposium.org — https://www.ndss-symposium.org/wp-content/uploads/2025-1407-paper.pdf</ref>
<ref name="ref4">C2Rust Demonstration — c2rust.com — https://c2rust.com/</ref>
<ref name="ref5">GitHub - immunant/c2rust: Migrate C code to Rust — github.com — https://github.com/immunant/c2rust</ref>
<ref name="ref6">C to Rust translator - community - The Rust Programming Language Forum — users.rust-lang.org — https://users.rust-lang.org/t/c-to-rust-translator/55381</ref>
<ref name="ref7">C2Rust is Back :: Immunant, Inc — immunant.com — https://immunant.com/blog/2022/06/back/</ref>
<ref name="ref8">c2rust vs Corrode — jamey.thesharps.us — https://jamey.thesharps.us/2018/06/30/c2rust-vs-corrode/</ref>
<ref name="ref9">Aliasing Limits on Translating C to Safe Rust — dl.acm.org — https://dl.acm.org/doi/pdf/10.1145/3586046</ref>
<ref name="ref10">To Tag, or Not to Tag: Translating C’s Unions to Rust’s Tagged Unions — arxiv.org — https://arxiv.org/html/2408.11418v2</ref>
<ref name="ref11">CRustS — Rust application // Lib.rs — lib.rs — https://lib.rs/crates/crusts</ref>
<ref name="ref12">C2SaferRust: Transforming C Projects into Safer Rust with NeuroSymbolic Techniques — arxiv.org — https://arxiv.org/html/2501.14257v1</ref>
<ref name="ref13">RustMap: Towards Project-Scale C-to-Rust Migration via Program Analysis and LLM — arxiv.org — https://arxiv.org/html/2503.17741v1</ref>
<ref name="ref14">Online C to Rust Converter - CodeConvert AI — codeconvert.ai — https://www.codeconvert.ai/c-to-rust-converter</ref>
</references>

Main Page

2026-02-02T21:23:26Z

Timo.stripf: /* Other Topics */

__NOTOC__

Welcome to the emmtrix Technologies Wiki. As a company with a deep-rooted passion for compilers, we specialize in source-to-source compilers designed to analyze, optimize and transform your code. This Wiki aims to offer detailed, technical background information that complements the tools and resources available on our official website. Here, you'll find in-depth explanations, usage guidelines, and insights into the engineering behind our specialized software solutions. Whether you're a developer or a technically-inclined enthusiast, this space is designed to deepen your understanding of what makes our tools essential for your projects.

{| border="0" style="padding: 10px"
| style="vertical-align: top; padding: 10px;" |
=== emmtrix Products ===


* [[emmtrix Dependency Analyzer]]
* [[emmtrix Performance Estimator]]
* [[emmtrix Code Vectorizer]]
* [[emmtrix C++ to C Compiler]]
** [https://online-ecpp2c.emmtrix.com emmtrix C++ to C Compiler Online]
* [[emmtrix C to Rust Compiler]]
* emmtrix Parallel Studio
* Generic information
** [[emmtrix Studio Release Notes]]
** [[:Category:emmtrix Studio FAQ|FAQ]]
* Discontinued product
** [[emmtrix Code Generator]]
** [[emmtrix Model Code Generator]]
* Other Solutions
** [[TC4x PPU Coverage Analysis]]
** [[emmtrix Link Stubber]]
| style="vertical-align: top; padding: 10px;" |
=== Supported Architectures ===
* [[TriCore Instruction Set Architecture]]
** [[Infineon AURIX TC2xx]]
** [[Infineon AURIX TC3xx]]
** [[Infineon AURIX TC4x]], including [[Infineon AURIX TC4x Parallel Processing Unit (PPU)|Parallel Processing Unit (PPU)]]
* ... and many more
|-
|
|
|}

=== Compiler Background ===

* [[Loop Transformations]]
* [[:Category:Code Transformation]]
*[[Demystifying C++]]
*[[The alias Attribute|The alias attribute]]
*[[Clang Diagnostics Overview]]
*[[:Category:Clang Diagnostics]]

=== Other Topics ===
* [[C to Z3 Cheat Sheet]]
* [[Eclipse Xcore Cheat Sheet]]
* [[Logical Execution Time (LET)]]
* [[ULP Difference of Float Numbers]]
* [[Numerical Precision in ONNX and AI Inference]]
* [[Automatic C to Rust Translation]]

Numerical Precision in ONNX and AI Inference

2026-02-01T22:59:38Z

Timo.stripf:

Numerical Precision in ONNX and AI Inference

2026-02-01T22:58:12Z

Timo.stripf: /* 5.2. MatMul, Conv, and Reduction Operators */

'''Open Neural Network Exchange (ONNX)''' is an open standard format for representing machine learning models and neural network computations across different frameworks and hardware<ref name="ref1"/>. As models are exported and deployed via ONNX, the '''numerical precision''' of computations becomes critical. Deep learning inference involves a variety of floating-point operations, and small numerical discrepancies can accumulate and affect model accuracy or reproducibility. This article provides an in-depth overview of floating-point precision issues in numerical computing, with a focus on how they manifest in AI inference and the ONNX ecosystem. We cover floating-point formats and rounding behavior, accumulation strategies for summations, the use of low-precision formats (FP16, BFloat16, FP8) in modern AI, and how ONNX defines (or leaves unspecified) the precision of accumulators in operations. Strategies for maintaining numerical stability in deep neural networks are discussed, along with the trade-offs between computational performance and arithmetic accuracy. The goal is to give engineers and researchers a comprehensive, encyclopedic reference on numerical precision in ONNX-based AI inference systems.

== Floating-Point Precision in Numerical Computing ==

=== 2.1. IEEE 754 Floating-Point Formats ===

Modern computers represent real numbers using the IEEE 754 floating-point standard, which defines formats like 32-bit ''single precision'' (FP32) and 64-bit ''double precision'' (FP64). In IEEE 754 binary formats, a floating-point number is composed of three fields: 1 sign bit, several exponent bits (with a bias), and several fraction (mantissa) bits. For example, FP32 consists of 1 sign bit, 8 exponent bits, and 23 fraction bits (24 bits of significand precision including the implicit leading 1)<ref name="ref2"/>. This provides roughly 7 decimal digits of precision and an exponent range allowing values on the order of <math>10^{\pm38}</math>. FP64 allocates 1 sign bit, 11 exponent bits, and 52 fraction bits (53-bit precision), yielding about 15–16 decimal digits of precision and a vastly larger range (~<math>10^{\pm308}</math>)<ref name="ref2"/>. These larger formats (especially FP64) are the "gold standard" for numerical accuracy in scientific computing, as they reduce round-off error in complex calculations<ref name="ref2"/>. However, the increased precision comes at the cost of doubled storage and typically reduced computational throughput (e.g. many GPUs execute FP64 instructions at a much lower rate than FP32<ref name="ref2"/>).

In addition to single and double precision, the IEEE 754-2008 standard introduced a 16-bit ''half precision'' format (FP16, or binary16). FP16 has 1 sign bit, 5 exponent bits, and 10 fraction bits (11-bit significand including the implicit 1)<ref name="ref3"/>. This format covers a much smaller dynamic range (approximately <math>6.1\times10^{-5}</math> to <math>6.5\times10^4</math> for normalized values<ref name="ref3"/>) and offers only around 3 decimal digits of precision. Half precision was originally used in graphics, but it has gained popularity in machine learning for its speed and memory advantages (discussed later in Section 4). Another format relevant to AI is '''BFloat16''' (Brain Float 16), a 16-bit float with '''8 exponent bits and 7 fraction bits'''<ref name="ref3"/>. BFloat16 sacrifices precision (only ~7 fraction bits ≈ 2–3 decimal digits) in order to have the '''same exponent range as FP32'''<ref name="ref3"/>. This means BFloat16 can represent very large or very small numbers (∼<math>10^{\pm38}</math>, similar range as FP32) but with much less precision between representable values<ref name="ref3"/>. BFloat16 was introduced for deep learning by Google (for TPUs) to allow reduced precision ''training'' without frequent overflow/underflow, relying on the wide range and accepting the coarse precision.

Each floating-point format has a '''unit roundoff''' or machine precision, denoted <math>\epsilon</math>. This is effectively the spacing between 1.0 and the next representable number. For FP32, <math>\epsilon \approx 2^{-23} \approx 1.19\times10^{-7}</math>, corresponding to about 7–8 decimal digits of precision<ref name="ref3"/>. In FP64, <math>\epsilon = 2^{-52} \approx 2.22\times10^{-16}</math> (15–16 decimal digits). Half precision is much less precise: FP16 has <math>\epsilon = 2^{-10} \approx 9.76\times10^{-4}</math><ref name="ref3"/>, and BFloat16 has <math>\epsilon = 2^{-7} = 7.8125\times10^{-3}</math><ref name="ref3"/>. These values quantify how finely the continuum of real numbers is quantized in each format.

=== 2.2. Machine Epsilon and Rounding Error ===

'''Machine epsilon (<math>\varepsilon</math>)''' is formally defined as the smallest positive number such that <math>1.0 + \varepsilon > 1.0</math> in the floating-point format<ref name="ref3"/>. It effectively measures the '''precision''' of the representation. In FP32, <math>\varepsilon = 2^{-23} \approx 1.19\times10^{-7}</math>, meaning any result is rounded to about 7 decimal digits. FP16’s <math>\varepsilon = 2^{-10} \approx 9.77\times10^{-4}</math>, over 1000× larger (so only ~3 decimal digits of precision), and BFloat16’s <math>\varepsilon = 2^{-7} \approx 7.8\times10^{-3}</math>, reflecting its mere ~8-bit precision<ref name="ref3"/>. Rounding in IEEE 754 by default uses '''round-to-nearest (ties to even)''', which ensures the rounding error for any single operation is at most half a unit in the last place (0.5 ULP). We can model a floating-point addition as:

<math>$\text{fl}(a+b) = (a+b)\,(1 + \delta),</math>$

where <math>|\delta| \le \varepsilon</math> for that format<ref name="ref3"/>. Here <math>\varepsilon/2</math> is the maximum '''relative error''' in one rounding (also called unit roundoff). For example, when adding two FP32 numbers, the result is accurate to within about <math>10^{-7}</math> of the true sum (relative), whereas in BF16, the result can deviate by up to <math>\sim7.8\times10^{-3}</math> (nearly 0.78% relative error per operation)<ref name="ref3"/>. Thus, '''each arithmetic operation incurs a small rounding error''' (on the order of machine epsilon) that can propagate or accumulate through a sequence of calculations.

Because floating-point representations have finite exponent range, we must also consider '''overflow''' and '''underflow'''. If a result’s magnitude exceeds the maximum representable value, it overflows to infinity; if it falls below the minimum normal (or minimum subnormal) value, it underflows to zero (or a denormalized tiny value). FP32 and BF16 share an exponent size (8 bits), so their range for normal numbers is roughly <math>[10^{-38}, 10^{38}]</math> (approx <math>~\!3.4\times10^{38}</math> max)<ref name="ref3"/><ref name="ref3"/>. FP16’s 5-bit exponent allows max <math>\sim 6.55\times10^4</math> and min normal <math>\sim 6.10\times10^{-5}</math><ref name="ref3"/>. In practice, '''underflow/overflow''' can occur in low-precision arithmetic if values are not appropriately scaled. For instance, summing a large number of moderately sized terms in FP16 could overflow where FP32 would not. As an example, summing on the order of <math>10^4</math> values of magnitude ~<math>10^1</math> (say, adding 10,000 terms around 10 each) would produce a total <math>\sim10^5</math>, which exceeds FP16’s max finite value ~<math>6.5\times10^4</math> and would overflow to <math>+\infty</math> in half precision<ref name="ref3"/>. Underflow is usually a concern when subtracting nearly equal numbers (catastrophic cancellation yielding a tiny difference that might flush to zero in low precision).

=== 2.3. Accumulation Error in Reductions ===

Floating-point addition is '''not associative''': <math>(a+b)+c</math> can differ from <math>a+(b+c)</math> due to rounding, and thus the order in which we sum a list of numbers can change the result slightly. When summing a large sequence of <math>N</math> numbers <math>x_1, x_2, ..., x_N</math>, the '''accumulated rounding error''' can be significant. In the ''worst case'' (e.g., adding many positive and negative terms that cause maximal cancellation or always rounding one way), the error can grow on the order of <math>O(N \varepsilon)</math> relative to the exact sum<ref name="ref4"/><ref name="ref5"/>. Simply summing <math>N</math> numbers in sequence (the '''naive linear summation''') has a worst-case error bound proportional to <math>N\,\varepsilon</math><ref name="ref4"/>. This worst-case occurs for adversarial arrangements of values; in more typical scenarios with rounding errors of random signs, one can model the error as a random walk, yielding an expected root-mean-square (RMS) error growth on the order of <math>O(\sqrt{N}\,\varepsilon)</math><ref name="ref4"/><ref name="ref5"/>. Still, with very large <math>N</math>, even <math>\sqrt{N}</math> times <math>\varepsilon</math> can become large enough to matter.

'''Cancellation''' is a particular problem: if we add two nearly equal numbers of opposite sign, most leading digits cancel and the result loses significant bits of precision. For example, in single precision, subtracting two numbers that agree in the first 6 decimal digits will yield a result with only ~1 digit of accuracy left. Catastrophic cancellation can dramatically increase relative error. Summation is especially sensitive to cancellation if large positive and negative terms are summed naively. The '''condition number''' of a summation (the ratio of the sum of absolute values <math>\sum|x_i|</math> to the absolute value of the true sum <math>|\sum x_i|</math>) indicates how sensitive the result is to perturbations<ref name="ref5"/>. If the numbers have mixed signs and nearly cancel out (making the condition number large), even an optimal summation order will have a large relative error. For instance, summing [1.0, <math>10^{100}</math>, 1.0, <math>-10^{100}</math>] in double precision has an exact sum of 2.0, but naive summation may yield 0.0 due to cancellation; an improved summation algorithm yields the correct 2.0<ref name="ref4"/><ref name="ref4"/>.

In practical terms, when summing a long list of floating-point numbers (such as accumulating a dot product or reducing a tensor), the '''accumulation error''' can become noticeable if no precautions are taken. Techniques to mitigate this error include carefully choosing the summation order or using higher precision for intermediate sums. We will explore these strategies in the next section.

== Accumulators and Reduction Strategies ==

When performing reductions (sums, dot products, etc.), the method of accumulation can greatly affect the accuracy of the result. Below, we discuss various strategies:

=== 3.1. Naive Linear Accumulation ===

The simplest accumulation strategy is to use a single accumulator and add each element sequentially. In C-like pseudocode:

<syntaxhighlight lang="c">
float sum = 0.0f;
for (int i = 0; i < N; ++i) {
sum += array[i];
}
</syntaxhighlight>

This '''naive linear summation''' adds values one by one to the running total. While straightforward and fast, it is prone to the rounding error accumulation described earlier. The partial sum <code>sum</code> continually grows (in absolute value) as terms are added, so new small terms may fall outside its precision. For example, if <code>sum</code> is large and positive, adding a comparatively tiny positive number may produce no change because the small term's significant digits vanish in the presence of the large sum (the increment is below the rounding threshold of <code>sum</code>). Conversely, adding a small number of opposite sign can suffer from cancellation. The order of addition matters: summing from smallest magnitude to largest (ascending order) tends to yield a more accurate result than summing in descending order, because adding small terms first preserves their contributions before the sum becomes very large. The naive loop above implicitly sums in the given index order; if the data is unsorted, this order is essentially arbitrary.

The performance of linear accumulation is excellent (minimal overhead), and for many applications the resulting error is within acceptable limits. However, for '''very large N''' or ill-conditioned sums, the inaccuracy can become a problem. In a reduction involving millions of elements, a single-precision sum could lose several digits of accuracy. In deep learning, this could translate to small discrepancies in layer outputs or aggregated gradients if done naively in low precision.

=== 3.2. Pairwise and Tree-Based Reductions ===

A better approach to summation is '''pairwise summation''', which sums numbers in a binary tree structure rather than linearly. The idea is to recursively split the array into halves, sum each half, then add the two partial sums. This can be implemented as:

<syntaxhighlight lang="text">
function pairwise_sum(x[1..n]):
if n == 0:
return 0
if n == 1:
return x[1]
m = floor(n/2)
left_sum = pairwise_sum(x[1..m])
right_sum = pairwise_sum(x[m+1..n])
return left_sum + right_sum
</syntaxhighlight>

By always adding numbers of similar magnitudes first (small subsets are reduced to subtotals, then those are added, etc.), pairwise summation avoids the situation of a very large running total absorbing small additions until the final steps. The error of pairwise summation grows logarithmically with <math>N</math> in the worst case (roughly <math>O(\varepsilon \log_2 N)</math>)<ref name="ref5"/><ref name="ref5"/>, instead of linearly. Intuitively, at each level of the tree, rounding error is constrained, and there are about <math>\log_2 N</math> levels. In fact, the worst-case relative error bound for pairwise summation (base case of 1 element) is:

<math display="block">|E_N| \le \frac{\varepsilon \log_2 N}{1 - \varepsilon \log_2 N} \sum_{i=1}^N |x_i|,</math>

which for practical <math>N</math> simplifies to on the order of <math>\varepsilon \log_2 N</math> times the condition number of the sum<ref name="ref5"/><ref name="ref5"/>. In typical random-error scenarios, the error behaves even better (on the order of <math>\sqrt{\log N}</math>)<ref name="ref4"/><ref name="ref5"/>. The key is that '''pairwise (tree) reduction balances the accumulations''', preventing one sum from overwhelming the others too early.

From a performance standpoint, pairwise summation has the same <math>O(N)</math> total additions as linear summation, just a different order. It is also highly amenable to parallelization: independent partial sums can be computed in parallel at each level of the tree. Many numerical libraries and BLAS implementations use a form of blocked or pairwise summation in dot products for improved accuracy at negligible performance cost. For example, NumPy and Julia perform partial pairwise summation by default for better numerical behavior<ref name="ref4"/>.

=== 3.3. Compensated Summation (Kahan, Neumaier) ===

'''Compensated summation''' is an alternative technique to improve summation accuracy by tracking the small “lost” quantities. The most famous algorithm is the '''Kahan summation algorithm''' (also called Kahan–Babuška)<ref name="ref4"/>. Kahan summation uses an extra variable to accumulate the round-off error. In pseudocode:

<syntaxhighlight lang="c">
double sum = 0.0;
double c = 0.0; // compensation for lost low-order bits
for (i = 1; i <= N; ++i) {
double y = x[i] - c; // recover low-order bits by subtracting compensation
double t = sum + y; // perform the addition
c = (t - sum) - y; // compute new compensation (the error in t)
sum = t;
}
return sum;
</syntaxhighlight>

Here <code>c</code> accumulates the tiny leftovers that would otherwise be dropped<ref name="ref4"/><ref name="ref4"/>. Each iteration essentially subtracts the last error from the new addend (<code>y = x[i] - c</code>) so that <code>t = sum + y</code> includes the small bits from the previous operation. The new error is then stored in <code>c</code> for the next iteration. The result is that the final <code>sum</code> is as if computed with higher precision than the machine’s: Kahan’s algorithm can '''greatly reduce the error''' in summing a sequence of floats. In fact, with exact arithmetic for the compensation, Kahan summation yields a result accurate to within 1 or 2 ULP of the true sum, regardless of N (assuming the sequence of partial sums doesn’t overflow)<ref name="ref4"/>. More precisely, Kahan summation achieves a '''worst-case error independent of N''' (error bound depends only on machine precision, not the number of terms)<ref name="ref4"/>. In practice, using a double accumulator for summing single-precision numbers via Kahan can give near double-precision accuracy.

An important subtlety is that the compensation variable <code>c</code> should be kept in ''full precision''. In the implementation above, we used <code>double</code> for both sum and c. If summing FP32 values, one might use <code>float sum</code> and <code>float c</code> (both 32-bit), which still improves accuracy, but using a higher precision for the accumulator and compensation (i.e. double) yields even better results. Many implementations of Kahan summation use the same type for sum and c, but it's assumed to be higher precision than the inputs when possible.

Kahan’s algorithm carries a performance cost: each iteration does five arithmetic operations (two adds, one subtract, etc.) and has a data dependency, which can hinder parallelism or pipelining<ref name="ref4"/>. Thus, it runs slower than naive summation (roughly 2–3× slower or more, depending on CPU pipeline). In loops where summation is a bottleneck, this may be significant. Nonetheless, for modest N or where maximum accuracy is required, compensated summation is extremely useful. In fact, Python’s <code>math.fsum</code> function uses a variant of Kahan’s algorithm (actually a more advanced multi-part compensation) to sum iterables with high precision.

There are variations and improvements on basic Kahan summation. '''Neumaier’s algorithm''' (sometimes called ''Kahan-Babuška-Neumaier'') improves Kahan’s method by also handling cases where the next input is larger in magnitude than the running sum<ref name="ref4"/><ref name="ref4"/>. In such cases, the original Kahan formula could suffer error; Neumaier’s version effectively swaps roles when a huge value comes in. Its pseudocode is:

<syntaxhighlight lang="text">
function NeumaierSum(array):
sum = 0.0
compensation = 0.0
for each x in array:
t = sum + x
if (fabs(sum) >= fabs(x))
compensation += (sum - t) + x;
else
compensation += (x - t) + sum;
sum = t;
return sum + compensation;
</syntaxhighlight>

This algorithm ensures the compensation covers both scenarios: when the sum is the bigger addend or when the new term is bigger<ref name="ref4"/>. In the end, it adds the final <code>compensation</code> once to the result. Neumaier’s method can produce the correct result in cases where basic Kahan fails. For instance, summing [1.0, <math>+10^{100}</math>, 1.0, <math>-10^{100}</math>] in double precision: regular Kahan summation might give 0.0 (losing the small terms), whereas Neumaier’s algorithm yields 2.0 exactly<ref name="ref4"/><ref name="ref4"/>. There are also higher-order compensation methods (carrying multiple compensation terms, e.g., the '''Klein summation''' variant<ref name="ref4"/> or using arbitrary precision for intermediate sums) that further reduce error at increased computational cost<ref name="ref4"/><ref name="ref4"/>.

In summary, compensated summation algorithms trade extra computation for greatly improved numerical accuracy. They are especially valuable when summing large arrays of numbers with varying magnitudes, where naive summation could lose most of the low-order bits. Many scientific computing environments provide compensated sum routines for this reason. In deep learning, however, a different approach to improved precision is often taken: '''using a wider accumulator type'''.

=== 3.4. Mixed-Precision Accumulation ===

'''Mixed-precision accumulation''' refers to performing arithmetic in a higher precision than the input data. A common scenario is accumulating sums in 32-bit floats when the operands are 16-bit floats. Rather than implementing a compensation algorithm, one can simply use a wider accumulator so that intermediate rounding error is smaller. This is effectively what modern GPUs do for tensor operations: for example, NVIDIA '''Tensor Cores operate on FP16 input data with an FP32 accumulator''' (the 16-bit products are accumulated in 32-bit)<ref name="ref6"/>. In this way, summing 2048 products in a dot product has far less precision loss than it would with a 16-bit accumulator. Hardware that supports mixed precision can give the best of both worlds: fast low-precision multiplies and high-precision accumulation.

Software can also leverage mixed precision. For instance, one can sum an array of <code>float</code> values into a <code>double</code> variable, accumulating the result in double precision and then converting back to float at the end:

<syntaxhighlight lang="c">
double sum = 0.0;
for (int i=0; i<N; ++i) {
sum += (double) arr_float[i];
}
float result = (float) sum;
</syntaxhighlight>

This approach often yields a more accurate <code>result</code> than summing in single precision directly, because the intermediate sum carries 52-bit precision. The final rounding to float happens only once at the end (minimizing overall rounding error). In fact, the above is one way to implement <code>math.fsum</code> in Python (it internally does something similar with multiple partial sums in double precision).

In machine learning training and inference, '''mixed precision''' is widely used. When using FP16 for speed, it's common to accumulate with FP32. For example, '''Google's TPUs use BFloat16 for multiplication but accumulate in FP32''' for matrix ops<ref name="ref6"/>. AWS’s Inferentia (Neuron) hardware similarly supports FP16/BF16 matrix multiplication with FP32 accumulation<ref name="ref7"/>. The motivation is that summing hundreds or thousands of half-precision values would otherwise introduce intolerable error or risk overflow. With an FP32 accumulator, the reduction error is drastically reduced (by about <math>2^{13}</math> times, since FP32 has 13 more precision bits than FP16). Mixed-precision accumulation thus provides a simple and effective form of compensation: '''use a larger bucket to collect many small contributions'''.

There is a spectrum of options: summing int8 or int16 integers might use 32-bit integers as accumulators for the same reason (to avoid overflow). Some BLAS libraries even offer ''extended precision accumulators'' for single precision dot products (accumulating in double). The downside of mixed precision is the extra memory and maybe throughput cost of the wider type. But on many platforms, the add latency is dominated by hardware design rather than bit-width, so using a 32-bit adder vs 16-bit has negligible speed impact if supported natively. The key is whether data movement or register pressure increases.

In summary, '''using a higher precision accumulator is often the easiest way to improve numerical accuracy''' for reductions. It is a primary strategy in deep learning hardware to maintain stability while exploiting low precision for storage and initial computations.

== Precision Formats in AI and Machine Learning ==

Modern AI systems use a variety of numeric formats to balance speed, memory, and accuracy. We discuss common precisions and their roles:

=== 4.1. FP32 and FP64 in Training and Inference ===

'''FP32 (single precision)''' has been the workhorse of deep learning for years. Most training scripts in frameworks like TensorFlow and PyTorch used FP32 by default, as it offers sufficient precision to propagate gradients and accumulate weight updates over millions of iterations without excessive error. With ~7 decimal digits of precision and a wide exponent range, FP32 can handle the dynamic range of activations and gradients in typical neural networks. In inference, FP32 is often more precision than strictly necessary for final predictions, but it has remained common because of its ease and the fact that many GPU operations were optimized for FP32.

'''FP64 (double precision)''' is rarely used in deep learning model training or inference, except in specific scenarios requiring extreme precision. FP64 is common in scientific computing and certain types of numerical simulations. In neural networks, however, the extra precision is usually not needed for model accuracy—using doubles tends to yield negligible improvements in model metrics but at a large performance cost (since on GPUs FP64 throughput can be 1/2, 1/8, or even 1/32 of FP32 throughput depending on hardware). One exception might be when integrating neural nets into larger physics simulations or when solving ill-conditioned problems as part of the model. Some researchers use FP64 to diagnose numerical issues or ensure reproducibility. Generally though, deep learning has favored lower precision for efficiency, and FP64 is considered overkill for most networks<ref name="ref2"/>.

That said, certain classical machine learning algorithms benefit from double precision. For example, computing the inverse of a matrix (as in a Gaussian Process regression or Kalman filter embedded in a model) can incur large rounding errors in single precision, so using FP64 can significantly improve the result<ref name="ref1"/><ref name="ref1"/>. The ONNX standard supports a float64 tensor type for such needs. In ONNX Runtime, however, emphasis has been on FP32 performance, and some pipelines default to float even if the original model was double<ref name="ref1"/>. If a model’s prediction computation involves operations that are numerically sensitive (like matrix inversion, Cholesky decomposition, or summing extremely large counts), then FP64 may be required to avoid noticeable discrepancies<ref name="ref1"/><ref name="ref1"/>. A good practice is to test a model in both float and double if you suspect precision issues; significant differences indicate an ill-conditioned calculation that might need higher precision.

In summary, FP32 remains the '''default''' format for most training and inference due to its balance of accuracy and efficiency<ref name="ref2"/>, while FP64 is reserved for niche cases or precision-critical subroutines. FP32 typically achieves the same accuracy as FP64 for end-to-end deep learning tasks, because network training can often compensate for small noise, and other sources of error (like generalization error) dominate. Therefore, the community trend has been moving to '''lower''' precisions, not higher, to gain speed — provided the models still converge and perform well.

=== 4.2. FP16 and BFloat16 in Deep Learning ===

To further speed up neural network training and inference, '''half-precision (FP16)''' was adopted, especially with the advent of GPUs supporting FP16 math at twice the rate of FP32. Using FP16 reduces memory storage and bandwidth by 50% compared to FP32, which is significant for large models. However, FP16’s small exponent range and precision posed challenges: training a large network purely in FP16 can fail due to gradient underflow/overflow or weight updates not registering (because changes are below the 0.001 quantization step). To address this, frameworks introduced '''mixed-precision training''', where compute-intensive parts (matrix multiplies, convolutions) run in FP16, but certain accumulations and model parameters remain in FP32. For instance, NVIDIA’s '''Tensor Core''' units require FP16 (or BF16) inputs but accumulate results in FP32, effectively ensuring that reductions are done in higher precision<ref name="ref6"/>. Research showed that with proper loss scaling and mixed precision, models can train to FP32-level accuracy using mostly FP16 arithmetic, achieving significant speedups.

'''BFloat16 (BF16)''' emerged as an alternative 16-bit format primarily from Google’s TPU training systems. BF16 has a larger exponent (8 bits, same as FP32) but only 7 fraction bits<ref name="ref3"/>. This means BF16 can handle the '''same dynamic range''' as FP32 (no overflows where FP32 wouldn’t) but at the cost of even lower precision than FP16 (BF16 has about 2–3 decimal digits of precision vs FP16’s ~3–4). Despite its coarse precision, BF16 turns out to be effective for training deep networks because the algorithms (stochastic gradient descent, etc.) are somewhat robust to that noise, and having the wide range prevents catastrophic loss of signal (no gradient will overflow to Inf unless it was going to in FP32 anyway). Many training frameworks (PyTorch, TensorFlow) support BF16 training on hardware like '''TPU, NVIDIA A100+, Intel CPUs with AVX-512 BF16, etc'''. Typically, when using BF16, '''accumulations are still done in FP32''' (for example, a TPU v2/v3 multiplies BF16 matrices and accumulates into an FP32 result for the dot product<ref name="ref6"/>).

In inference, both FP16 and BF16 are used to deploy models for faster speed or lower memory. FP16 is popular on NVIDIA GPUs (which have specialized FP16 units). BF16 is advantageous on TPUs and some CPUs that support it, and increasingly on GPUs (Ampere and newer GPUs also support BF16 arithmetic). The choice between FP16 and BF16 can depend on hardware: BF16’s advantage is easier software conversion (you can often take an FP32-trained model and cast weights to BF16 with minimal loss, since the dynamic range is intact), whereas FP16 might require adjusting the model or ensuring values fit in range (sometimes needing techniques like maintaining a master copy of weights in FP32 during training, or handling out-of-range values).

For ONNX, both Float16 and BFloat16 types are part of the standard. ONNX models can be converted to FP16 (truncating weights and inserting cast nodes for computations) using tools like <code>float16_converter</code> in ONNX Runtime<ref name="ref8"/>. ONNX Runtime on GPUs will typically execute those FP16 ops with mixed precision on tensor cores (FP16 input, FP32 accumulate), whereas on CPU it may emulate FP16 or upcast to FP32 (as many CPU implementations do not natively support half-precision math)<ref name="ref9"/><ref name="ref9"/>. In fact, as an example, running an ONNX model in FP16 on a CPU might insert hidden Cast operations to FP32 behind the scenes, because the CPU kernel for that op only exists in FP32<ref name="ref9"/><ref name="ref9"/>. This ensures correctness (the computation is done in higher precision) but can reduce performance. We will revisit such backend-dependent behavior in Section 5.3.

In summary, '''FP16 and BF16''' are now established as important low-precision formats in deep learning. FP16 offers more precision bits, which can matter for e.g. small differences, but has a limited range (care needed to avoid overflow). BF16 offers safety in range but less precision (which can slightly affect convergence or final accuracy in some cases). Both typically rely on FP32 for accumulation to achieve acceptable accuracy. When deploying models, one should choose a format supported efficiently by the target hardware and verify that the accuracy is not significantly degraded. Often, CNNs and transformers can run in FP16/BF16 with virtually no drop in score, but certain operations (like softmax, see Section 6.3) might need special handling.

=== 4.3. FP8 Formats (E4M3, E5M2) and Scaling ===

The push for ever lower precision has led to experimental '''8-bit floating point (FP8)''' formats for deep learning. In 2022, researchers from NVIDIA, Intel, and Arm proposed an FP8 standard with two variants: '''E4M3''' (4 exponent bits, 3 fraction bits) and '''E5M2''' (5 exponent bits, 2 fraction bits)<ref name="ref10"/>. Both have 1 sign bit, for a total of 8 bits. E5M2 has a wider exponent range (5 bits gives bias 15) but one fewer mantissa bit than E4M3. The idea is to use E4M3 for numbers that need more precision (like weights or activations around 1.0) and E5M2 for numbers that need more dynamic range (like gradient updates which can be very small or large). In fact, the study found that using E4M3 for forward activations/weights and E5M2 for backward gradients can allow training of large networks with FP8 to accuracy on par with FP16 training<ref name="ref11"/><ref name="ref10"/>.

FP8 formats are ''not'' IEEE 754 standard (though IEEE is considering lower precision in its 2019 revision). Notably, the E4M3 format as used by NVIDIA does not allocate any bit-pattern for <math>+\infty</math> or <math>-\infty</math> – it has only NaN for invalid results, using that freed bit pattern to extend the representable range by one exponent value<ref name="ref10"/>. Essentially, E4M3 can represent numbers up to ~<math>2^{15}</math> (around 3e4) instead of capping at infinity at <math>2^{16}</math>, by foregoing infinities<ref name="ref10"/>. Variations also exist regarding negative zero and how NaNs are handled<ref name="ref10"/>. For instance, GraphCore’s IPU uses FP8 formats that disallow negative zero and infinities, to maximize useable range<ref name="ref10"/>.

Using 8-bit floats for neural networks requires additional techniques, chiefly '''scaling'''. Because 8-bit has very limited precision and range, one typically applies a power-of-two scale factor to groups of values to normalize them into an FP8-friendly range. This is akin to block floating-point or “microscaling” per layer or per tensor. For example, one might keep a FP32 scale for each channel or tensor, and represent values as <math>(FP8\_value) \times 2^{scale}</math>. The ONNX standard has introduced support for FP8 in version 1.15, including types for E4M3 and E5M2 (with variations FNUZ: finite-only and no negative zero)<ref name="ref10"/><ref name="ref10"/>, and even a type called '''E8M0''' which is an 8-bit exponent-only number used purely as a scale factor<ref name="ref10"/>. The E8M0 type has 8 exponent bits, no mantissa, effectively representing powers of two from <math>2^{-127}</math> to <math>2^{128}</math> for scaling purposes<ref name="ref10"/><ref name="ref10"/>. This indicates an approach where an FP8 tensor would be accompanied by an E8M0 scale (or scales) to interpret its value correctly.

'''Rounding''' becomes critical when casting values to FP8. The quantization error is much larger relative to FP16 or FP32. Studies have shown that choosing the right rounding mode (e.g., round-to-nearest, stochastic rounding, etc.) and clamping behavior (saturating to max instead of inf) can affect model accuracy<ref name="ref10"/>. ONNX added a <code>round_mode</code> attribute to its Cast operator to allow controlling this for FP8<ref name="ref10"/>. In particular, one paper found that '''rounding up with saturation''' (round-to-nearest ties away from 0, and clamp overflows to max finite rather than inf) gave better accuracy in large language model training than other rounding modes<ref name="ref10"/>.

At present, FP8 is on the cutting edge. NVIDIA’s H100 GPU supports FP8 tensor cores (for both E4M3 and E5M2), and early results show minimal accuracy loss on some vision and NLP tasks when properly tuned. However, using FP8 requires fine-tuning hyperparameters like scaling factors for each layer and perhaps tweaking training procedures. ONNX’s inclusion of FP8 types means you can represent and transport models using FP8 weights or activations, but actual hardware support is limited and typically paired with calibration or dynamic scaling logic outside of ONNX graph (or using custom ONNX functions for scaling). It remains an '''open research area''' how to fully automate FP8 quantization for arbitrary models without accuracy loss.

In summary, FP8 formats promise further compression of neural network computation and data. They are an order of magnitude more error-prone than FP16, but when combined with careful scaling, mixed usage of E4M3/E5M2, and maybe modified training techniques, they have shown surprising success. ONNX is actively evolving to support these ultra-low precision types, signaling their potential importance in future AI inference deployments.

== Accumulation Semantics in ONNX ==

One of the roles of the ONNX specification is to define the mathematical behavior of operators (Conv, MatMul, Sum, etc.) in a framework-agnostic way. However, ONNX mostly defines ''what'' an operator computes in exact arithmetic, not ''how'' the computation is carried out in finite precision. This leaves certain details – like accumulator precision or summation order – up to the implementation (runtime or hardware). We'll examine how ONNX treats accumulation and what guarantees (or lack thereof) are provided.

=== 5.1. Operator Definitions and Numerical Guarantees ===

Each ONNX operator is described as performing an ideal mathematical function on its inputs. For example, <code>MatMul</code> is defined as the standard matrix product of two tensors, and <code>ReduceSum</code> computes a summation along given axes. The ONNX spec, as of current versions, '''does not mandate the precision of intermediate calculations''' beyond the types of the inputs and outputs. If an operator’s inputs and outputs are float16, the spec implies the computation is done in float16 in a sense, but it does not forbid an implementation from using higher precision internally – it simply doesn't mention it. There is '''no explicit guarantee of identical bit-for-bit results''' across different ONNX backends for floating-point ops. Instead, results are expected to be numerically close, within tolerances one would normally expect from floating-point differences (due to different hardware or algorithms). The ONNX Backend Test suite typically uses a relative or absolute error tolerance when validating outputs for this reason.

Because of this, important details like the use of extended precision or accumulation in higher precision are left ambiguous. This has led to discussions in the ONNX community about making such behavior more explicit. For example, a proposal was raised to allow specification of accumulation precision for certain ops like MatMul<ref name="ref12"/>. The reasoning is that as lower-precision types (float16, bfloat16, float8) become more common, knowing or controlling whether an op internally accumulates in a higher precision is important for both accuracy and reproducibility<ref name="ref12"/>. In June 2025, an issue on the ONNX GitHub suggested adding an attribute to MatMul (and similarly Gemm, Softmax, etc.) to specify the accumulator type<ref name="ref12"/><ref name="ref12"/>. By default it would use input precision (status quo), but an attribute could request (for instance) 32-bit accumulation even if inputs are 16-bit<ref name="ref12"/>. As of this writing, however, no such attribute exists in the official spec – it's an open issue. Thus, the numerical semantics are effectively: ''“compute the result as if in the same type as inputs, but actual implementations may use more precision internally as an optimization.”''

From a guarantee standpoint, ONNX operators promise the mathematically correct result assuming real-number arithmetic, but they '''do not guarantee bit-exact results''' due to floating-point effects. There is an understanding that differences can arise from different summation orders, fused operations (e.g., an implementation might use an FMA (fused multiply-add) which has one rounding instead of two), or use of extended precision registers. ONNX does not formally define rounding modes either – it relies on IEEE 754 defaults, but if a backend used something like deterministic summation or alternative rounding, ONNX has no mechanism to express that.

In summary, ONNX's current stance is largely '''implementation-dependent''' for numerical precision. It provides minimal explicit numerical guarantees beyond type constraints. This is why an ONNX model run on two different hardware or runtime libraries can have slightly different results, especially in reduced precision. The burden is on backends to produce results "close enough" to each other that they don't affect the model's decisions in any significant way.

'''Open Neural Network Exchange (ONNX)''' is an open standard format for representing machine learning models and neural network computations across different frameworks and hardware<ref name="ref1"/>. As models are exported and deployed via ONNX, the '''numerical precision''' of computations becomes critical. Deep learning inference involves a variety of floating-point operations, and small numerical discrepancies can accumulate and affect model accuracy or reproducibility. This article provides an in-depth overview of floating-point precision issues in numerical computing, with a focus on how they manifest in AI inference and the ONNX ecosystem. We cover floating-point formats and rounding behavior, accumulation strategies for summations, the use of low-precision formats (FP16, BFloat16, FP8) in modern AI, and how ONNX defines (or leaves unspecified) the precision of accumulators in operations. Strategies for maintaining numerical stability in deep neural networks are discussed, along with the trade-offs between computational performance and arithmetic accuracy. The goal is to give engineers and researchers a comprehensive, encyclopedic reference on numerical precision in ONNX-based AI inference systems.

== Floating-Point Precision in Numerical Computing ==

=== 2.1. IEEE 754 Floating-Point Formats ===

Modern computers represent real numbers using the IEEE 754 floating-point standard, which defines formats like 32-bit ''single precision'' (FP32) and 64-bit ''double precision'' (FP64). In IEEE 754 binary formats, a floating-point number is composed of three fields: 1 sign bit, several exponent bits (with a bias), and several fraction (mantissa) bits. For example, FP32 consists of 1 sign bit, 8 exponent bits, and 23 fraction bits (24 bits of significand precision including the implicit leading 1)<ref name="ref2"/>. This provides roughly 7 decimal digits of precision and an exponent range allowing values on the order of ''10±38''. FP64 allocates 1 sign bit, 11 exponent bits, and 52 fraction bits (53-bit precision), yielding about 15–16 decimal digits of precision and a vastly larger range (~''10±308'')<ref name="ref2"/>. These larger formats (especially FP64) are the "gold standard" for numerical accuracy in scientific computing, as they reduce round-off error in complex calculations<ref name="ref2"/>. However, the increased precision comes at the cost of doubled storage and typically reduced computational throughput (e.g. many GPUs execute FP64 instructions at a much lower rate than FP32<ref name="ref2"/>).

In addition to single and double precision, the IEEE 754-2008 standard introduced a 16-bit ''half precision'' format (FP16, or binary16). FP16 has 1 sign bit, 5 exponent bits, and 10 fraction bits (11-bit significand including the implicit 1)<ref name="ref3"/>. This format covers a much smaller dynamic range (approximately ''6.1 × 10-5'' to ''6.5 × 104'' for normalized values<ref name="ref3"/>) and offers only around 3 decimal digits of precision. Half precision was originally used in graphics, but it has gained popularity in machine learning for its speed and memory advantages (discussed later in Section 4). Another format relevant to AI is '''BFloat16''' (Brain Float 16), a 16-bit float with '''8 exponent bits and 7 fraction bits'''<ref name="ref3"/>. BFloat16 sacrifices precision (only ~7 fraction bits ≈ 2–3 decimal digits) in order to have the '''same exponent range as FP32'''<ref name="ref3"/>. This means BFloat16 can represent very large or very small numbers (∼''10±38'', similar range as FP32) but with much less precision between representable values<ref name="ref3"/>. BFloat16 was introduced for deep learning by Google (for TPUs) to allow reduced precision ''training'' without frequent overflow/underflow, relying on the wide range and accepting the coarse precision.

Each floating-point format has a '''unit roundoff''' or machine precision, denoted ''ε''. This is effectively the spacing between 1.0 and the next representable number. For FP32, ''ε ≈ 2-23 ≈ 1.19 × 10-7'', corresponding to about 7–8 decimal digits of precision<ref name="ref3"/>. In FP64, ''ε = 2-52 ≈ 2.22 × 10-16'' (15–16 decimal digits). Half precision is much less precise: FP16 has ''ε = 2-10 ≈ 9.76 × 10-4''<ref name="ref3"/>, and BFloat16 has ''ε = 2-7 = 7.8125 × 10-3''<ref name="ref3"/>. These values quantify how finely the continuum of real numbers is quantized in each format.

=== 2.2. Machine Epsilon and Rounding Error ===

'''Machine epsilon (''ε'')''' is formally defined as the smallest positive number such that ''1.0 + ε > 1.0'' in the floating-point format<ref name="ref3"/>. It effectively measures the '''precision''' of the representation. In FP32, ''ε = 2-23 ≈ 1.19 × 10-7'', meaning any result is rounded to about 7 decimal digits. FP16’s ''ε = 2-10 ≈ 9.77 × 10-4'', over 1000× larger (so only ~3 decimal digits of precision), and BFloat16’s ''ε = 2-7 ≈ 7.8 × 10-3'', reflecting its mere ~8-bit precision<ref name="ref3"/>. Rounding in IEEE 754 by default uses '''round-to-nearest (ties to even)''', which ensures the rounding error for any single operation is at most half a unit in the last place (0.5 ULP). We can model a floating-point addition as:

<math>\text{fl}(a+b) = (a+b)\,(1 + \delta),</math>

where ''|δ| ≤ ε'' for that format<ref name="ref3"/>. Here ''ε/2'' is the maximum '''relative error''' in one rounding (also called unit roundoff). For example, when adding two FP32 numbers, the result is accurate to within about ''10-7'' of the true sum (relative), whereas in BF16, the result can deviate by up to ''∼7.8 × 10-3'' (nearly 0.78% relative error per operation)<ref name="ref3"/>. Thus, '''each arithmetic operation incurs a small rounding error''' (on the order of machine epsilon) that can propagate or accumulate through a sequence of calculations.

Because floating-point representations have finite exponent range, we must also consider '''overflow''' and '''underflow'''. If a result’s magnitude exceeds the maximum representable value, it overflows to infinity; if it falls below the minimum normal (or minimum subnormal) value, it underflows to zero (or a denormalized tiny value). FP32 and BF16 share an exponent size (8 bits), so their range for normal numbers is roughly ''[10-38, 1038]'' (approx <math>~\!3.4\times10^{38}</math> max)<ref name="ref3"/><ref name="ref3"/>. FP16’s 5-bit exponent allows max ''∼ 6.55 × 104'' and min normal ''∼ 6.10 × 10-5''<ref name="ref3"/>. In practice, '''underflow/overflow''' can occur in low-precision arithmetic if values are not appropriately scaled. For instance, summing a large number of moderately sized terms in FP16 could overflow where FP32 would not. As an example, summing on the order of ''104'' values of magnitude ~''101'' (say, adding 10,000 terms around 10 each) would produce a total ''∼105'', which exceeds FP16’s max finite value ~''6.5 × 104'' and would overflow to ''+∞'' in half precision<ref name="ref3"/>. Underflow is usually a concern when subtracting nearly equal numbers (catastrophic cancellation yielding a tiny difference that might flush to zero in low precision).

=== 2.3. Accumulation Error in Reductions ===

Floating-point addition is '''not associative''': ''(a+b)+c'' can differ from ''a+(b+c)'' due to rounding, and thus the order in which we sum a list of numbers can change the result slightly. When summing a large sequence of ''N'' numbers ''x1, x2, ..., xN'', the '''accumulated rounding error''' can be significant. In the ''worst case'' (e.g., adding many positive and negative terms that cause maximal cancellation or always rounding one way), the error can grow on the order of ''O(N ε)'' relative to the exact sum<ref name="ref4"/><ref name="ref5"/>. Simply summing ''N'' numbers in sequence (the '''naive linear summation''') has a worst-case error bound proportional to <math>N\,\varepsilon</math><ref name="ref4"/>. This worst-case occurs for adversarial arrangements of values; in more typical scenarios with rounding errors of random signs, one can model the error as a random walk, yielding an expected root-mean-square (RMS) error growth on the order of <math>O(\sqrt{N}\,\varepsilon)</math><ref name="ref4"/><ref name="ref5"/>. Still, with very large ''N'', even <math>\sqrt{N}</math> times ''ε'' can become large enough to matter.

'''Cancellation''' is a particular problem: if we add two nearly equal numbers of opposite sign, most leading digits cancel and the result loses significant bits of precision. For example, in single precision, subtracting two numbers that agree in the first 6 decimal digits will yield a result with only ~1 digit of accuracy left. Catastrophic cancellation can dramatically increase relative error. Summation is especially sensitive to cancellation if large positive and negative terms are summed naively. The '''condition number''' of a summation (the ratio of the sum of absolute values <math>\sum|x_i|</math> to the absolute value of the true sum <math>|\sum x_i|</math>) indicates how sensitive the result is to perturbations<ref name="ref5"/>. If the numbers have mixed signs and nearly cancel out (making the condition number large), even an optimal summation order will have a large relative error. For instance, summing [1.0, ''10100'', 1.0, ''-10100''] in double precision has an exact sum of 2.0, but naive summation may yield 0.0 due to cancellation; an improved summation algorithm yields the correct 2.0<ref name="ref4"/><ref name="ref4"/>.

In practical terms, when summing a long list of floating-point numbers (such as accumulating a dot product or reducing a tensor), the '''accumulation error''' can become noticeable if no precautions are taken. Techniques to mitigate this error include carefully choosing the summation order or using higher precision for intermediate sums. We will explore these strategies in the next section.

== Accumulators and Reduction Strategies ==

When performing reductions (sums, dot products, etc.), the method of accumulation can greatly affect the accuracy of the result. Below, we discuss various strategies:

=== 3.1. Naive Linear Accumulation ===

The simplest accumulation strategy is to use a single accumulator and add each element sequentially. In C-like pseudocode:

<syntaxhighlight lang="c">
float sum = 0.0f;
for (int i = 0; i < N; ++i) {
sum += array[i];
}
</syntaxhighlight>

This '''naive linear summation''' adds values one by one to the running total. While straightforward and fast, it is prone to the rounding error accumulation described earlier. The partial sum <code>sum</code> continually grows (in absolute value) as terms are added, so new small terms may fall outside its precision. For example, if <code>sum</code> is large and positive, adding a comparatively tiny positive number may produce no change because the small term's significant digits vanish in the presence of the large sum (the increment is below the rounding threshold of <code>sum</code>). Conversely, adding a small number of opposite sign can suffer from cancellation. The order of addition matters: summing from smallest magnitude to largest (ascending order) tends to yield a more accurate result than summing in descending order, because adding small terms first preserves their contributions before the sum becomes very large. The naive loop above implicitly sums in the given index order; if the data is unsorted, this order is essentially arbitrary.

The performance of linear accumulation is excellent (minimal overhead), and for many applications the resulting error is within acceptable limits. However, for '''very large N''' or ill-conditioned sums, the inaccuracy can become a problem. In a reduction involving millions of elements, a single-precision sum could lose several digits of accuracy. In deep learning, this could translate to small discrepancies in layer outputs or aggregated gradients if done naively in low precision.

=== 3.2. Pairwise and Tree-Based Reductions ===

A better approach to summation is '''pairwise summation''', which sums numbers in a binary tree structure rather than linearly. The idea is to recursively split the array into halves, sum each half, then add the two partial sums. This can be implemented as:

<syntaxhighlight lang="text">
function pairwise_sum(x[1..n]):
if n == 0:
return 0
if n == 1:
return x[1]
m = floor(n/2)
left_sum = pairwise_sum(x[1..m])
right_sum = pairwise_sum(x[m+1..n])
return left_sum + right_sum
</syntaxhighlight>

By always adding numbers of similar magnitudes first (small subsets are reduced to subtotals, then those are added, etc.), pairwise summation avoids the situation of a very large running total absorbing small additions until the final steps. The error of pairwise summation grows logarithmically with ''N'' in the worst case (roughly ''O(ε log2 N)'')<ref name="ref5"/><ref name="ref5"/>, instead of linearly. Intuitively, at each level of the tree, rounding error is constrained, and there are about ''log2 N'' levels. In fact, the worst-case relative error bound for pairwise summation (base case of 1 element) is:

<math>|E_N| \le \frac{\varepsilon \log_2 N}{1 - \varepsilon \log_2 N} \sum_{i=1}^N |x_i|,</math>

which for practical ''N'' simplifies to on the order of ''ε log2 N'' times the condition number of the sum<ref name="ref5"/><ref name="ref5"/>. In typical random-error scenarios, the error behaves even better (on the order of <math>\sqrt{\log N}</math>)<ref name="ref4"/><ref name="ref5"/>. The key is that '''pairwise (tree) reduction balances the accumulations''', preventing one sum from overwhelming the others too early.

From a performance standpoint, pairwise summation has the same ''O(N)'' total additions as linear summation, just a different order. It is also highly amenable to parallelization: independent partial sums can be computed in parallel at each level of the tree. Many numerical libraries and BLAS implementations use a form of blocked or pairwise summation in dot products for improved accuracy at negligible performance cost. For example, NumPy and Julia perform partial pairwise summation by default for better numerical behavior<ref name="ref4"/>.

=== 3.3. Compensated Summation (Kahan, Neumaier) ===

'''Compensated summation''' is an alternative technique to improve summation accuracy by tracking the small “lost” quantities. The most famous algorithm is the '''Kahan summation algorithm''' (also called Kahan–Babuška)<ref name="ref4"/>. Kahan summation uses an extra variable to accumulate the round-off error. In pseudocode:

<syntaxhighlight lang="c">
double sum = 0.0;
double c = 0.0; // compensation for lost low-order bits
for (i = 1; i <= N; ++i) {
double y = x[i] - c; // recover low-order bits by subtracting compensation
double t = sum + y; // perform the addition
c = (t - sum) - y; // compute new compensation (the error in t)
sum = t;
}
return sum;
</syntaxhighlight>

Here <code>c</code> accumulates the tiny leftovers that would otherwise be dropped<ref name="ref4"/><ref name="ref4"/>. Each iteration essentially subtracts the last error from the new addend (<code>y = x[i] - c</code>) so that <code>t = sum + y</code> includes the small bits from the previous operation. The new error is then stored in <code>c</code> for the next iteration. The result is that the final <code>sum</code> is as if computed with higher precision than the machine’s: Kahan’s algorithm can '''greatly reduce the error''' in summing a sequence of floats. In fact, with exact arithmetic for the compensation, Kahan summation yields a result accurate to within 1 or 2 ULP of the true sum, regardless of N (assuming the sequence of partial sums doesn’t overflow)<ref name="ref4"/>. More precisely, Kahan summation achieves a '''worst-case error independent of N''' (error bound depends only on machine precision, not the number of terms)<ref name="ref4"/>. In practice, using a double accumulator for summing single-precision numbers via Kahan can give near double-precision accuracy.

An important subtlety is that the compensation variable <code>c</code> should be kept in ''full precision''. In the implementation above, we used <code>double</code> for both sum and c. If summing FP32 values, one might use <code>float sum</code> and <code>float c</code> (both 32-bit), which still improves accuracy, but using a higher precision for the accumulator and compensation (i.e. double) yields even better results. Many implementations of Kahan summation use the same type for sum and c, but it's assumed to be higher precision than the inputs when possible.

Kahan’s algorithm carries a performance cost: each iteration does five arithmetic operations (two adds, one subtract, etc.) and has a data dependency, which can hinder parallelism or pipelining<ref name="ref4"/>. Thus, it runs slower than naive summation (roughly 2–3× slower or more, depending on CPU pipeline). In loops where summation is a bottleneck, this may be significant. Nonetheless, for modest N or where maximum accuracy is required, compensated summation is extremely useful. In fact, Python’s <code>math.fsum</code> function uses a variant of Kahan’s algorithm (actually a more advanced multi-part compensation) to sum iterables with high precision.

There are variations and improvements on basic Kahan summation. '''Neumaier’s algorithm''' (sometimes called ''Kahan-Babuška-Neumaier'') improves Kahan’s method by also handling cases where the next input is larger in magnitude than the running sum<ref name="ref4"/><ref name="ref4"/>. In such cases, the original Kahan formula could suffer error; Neumaier’s version effectively swaps roles when a huge value comes in. Its pseudocode is:

<syntaxhighlight lang="text">
function NeumaierSum(array):
sum = 0.0
compensation = 0.0
for each x in array:
t = sum + x
if (fabs(sum) >= fabs(x))
compensation += (sum - t) + x;
else
compensation += (x - t) + sum;
sum = t;
return sum + compensation;
</syntaxhighlight>

This algorithm ensures the compensation covers both scenarios: when the sum is the bigger addend or when the new term is bigger<ref name="ref4"/>. In the end, it adds the final <code>compensation</code> once to the result. Neumaier’s method can produce the correct result in cases where basic Kahan fails. For instance, summing [1.0, ''+10100'', 1.0, ''-10100''] in double precision: regular Kahan summation might give 0.0 (losing the small terms), whereas Neumaier’s algorithm yields 2.0 exactly<ref name="ref4"/><ref name="ref4"/>. There are also higher-order compensation methods (carrying multiple compensation terms, e.g., the '''Klein summation''' variant<ref name="ref4"/> or using arbitrary precision for intermediate sums) that further reduce error at increased computational cost<ref name="ref4"/><ref name="ref4"/>.

In summary, compensated summation algorithms trade extra computation for greatly improved numerical accuracy. They are especially valuable when summing large arrays of numbers with varying magnitudes, where naive summation could lose most of the low-order bits. Many scientific computing environments provide compensated sum routines for this reason. In deep learning, however, a different approach to improved precision is often taken: '''using a wider accumulator type'''.

=== 3.4. Mixed-Precision Accumulation ===

'''Mixed-precision accumulation''' refers to performing arithmetic in a higher precision than the input data. A common scenario is accumulating sums in 32-bit floats when the operands are 16-bit floats. Rather than implementing a compensation algorithm, one can simply use a wider accumulator so that intermediate rounding error is smaller. This is effectively what modern GPUs do for tensor operations: for example, NVIDIA '''Tensor Cores operate on FP16 input data with an FP32 accumulator''' (the 16-bit products are accumulated in 32-bit)<ref name="ref6"/>. In this way, summing 2048 products in a dot product has far less precision loss than it would with a 16-bit accumulator. Hardware that supports mixed precision can give the best of both worlds: fast low-precision multiplies and high-precision accumulation.

Software can also leverage mixed precision. For instance, one can sum an array of <code>float</code> values into a <code>double</code> variable, accumulating the result in double precision and then converting back to float at the end:

<syntaxhighlight lang="c">
double sum = 0.0;
for (int i=0; i<N; ++i) {
sum += (double) arr_float[i];
}
float result = (float) sum;
</syntaxhighlight>

This approach often yields a more accurate <code>result</code> than summing in single precision directly, because the intermediate sum carries 52-bit precision. The final rounding to float happens only once at the end (minimizing overall rounding error). In fact, the above is one way to implement <code>math.fsum</code> in Python (it internally does something similar with multiple partial sums in double precision).

In machine learning training and inference, '''mixed precision''' is widely used. When using FP16 for speed, it's common to accumulate with FP32. For example, '''Google's TPUs use BFloat16 for multiplication but accumulate in FP32''' for matrix ops<ref name="ref6"/>. AWS’s Inferentia (Neuron) hardware similarly supports FP16/BF16 matrix multiplication with FP32 accumulation<ref name="ref7"/>. The motivation is that summing hundreds or thousands of half-precision values would otherwise introduce intolerable error or risk overflow. With an FP32 accumulator, the reduction error is drastically reduced (by about ''213'' times, since FP32 has 13 more precision bits than FP16). Mixed-precision accumulation thus provides a simple and effective form of compensation: '''use a larger bucket to collect many small contributions'''.

There is a spectrum of options: summing int8 or int16 integers might use 32-bit integers as accumulators for the same reason (to avoid overflow). Some BLAS libraries even offer ''extended precision accumulators'' for single precision dot products (accumulating in double). The downside of mixed precision is the extra memory and maybe throughput cost of the wider type. But on many platforms, the add latency is dominated by hardware design rather than bit-width, so using a 32-bit adder vs 16-bit has negligible speed impact if supported natively. The key is whether data movement or register pressure increases.

In summary, '''using a higher precision accumulator is often the easiest way to improve numerical accuracy''' for reductions. It is a primary strategy in deep learning hardware to maintain stability while exploiting low precision for storage and initial computations.

== Precision Formats in AI and Machine Learning ==

Modern AI systems use a variety of numeric formats to balance speed, memory, and accuracy. We discuss common precisions and their roles:

=== 4.1. FP32 and FP64 in Training and Inference ===

'''FP32 (single precision)''' has been the workhorse of deep learning for years. Most training scripts in frameworks like TensorFlow and PyTorch used FP32 by default, as it offers sufficient precision to propagate gradients and accumulate weight updates over millions of iterations without excessive error. With ~7 decimal digits of precision and a wide exponent range, FP32 can handle the dynamic range of activations and gradients in typical neural networks. In inference, FP32 is often more precision than strictly necessary for final predictions, but it has remained common because of its ease and the fact that many GPU operations were optimized for FP32.

'''FP64 (double precision)''' is rarely used in deep learning model training or inference, except in specific scenarios requiring extreme precision. FP64 is common in scientific computing and certain types of numerical simulations. In neural networks, however, the extra precision is usually not needed for model accuracy—using doubles tends to yield negligible improvements in model metrics but at a large performance cost (since on GPUs FP64 throughput can be 1/2, 1/8, or even 1/32 of FP32 throughput depending on hardware). One exception might be when integrating neural nets into larger physics simulations or when solving ill-conditioned problems as part of the model. Some researchers use FP64 to diagnose numerical issues or ensure reproducibility. Generally though, deep learning has favored lower precision for efficiency, and FP64 is considered overkill for most networks<ref name="ref2"/>.

That said, certain classical machine learning algorithms benefit from double precision. For example, computing the inverse of a matrix (as in a Gaussian Process regression or Kalman filter embedded in a model) can incur large rounding errors in single precision, so using FP64 can significantly improve the result<ref name="ref1"/><ref name="ref1"/>. The ONNX standard supports a float64 tensor type for such needs. In ONNX Runtime, however, emphasis has been on FP32 performance, and some pipelines default to float even if the original model was double<ref name="ref1"/>. If a model’s prediction computation involves operations that are numerically sensitive (like matrix inversion, Cholesky decomposition, or summing extremely large counts), then FP64 may be required to avoid noticeable discrepancies<ref name="ref1"/><ref name="ref1"/>. A good practice is to test a model in both float and double if you suspect precision issues; significant differences indicate an ill-conditioned calculation that might need higher precision.

In summary, FP32 remains the '''default''' format for most training and inference due to its balance of accuracy and efficiency<ref name="ref2"/>, while FP64 is reserved for niche cases or precision-critical subroutines. FP32 typically achieves the same accuracy as FP64 for end-to-end deep learning tasks, because network training can often compensate for small noise, and other sources of error (like generalization error) dominate. Therefore, the community trend has been moving to '''lower''' precisions, not higher, to gain speed — provided the models still converge and perform well.

=== 4.2. FP16 and BFloat16 in Deep Learning ===

To further speed up neural network training and inference, '''half-precision (FP16)''' was adopted, especially with the advent of GPUs supporting FP16 math at twice the rate of FP32. Using FP16 reduces memory storage and bandwidth by 50% compared to FP32, which is significant for large models. However, FP16’s small exponent range and precision posed challenges: training a large network purely in FP16 can fail due to gradient underflow/overflow or weight updates not registering (because changes are below the 0.001 quantization step). To address this, frameworks introduced '''mixed-precision training''', where compute-intensive parts (matrix multiplies, convolutions) run in FP16, but certain accumulations and model parameters remain in FP32. For instance, NVIDIA’s '''Tensor Core''' units require FP16 (or BF16) inputs but accumulate results in FP32, effectively ensuring that reductions are done in higher precision<ref name="ref6"/>. Research showed that with proper loss scaling and mixed precision, models can train to FP32-level accuracy using mostly FP16 arithmetic, achieving significant speedups.

'''BFloat16 (BF16)''' emerged as an alternative 16-bit format primarily from Google’s TPU training systems. BF16 has a larger exponent (8 bits, same as FP32) but only 7 fraction bits<ref name="ref3"/>. This means BF16 can handle the '''same dynamic range''' as FP32 (no overflows where FP32 wouldn’t) but at the cost of even lower precision than FP16 (BF16 has about 2–3 decimal digits of precision vs FP16’s ~3–4). Despite its coarse precision, BF16 turns out to be effective for training deep networks because the algorithms (stochastic gradient descent, etc.) are somewhat robust to that noise, and having the wide range prevents catastrophic loss of signal (no gradient will overflow to Inf unless it was going to in FP32 anyway). Many training frameworks (PyTorch, TensorFlow) support BF16 training on hardware like '''TPU, NVIDIA A100+, Intel CPUs with AVX-512 BF16, etc'''. Typically, when using BF16, '''accumulations are still done in FP32''' (for example, a TPU v2/v3 multiplies BF16 matrices and accumulates into an FP32 result for the dot product<ref name="ref6"/>).

In inference, both FP16 and BF16 are used to deploy models for faster speed or lower memory. FP16 is popular on NVIDIA GPUs (which have specialized FP16 units). BF16 is advantageous on TPUs and some CPUs that support it, and increasingly on GPUs (Ampere and newer GPUs also support BF16 arithmetic). The choice between FP16 and BF16 can depend on hardware: BF16’s advantage is easier software conversion (you can often take an FP32-trained model and cast weights to BF16 with minimal loss, since the dynamic range is intact), whereas FP16 might require adjusting the model or ensuring values fit in range (sometimes needing techniques like maintaining a master copy of weights in FP32 during training, or handling out-of-range values).

For ONNX, both Float16 and BFloat16 types are part of the standard. ONNX models can be converted to FP16 (truncating weights and inserting cast nodes for computations) using tools like <code>float16_converter</code> in ONNX Runtime<ref name="ref8"/>. ONNX Runtime on GPUs will typically execute those FP16 ops with mixed precision on tensor cores (FP16 input, FP32 accumulate), whereas on CPU it may emulate FP16 or upcast to FP32 (as many CPU implementations do not natively support half-precision math)<ref name="ref9"/><ref name="ref9"/>. In fact, as an example, running an ONNX model in FP16 on a CPU might insert hidden Cast operations to FP32 behind the scenes, because the CPU kernel for that op only exists in FP32<ref name="ref9"/><ref name="ref9"/>. This ensures correctness (the computation is done in higher precision) but can reduce performance. We will revisit such backend-dependent behavior in Section 5.3.

In summary, '''FP16 and BF16''' are now established as important low-precision formats in deep learning. FP16 offers more precision bits, which can matter for e.g. small differences, but has a limited range (care needed to avoid overflow). BF16 offers safety in range but less precision (which can slightly affect convergence or final accuracy in some cases). Both typically rely on FP32 for accumulation to achieve acceptable accuracy. When deploying models, one should choose a format supported efficiently by the target hardware and verify that the accuracy is not significantly degraded. Often, CNNs and transformers can run in FP16/BF16 with virtually no drop in score, but certain operations (like softmax, see Section 6.3) might need special handling.

=== 4.3. FP8 Formats (E4M3, E5M2) and Scaling ===

The push for ever lower precision has led to experimental '''8-bit floating point (FP8)''' formats for deep learning. In 2022, researchers from NVIDIA, Intel, and Arm proposed an FP8 standard with two variants: '''E4M3''' (4 exponent bits, 3 fraction bits) and '''E5M2''' (5 exponent bits, 2 fraction bits)<ref name="ref10"/>. Both have 1 sign bit, for a total of 8 bits. E5M2 has a wider exponent range (5 bits gives bias 15) but one fewer mantissa bit than E4M3. The idea is to use E4M3 for numbers that need more precision (like weights or activations around 1.0) and E5M2 for numbers that need more dynamic range (like gradient updates which can be very small or large). In fact, the study found that using E4M3 for forward activations/weights and E5M2 for backward gradients can allow training of large networks with FP8 to accuracy on par with FP16 training<ref name="ref11"/><ref name="ref10"/>.

FP8 formats are ''not'' IEEE 754 standard (though IEEE is considering lower precision in its 2019 revision). Notably, the E4M3 format as used by NVIDIA does not allocate any bit-pattern for ''+∞'' or ''-∞'' – it has only NaN for invalid results, using that freed bit pattern to extend the representable range by one exponent value<ref name="ref10"/>. Essentially, E4M3 can represent numbers up to ~''215'' (around 3e4) instead of capping at infinity at ''216'', by foregoing infinities<ref name="ref10"/>. Variations also exist regarding negative zero and how NaNs are handled<ref name="ref10"/>. For instance, GraphCore’s IPU uses FP8 formats that disallow negative zero and infinities, to maximize useable range<ref name="ref10"/>.

Using 8-bit floats for neural networks requires additional techniques, chiefly '''scaling'''. Because 8-bit has very limited precision and range, one typically applies a power-of-two scale factor to groups of values to normalize them into an FP8-friendly range. This is akin to block floating-point or “microscaling” per layer or per tensor. For example, one might keep a FP32 scale for each channel or tensor, and represent values as <math>(FP8\_value) \times 2^{scale}</math>. The ONNX standard has introduced support for FP8 in version 1.15, including types for E4M3 and E5M2 (with variations FNUZ: finite-only and no negative zero)<ref name="ref10"/><ref name="ref10"/>, and even a type called '''E8M0''' which is an 8-bit exponent-only number used purely as a scale factor<ref name="ref10"/>. The E8M0 type has 8 exponent bits, no mantissa, effectively representing powers of two from ''2-127'' to ''2128'' for scaling purposes<ref name="ref10"/><ref name="ref10"/>. This indicates an approach where an FP8 tensor would be accompanied by an E8M0 scale (or scales) to interpret its value correctly.

'''Rounding''' becomes critical when casting values to FP8. The quantization error is much larger relative to FP16 or FP32. Studies have shown that choosing the right rounding mode (e.g., round-to-nearest, stochastic rounding, etc.) and clamping behavior (saturating to max instead of inf) can affect model accuracy<ref name="ref10"/>. ONNX added a <code>round_mode</code> attribute to its Cast operator to allow controlling this for FP8<ref name="ref10"/>. In particular, one paper found that '''rounding up with saturation''' (round-to-nearest ties away from 0, and clamp overflows to max finite rather than inf) gave better accuracy in large language model training than other rounding modes<ref name="ref10"/>.

At present, FP8 is on the cutting edge. NVIDIA’s H100 GPU supports FP8 tensor cores (for both E4M3 and E5M2), and early results show minimal accuracy loss on some vision and NLP tasks when properly tuned. However, using FP8 requires fine-tuning hyperparameters like scaling factors for each layer and perhaps tweaking training procedures. ONNX’s inclusion of FP8 types means you can represent and transport models using FP8 weights or activations, but actual hardware support is limited and typically paired with calibration or dynamic scaling logic outside of ONNX graph (or using custom ONNX functions for scaling). It remains an '''open research area''' how to fully automate FP8 quantization for arbitrary models without accuracy loss.

In summary, FP8 formats promise further compression of neural network computation and data. They are an order of magnitude more error-prone than FP16, but when combined with careful scaling, mixed usage of E4M3/E5M2, and maybe modified training techniques, they have shown surprising success. ONNX is actively evolving to support these ultra-low precision types, signaling their potential importance in future AI inference deployments.

== Accumulation Semantics in ONNX ==

One of the roles of the ONNX specification is to define the mathematical behavior of operators (Conv, MatMul, Sum, etc.) in a framework-agnostic way. However, ONNX mostly defines ''what'' an operator computes in exact arithmetic, not ''how'' the computation is carried out in finite precision. This leaves certain details – like accumulator precision or summation order – up to the implementation (runtime or hardware). We'll examine how ONNX treats accumulation and what guarantees (or lack thereof) are provided.

=== 5.1. Operator Definitions and Numerical Guarantees ===

Each ONNX operator is described as performing an ideal mathematical function on its inputs. For example, <code>MatMul</code> is defined as the standard matrix product of two tensors, and <code>ReduceSum</code> computes a summation along given axes. The ONNX spec, as of current versions, '''does not mandate the precision of intermediate calculations''' beyond the types of the inputs and outputs. If an operator’s inputs and outputs are float16, the spec implies the computation is done in float16 in a sense, but it does not forbid an implementation from using higher precision internally – it simply doesn't mention it. There is '''no explicit guarantee of identical bit-for-bit results''' across different ONNX backends for floating-point ops. Instead, results are expected to be numerically close, within tolerances one would normally expect from floating-point differences (due to different hardware or algorithms). The ONNX Backend Test suite typically uses a relative or absolute error tolerance when validating outputs for this reason.

Because of this, important details like the use of extended precision or accumulation in higher precision are left ambiguous. This has led to discussions in the ONNX community about making such behavior more explicit. For example, a proposal was raised to allow specification of accumulation precision for certain ops like MatMul<ref name="ref12"/>. The reasoning is that as lower-precision types (float16, bfloat16, float8) become more common, knowing or controlling whether an op internally accumulates in a higher precision is important for both accuracy and reproducibility<ref name="ref12"/>. In June 2025, an issue on the ONNX GitHub suggested adding an attribute to MatMul (and similarly Gemm, Softmax, etc.) to specify the accumulator type<ref name="ref12"/><ref name="ref12"/>. By default it would use input precision (status quo), but an attribute could request (for instance) 32-bit accumulation even if inputs are 16-bit<ref name="ref12"/>. As of this writing, however, no such attribute exists in the official spec – it's an open issue. Thus, the numerical semantics are effectively: ''“compute the result as if in the same type as inputs, but actual implementations may use more precision internally as an optimization.”''

From a guarantee standpoint, ONNX operators promise the mathematically correct result assuming real-number arithmetic, but they '''do not guarantee bit-exact results''' due to floating-point effects. There is an understanding that differences can arise from different summation orders, fused operations (e.g., an implementation might use an FMA (fused multiply-add) which has one rounding instead of two), or use of extended precision registers. ONNX does not formally define rounding modes either – it relies on IEEE 754 defaults, but if a backend used something like deterministic summation or alternative rounding, ONNX has no mechanism to express that.

In summary, ONNX's current stance is largely '''implementation-dependent''' for numerical precision. It provides minimal explicit numerical guarantees beyond type constraints. This is why an ONNX model run on two different hardware or runtime libraries can have slightly different results, especially in reduced precision. The burden is on backends to produce results "close enough" to each other that they don't affect the model's decisions in any significant way.

=== 5.2. MatMul, Conv, and Reduction Operators ===

Consider ONNX <code>MatMul</code>: it takes two tensors (often two matrices) and produces their matrix product. In pure math, <math>C_{ij} = \sum_k A_{ik} '' B_{kj}</math>. If A and B are float16, should that sum be computed in float16 or float32? As discussed, ONNX doesn’t specify – it simply says the output is a float16 matrix containing the product. In practice, many backends will ''promote'' the computation: for example, NVIDIA GPUs will do the multiplication and addition in a mixed FP16/FP32 mode by default (because of tensor cores), then convert the final result back to float16. ONNX Runtime on CPU might not support float16 multiplication natively, so it will insert cast ops to float32, perform the MatMul in float32, then cast back to float16<ref name="ref9"/><ref name="ref9"/>. From ONNX’s perspective, as long as the final result is within acceptable error of the “ideal” float16 computation, it’s fine. But note, an “ideal” float16 computation (where every multiply and add is rounded to FP16) might actually be ''less accurate* than what the backend did with FP32. This means that a backend using higher precision can produce a result that is not bit-equal to a pure FP16 implementation – it's actually closer to the true real number result. This is usually considered a good thing (improved accuracy), but it does highlight a lack of consistency: another backend might do everything in pure FP16 and get a slightly different answer.

The same situation occurs for '''convolution''' (<code>Conv</code> op) which is essentially a batched sum of products (like MatMul in sliding window), and for explicit reduction ops like <code>ReduceSum</code>, <code>ReduceMean</code>, etc. If you have an ONNX <code>ReduceSum</code> that sums 1000 float16 values, the spec doesn’t say how to sum them. A GPU kernel might sum in a tree using FP32 accumulators; a naive implementation might sum in order with FP16 and possibly overflow or underflow intermediate steps. Both are allowed as long as they output a float16 result that is “reasonable.” In practice, ONNX Runtime and others take care to avoid obviously bad behavior (like overflow) by using higher precision or by splitting the sum. But again, it’s not mandated by the ONNX standard explicitly.

<math>\mathrm{Softmax}(x)_i = \frac{\exp(x_i)}{\sum_j \exp(x_j)},</math>
typically applied along a specified axis. Directly computing this in float16 can be problematic for large vectors, as discussed in Section 6.3. Indeed, the issue of accumulation precision arises: should the sum <math>\sum_j \exp(x_j)</math> be done in FP16 or can it be in FP32? Many implementations will do it in FP32 even if ''x'' is FP16, because the dynamic range of <math>\exp(x)</math> is large. The ONNX Softmax spec doesn’t detail this, but it expects the result to be a proper probability distribution (summing to 1.0 within normal floating error). A pure FP16 softmax on, say, 1000 elements could sum to something like 0.996 or 1.004 due to rounding, whereas FP32 accumulation would be much closer to 1.000. ONNX likely allows both; the small difference is usually inconsequential, but if it were large (say the FP16 sum overflowed to inf), that would be a correctness issue. A robust backend will prevent that (by using the common max-subtraction trick and higher precision accumulation). The ONNX issue 7072 indeed calls out Softmax as another op where explicit control of accumulation precision would be useful<ref name="ref12"/>.

In summary, '''MatMul, Conv, and reduction ops''' in ONNX are defined mathematically but with ''implicit'' numerical behavior. Implementations commonly use higher precision for intermediate sums especially when input is low precision, but this is not standardized. Consequently, results can differ slightly between implementations. For critical applications, one must be aware of these differences. A model exported to ONNX with float16 weights might perform slightly differently on one runtime vs another if one does all float16 math and the other uses float32 accumulations. Both conform to ONNX. As a user, if you require a certain precision for accumulation, currently you have to ensure it by the way you export or run the model (for example, insert manual cast nodes in the ONNX graph to force certain ops to FP32). We may see future ONNX versions address this with attributes or metadata as mentioned, but for now these ops are effectively “compute with best effort accuracy”.

=== 5.3. Backend and Hardware-Dependent Behavior ===

Because ONNX is implemented by many backends (ONNX Runtime, TensorRT, CoreML, PyTorch JIT, etc.), and these in turn run on varied hardware (CPU, GPU, accelerators), the numeric behavior can be backend-dependent. Some examples:

* '''CPU backends:''' Many CPUs do not have native float16 vector arithmetic (with a few exceptions in recent CPUs). ONNX Runtime’s default CPU execution provider will often upcast float16 to float32, compute, then downcast<ref name="ref9"/><ref name="ref9"/>. This ensures accuracy but means that if you thought you were getting a full float16 pipeline, you’re not – the CPU is essentially ignoring the low precision in compute. On the other hand, x86 CPUs have extended 80-bit precision in x87 registers (though less used now) and 64-bit SIMD registers for doubles; some compiler optimizations might carry intermediate results in higher precision. This could make CPU results slightly different from a GPU’s if, say, a sum is done in 80-bit on CPU versus 32-bit on GPU. Generally, modern compilers avoid such extended precision unless explicitly asked, to improve reproducibility.

* '''GPU backends:''' GPUs often perform summations in parallel using tree reductions. The order of operations can change depending on how threads are scheduled. This means even on the same GPU, summing an array in one launch versus another might not give bit-identical results if the internal parallel reduction algorithm differs (though it should be deterministically the same given the same library version and input, but if you change GPU model or BLAS library, it might differ). Also, GPUs have fused multiply-add (FMA) instructions that do ''(a*b + c)'' in one step with one rounding, which can produce slightly different results than doing the multiply and add separately (which would round twice). If an ONNX backend uses a fused kernel (like a GEMM library) it might yield different low-order bits than a simple loop.

* '''Vendor libraries:''' ONNX Runtime can use vendor BLAS or DNN libraries (MKL, cuDNN, TensorRT, etc.). These libraries often implement their own strategies for numerical stability. For example, NVIDIA’s cuDNN might use FP32 accumulation for FP16 inputs by default. TensorRT (NVIDIA’s inference engine) will sometimes automatically promote precisions or insert scaling to maintain stability<ref name="ref13"/>. On the other hand, if you explicitly request FP16 in TensorRT with no FP32 fallback, it might do the entire op in FP16 and you’d see more error. Some libraries offer knobs: TensorRT allows choosing FP32, FP16, or INT8 precision for each layer, and will use FP32 internally for safety in some cases unless forced otherwise.

* '''Hardware-specific quirks:''' As another example, NVIDIA’s '''TensorFloat-32 (TF32)''' on Ampere GPUs is a precision mode where matrix multiplications use 10-bit mantissa (so about FP16 precision for the multiply) but keep 8-bit exponent (range like FP32). By default, CUDA libraries use TF32 for FP32 matrix multiplies on Ampere to speed them up, unless you disable it. This means if you run an ONNX model with FP32 on an A100 GPU, your matmuls might actually be lower precision than FP32 (though higher than FP16)<ref name="ref6"/>. It’s an example of hardware choosing an intermediate precision for performance. From ONNX’s perspective, the output is still FP32 and (hopefully) within error tolerances, but a user might be surprised that it’s not a “true” FP32 computation internally. If exactness is needed, one has to turn off TF32.

* '''Non-determinism:''' Some backends (especially on GPUs) allow non-deterministic execution for performance. For instance, atomic adds in parallel reductions can produce results that vary run-to-run because the order of accumulation is race-condition-dependent. ONNX’s spec doesn’t cover this explicitly, but it’s something to consider. In ONNX Runtime, there are settings for deterministic computing, but by default it may use all available performance optimizations, even if results vary at the 1e-7 level.

In short, the numerical behavior under ONNX can depend on the combination of backend software and hardware capabilities. The lack of a strict spec for accumulation means implementations have freedom to use clever strategies (higher precision, reordering) to improve accuracy or speed. Usually this is positive (e.g. getting better accuracy than a naive approach). But it could also lead to slight inconsistencies. For example, you might find that exporting a PyTorch model to ONNX and running it with one engine yields outputs slightly different from PyTorch’s – not because ONNX is wrong, but because the summation or convolution was done differently (maybe PyTorch summed in FP32 as well, but maybe not in exactly the same way).

The key takeaway is that ONNX ''by design'' abstracts away these details, which is good for portability but means you need to '''trust the backend''' or verify its precision behavior for sensitive models. If necessary, you can enforce certain behavior by inserting ops (for example, if you want to ensure FP32 accumulation, you could insert a cast to FP32 for inputs of a MatMul and cast the result back to FP16 – this way the ONNX graph itself mandates FP32 compute in between). This of course sacrifices some performance or device-specific magic.

Looking forward, as mentioned, there are proposals to enrich ONNX’s specification regarding precision. This might include explicit flags for “exact summation” or “accumulate in int32” etc., which would make models more self-descriptive but also potentially less portable (if a backend doesn’t support that, what then?). For now, users should be aware of these backend differences. Most of the time, well-engineered ONNX runtimes yield results close enough that they don't affect the application (e.g., classification top-1 remains the same even if low-level sums differ by 1e-5). But for high-stakes numerical computations, testing and maybe adjusting the model or backend settings is prudent.

== Numerical Stability in Deep Learning Models ==

Deep learning computations involve many operations that, while theoretically well-defined, can be sensitive to numerical issues when implemented in finite precision. We'll discuss a few common scenarios in neural networks where numerical stability is a concern, and how precision and accumulation play a role.

=== 6.1. Dot Products and Large-Scale Reductions ===

Neural networks are full of dot products: every fully-connected layer computes <math>\mathbf{y} = W\mathbf{x}</math>, which is a series of inner products between weight vectors and the input. Convolutional layers compute weighted sums of local regions. Attention mechanisms compute large matrix products of query/key/value matrices. All these boil down to summing a lot of terms: <math>\sum_{k=1}^K a_k b_k</math>. When ''K'' is large (it could be in the thousands or more for a single neuron in a fully connected layer of a large model), the potential for floating-point error in that sum increases.

If using FP32, the error may still be negligible in many cases, but in lower precision like FP16, it can become significant. For example, consider a dot product of length 2048 in FP16. Each product is in FP16 (with ~3 decimal digits precision) and if accumulated in FP16, the worst-case error could be on the order of ''2048 × ε_FP16 ≈ 2048 × 0.0009766 ≈ 2'' (i.e., you could lose a couple units in the third decimal place relative accuracy in worst case). Even in random error accumulation, you might see error on the order of <math>\sqrt{2048} \times 0.0009766 \approx 0.044</math> or 4.4% (though this is a rough estimate). That’s quite significant. Fortunately, as discussed, most hardware will accumulate such sums in a higher precision like FP32, dramatically reducing that error (~7 orders of magnitude smaller ''ε''). But if it didn’t, the network’s outputs could get noisy. A fully-connected layer with many inputs might output slightly different results depending on summation order if done in low precision.

'''Large-scale reductions''' also occur in operations like global average pooling (summing all elements of a feature map) or computing the total loss by summing loss over batch samples. If one sums 100,000 values in FP32, the worst-case error could be on the order of 100,000 '' 1e-7 ≈ 1e-2 (1% uncertainty), though typical error would be much less. In FP16, summing 100k values is likely impossible without overflow: as noted, adding ~10^4 moderate numbers can overflow FP16<ref name="ref3"/>. So large reductions ''must* be handled carefully (either in chunks or in higher precision). Frameworks generally do: for example, when performing a sum reduction on GPU, libraries break it into parallel blocks that accumulate in registers (usually 32-bit registers, even if inputs are 16-bit).

Another aspect is that many dot products in neural nets involve values of varying magnitude. If the distribution of terms is such that some are much larger than others, the smaller terms might not contribute much if summed last. Ideally, one might sort by magnitude or use pairwise summation to improve that. Most BLAS libraries effectively do a pairwise strategy due to their parallel structure. So, while worst-case analyses are grim, practical implementations mitigate a lot of it.

However, certain layers can suffer if not handled properly. One famous example is in RNNs or LSTMs: if one uses FP16 naive summation for the gating mechanism, errors could accumulate over many time steps and cause drift or instability. In modern practice, recurrent layers are often run in FP16 with FP32 accumulators to avoid this.

In summary, '''dot products and large reductions need sufficient precision''' or robust algorithms. If you are implementing a custom kernel (say a custom ONNX op) and you have to sum a huge array, you should consider using a technique from Section 3 (pairwise summation, Kahan, or at least double accumulator if possible). Many deep learning computations, by virtue of hardware and library design, already do something like this under the hood. But it's always good to be aware, especially if pushing into lower precision territories.

=== 6.2. Cancellation, Dynamic Range, and Conditioning ===

We introduced ''cancellation'' earlier in the context of summation. In deep learning, cancellation can appear in a few places. One is in the computation of certain norms or variances. For instance, Batch Normalization computes mean and variance of activations. The variance formula <math>\mathrm{Var}(x) = E[x^2] - (E[x])^2</math> is subject to catastrophic cancellation if ''E[x]'' and <math>\sqrt{E[x^2]}</math> are close in magnitude (which can happen if the variance is small compared to the mean). If implemented naively in single precision, you might get a negative variance due to numerical error (which would then be clamped to zero or sqrt would yield NaN). Robust implementations compute variance in a single pass or increase precision. Indeed, in many frameworks, batchnorm accumulation is done in FP32 even if the activations are FP16, precisely to avoid numerical issues in computing mean and variance. ONNX itself doesn’t specify this, but an ONNX backend is likely to do batchnorm stats in higher precision for safety. The ''conditioning'' of the variance calculation is poor when variance is tiny relative to mean, so it's a spot where higher precision is needed.

Another example is in certain loss functions or regularizations. If you subtract two large numbers to get a small difference, you lose precision. Say you have two nearly identical predictions and you subtract them as part of some operation – if done in float16, you might get zero whereas float32 would have a slight difference. This could affect, say, a gradient if that difference was supposed to propagate information. Fortunately, neural nets often avoid directly subtracting large nearly equal numbers; but it can happen. In attention mechanisms, the ''score normalization'' (softmax) subtracts a max to avoid huge exponentials, which is good for range but if multiple scores are close to the max, you subtract numbers that are close. However, those subtractions are not catastrophic in the same way; they're intentional to reduce range and are typically fine in FP32 or even FP16.

'''Dynamic range''' issues are everywhere in deep learning. Activation functions like ReLU and exponential can produce very large or very small values. If subsequent computations don’t account for that, you get infinities or zeros. For example, consider a network output that produces a logit of 100 in FP16. <math>\exp(100)</math> in FP16 overflows to infinity (since FP16 max ~ 6e4 and ''e100 ∼ 3.7e43''). If you then do softmax, that one infinity will dominate the sum and you might get a probability of 1 for that class (or NaNs if not handled). The standard softmax implementation subtracts the max logit, so in this case it would subtract 100 from all logits, making the largest exponent <math>\exp(0)=1</math> and others tiny – this avoids overflow. But what about underflow? If you had a very negative logit, <math>\exp(-100)</math> in FP16 would underflow to 0. In softmax, that just contributes ~0 probability, which is fine (it effectively is zero anyway). Underflow tends to be less dangerous than overflow in inference — underflow usually just means “this probability/neuron is effectively 0,” whereas overflow can poison the results with NaNs or Infs.

'''Conditioning''' refers to how sensitive a function is to input perturbations. Neural networks themselves as functions can be poorly conditioned (e.g., tiny changes in weights cause big changes in output for some architectures), but typically they are designed to be reasonably well-behaved. Certain internal computations like matrix inversion (if one does a pseudoinverse or something for a layer) can be extremely ill-conditioned if the matrix is near singular. That again would need double precision perhaps. In ML, one common ill-conditioned operation is the '''softmax denominator''': <math>\sum_j e^{x_j}</math> can be very large and also very sensitive to changes in the largest ''xj''. However, the softmax is more stable if computed carefully (max trick). Another is '''normalization''': computing <math>\frac{x - \mu}{\sigma}</math> for layer norm or batch norm – if <math>\sigma</math> is very small, then the division blows up noise. In those cases, a small ''ε'' is usually added to <math>\sigma^2</math> to prevent instability (this is a model-level fix, not a numeric one per se, but it acknowledges numeric limits).

In training, '''optimizers''' can suffer from precision issues. For example, the Adam optimizer keeps a moving average of squared gradients; in half precision, these small updates might flush to zero or not change at some point. Some research found that certain adaptive optimizers break in FP16 because the small differences are lost ("Why FP16 training breaks NAdam" etc.). The solution is often to keep the optimizer states in FP32 while doing the bulk of compute in FP16. This again is a form of mixed precision.

From an inference perspective, once the model is trained, the main concerns are avoiding overflow in activations and ensuring summations (like in softmax or reductions) are done with enough precision. The model’s weights are fixed, so we don't have to worry about cumulative error over iterations as in training, just the forward pass.

=== 6.3. Normalization Layers and Softmax ===

'''Normalization layers''' (BatchNorm, LayerNorm, etc.) and '''Softmax''' are specifically worth examining for numerical stability:

* '''Batch Normalization:''' During inference, batchnorm applies an affine transform using precomputed mean and variance (from training). It’s basically: <math>y = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \gamma + \beta</math>. This is a stable operation as long as ''ε'' is provided (to avoid division by zero). Since <math>\mu, \sigma^2</math> are constants in inference, it reduces to linear ops which are fine. The main numerical work (computing mean/variance) happened during training or model calibration, and frameworks usually did that in FP32. So batchnorm at inference time is not a source of instability (it’s just a scale and shift of the data).

* '''Layer Normalization / Group Normalization:''' These compute mean/var on the fly for each sample (and possibly channel group). In inference, they still need to compute these statistics for each forward pass. If implemented in low precision, the variance calculation can suffer cancellation as mentioned. However, it's likely that inference engines compute the mean and variance in FP32 even if inputs are FP16 (especially since these are per-sample operations, not huge batch reductions – it's feasible to do in higher precision). If someone implemented layer norm purely in FP16, they might see issues when the variance is very small. Usually there’s an ''ε'' added which might dominate a tiny variance and thus mitigate catastrophic cancellation (the result might just output something mostly from ''ε''). In summary, normalization layers are usually fine as long as the implementation uses enough precision for the summation of squares.

* '''Softmax:''' Softmax is notorious if done improperly. The safe algorithm: subtract the maximum input from all inputs (to bring the largest exponent to 1.0), exponentiate, sum them up, divide each exponent by the sum. The max subtraction handles overflow in <math>\exp()</math>. But what about the sum? As discussed, if there are ''N'' elements in the softmax, the sum is at most ''N'' (when all inputs equal the max). In classification, ''N'' might be the number of classes. For ImageNet, ''N=1000'', so worst sum ~1000 (which is fine in FP16, as 1000 << 65504). For an extreme case like a language model, ''N'' could be 50,000 (vocab size). Sum could be 50,000 in worst case (everyone equal probability). 50,000 is still below 65504, so actually an FP16 sum would ''not'' overflow in that scenario, and the rounding error on a sum of 50k identical terms would be maybe a few ULPs (which at 50k magnitude might be ~0.5 difference worst-case). So maybe softmax summation in FP16 is not catastrophic for output probabilities – you’d get essentially 1.0 after normalization instead of 0.99999 in some case. Usually fine.

:The bigger issue is if one or a few terms dominate the softmax. If one term is much larger, the others underflow to 0 after the max subtraction. That’s fine: you get a one-hot distribution essentially. If the model expected a softer distribution, that could be a problem. But if the logits were really that far apart, even FP32 would give nearly one-hot. So not really a precision issue, more of a model property.

:However, the '''stability during training''' is a big issue. The GitHub thread we saw (ColossalAI) mentioned training a 130B model where FP16 softmax in attention caused training to occasionally blow up<ref name="ref14"/><ref name="ref14"/>. They observed attention scores in some heads ranging up to 1e4 or as low as 1e-3<ref name="ref14"/>. 1e4 exponent in FP16 is definitely Inf (because <math>\exp(10^4)</math> is astronomically huge, but even exp(100) is inf in half). The method CogView introduced (Precision Bottleneck Relaxation) involved subtracting the max ''per head'' to reduce range<ref name="ref14"/>—which is basically the softmax max trick at a finer granularity or something similar. In the end, they found '''the simplest fix was to compute softmax in FP32''' for those layers<ref name="ref14"/>. They reported that it had negligible speed impact but significantly improved stability<ref name="ref14"/>. This highlights a general principle: certain operations like softmax (and sometimes matrix multiplications in attention) benefit from higher precision, and using FP32 in a few critical spots can save a mixed-precision training from crashing. In inference, using FP32 for softmax is also a safeguard if one is uncertain. It costs little because softmax is not a heavy compute compared to matrix multiplies.

:ONNX does have a Softmax operator; if the input is float16, a backend might do exactly what they described: internally cast to float32 for the softmax computation, then cast output back to float16. Some inference engines (like DeepSpeed’s inference or others) keep attention softmax in FP32 even if rest is FP16, for robustness<ref name="ref14"/>.

Apart from softmax, other places where exponentials occur are in loss functions like cross-entropy (which uses log-softmax internally), or in activation functions like Sigmoid and Tanh (which are basically scaled logistic functions). Sigmoid(''x)=1/(1+e-x)''. If ''x'' is moderately large positive, ''e-x'' underflows to 0 in FP16 if <math>x>~11</math>. That gives a sigmoid output of 1.0 exactly. In FP32, ''e-x'' for ''x=11'' is about ''5e-5'', not underflow, so sigmoid ~0.99995. So FP16 will saturate some outputs to exactly 0 or 1 sooner than FP32 would. This can slightly change behaviors (e.g., a neuron might go to full saturation a bit early). Usually, this isn’t a huge deal, but it could make gradients zero earlier. In inference, we don’t care about gradients, just the output value. Getting exactly 1.0 vs 0.99995 probably has no practical difference for a classification output. But one should be aware that '''activation functions can saturate faster''' in lower precision due to limited range or precision. ReLU has no issue (it’s linear or 0), but exponential-based activations do. In practice, again, frameworks often simply compute those in the given precision and accept the slight difference. If it were a problem, one could do those in FP32 as well.

'''Summary:''' Numerical stability in inference is maintained by using robust computation methods: subtracting maxima in softmax/LogSumExp, adding epsilon in denominators for normalization, and using higher precision for summations or critical steps. Most of these are built into the algorithms. ONNX models assume these standard practices (they don't need to explicitly do them in the graph because the backend inherently will for ops like Softmax). It's wise when building or exporting models to ONNX to test them in the target precision and ensure there are no surprises (like an output becomes NaN or Inf due to precision). Usually if training was done in mixed precision, the model will have been vetted in that environment already.

== Performance vs. Accuracy Trade-offs ==

There is an inherent trade-off between numerical precision and computational performance (speed, memory usage) in AI systems. Choosing lower precision can dramatically increase throughput and reduce memory, but it may introduce arithmetic errors. Engineers must balance these considerations:

=== 7.1. Throughput, Memory Bandwidth, and Precision ===

'''Throughput (speed)''' is often higher for lower-precision operations. Modern hardware can execute more low-precision ops in parallel than high-precision ops. For example, an NVIDIA V100 GPU can perform 2× more half-precision FLOPs than single-precision per clock, and newer architectures with Tensor Cores can do 8× more (since a Tensor Core warp operation might perform 64 FP16 FMAs per cycle vs 16 FP32). Similarly, int8 operations can be even faster and more energy efficient. Lower precision numbers also use less memory bandwidth – fetching 16-bit data is twice as fast as 32-bit data from memory (assuming the same bus width), which can relieve memory bottlenecks. Many deep learning workloads are memory-bound, so halving the data size can nearly double throughput if compute is not the limiting factor. This is a huge incentive to use lower precision.

'''Memory and storage''': Using FP16 or int8 weights reduces model size, which is important for deploying large models to memory-limited environments (like edge devices or GPUs with limited VRAM). It also means you can fit larger batch sizes in memory, or more models on a chip, etc. There is also a cascade effect: smaller data means better cache utilization, which further improves performance.

However, '''accuracy''' suffers if precision is too low. We have discussed how rounding error and limited range can cause deviations in outputs. The question is: do those deviations matter for the application? Often for neural networks, a small amount of noise or error in intermediate calculations does not change the final prediction. Neural nets are somewhat error-tolerant due to their redundant distributed representations. This is why quantization and FP16 techniques can work at all. The final model accuracy (e.g. top-1 accuracy on ImageNet) might drop only 0.5% or less when using FP16 instead of FP32, or int8 instead of FP16, if done properly. For many use-cases, this loss is acceptable given the speed gains.

If one goes too low (like naive FP8 or int4 without special care), accuracy might drop severely – perhaps making the model unusable. Thus the trade-off: as precision decreases, performance increases but at some point accuracy degrades beyond an acceptable threshold. Engineers aim to find the lowest precision that still preserves accuracy.

There is also the consideration of '''accumulation strategy''' affecting performance: for instance, Kahan summation (compensated) is more accurate but ~4× slower than naive summation. If you need that accuracy, you pay a performance cost. In many AI inference scenarios, the slight extra accuracy from something like Kahan isn't needed, so it’s not used. But in a scenario where you sum millions of elements for a final result that is thresholded, maybe you do need it.

Another dimension is '''determinism vs speed'''. Enforcing a deterministic reduction order (for reproducibility) might require disabling some parallelism, which could slow things down. If exact repeatability is critical (like certain scientific or financial applications of neural nets), one might sacrifice some performance to ensure the summation happens the same way every time. ONNX allows both modes depending on the runtime.

From a memory perspective, if precision is higher than needed, you waste memory. For example, storing intermediate activations in FP32 when they could be FP16 uses 2× memory, which might mean lower batch size or more memory traffic. So there's strong motivation to keep things as low-precision as possible ''except'' where accuracy truly demands.

In summary, the trade-off often manifests as ''diminishing returns'': FP64 vs FP32 – huge slowdown for usually negligible accuracy gain in ML; FP32 vs FP16 – moderate speedup for usually minor accuracy hit; FP16 vs int8 – another big speedup but maybe more accuracy loss unless calibration is done; int8 vs int4 – speed gain but often big accuracy drop that might not be worth it for current models.

Engineers use techniques like '''mixed precision''' to target high precision only where needed. For example, run most of the model in FP16, but keep a few sensitive layers in FP32. This yields most of the performance benefits while ensuring critical parts are accurate. We already saw softmax as one example. Another might be final output layer: sometimes using FP32 for the logits and softmax can be safer.

Hardware is also trending to support mixed modes – like FP8 Tensor Cores accumulate in FP32, etc. So hardware tries to give accuracy cheaply.

=== 7.2. Hardware Accelerators and Tensor Cores ===

'''Hardware accelerators''' (GPUs, TPUs, FPGAs, ASICs like GraphCore IPU, etc.) have specific support for certain precisions. Leveraging these can provide massive speedups:

* '''NVIDIA Tensor Cores:''' These started in Volta (V100) with support for 4x4 matrix multiply-adds on FP16 inputs with FP32 accumulation<ref name="ref6"/>. Using Tensor Cores can give up to 8x throughput compared to using regular FP32 cores. But to use them, the data needs to be FP16 (or now BF16 or FP8 on newer GPUs) and aligned to certain dimensions (multiples of 8 etc. in matrix sizes, which frameworks handle by padding if needed). If an ONNX model is all FP32, by default it would not use Tensor Cores unless the runtime automatically converts some ops to mixed precision (which some do via a flag or environment variable enabling TF32 or FP16). The introduction of '''TensorFloat-32 (TF32)''' in Ampere GPUs is essentially a compromise: treat FP32 inputs as two 16-bit significand halves (10-bit mantissa actually) so that it can run on Tensor Cores, giving ~8x speed, while still outputting FP32. TF32 has about the precision of FP16 but range of FP32, and it was chosen because it typically doesn’t hurt convnet accuracy. ONNX Runtime or others on Ampere will use TF32 by default for conv and matmul unless disabled, giving a free speed boost at minimal accuracy cost. If exact FP32 is needed, users have to turn it off (sacrificing speed).

* '''TPU (Google):''' TPUs have native support for BF16 matrix ops with FP32 accumulators. They were designed specifically for the mixed precision approach in training. For inference, TPUs also support int8 and other quantized ops (e.g., via Edge TPU for lower-power). If exporting ONNX to run on a TPU (via some bridge), one might get automatic BF16 usage. BF16 has the advantage of no loss in range, so TPUs can more straightforwardly run models in BF16 without modifications.

* '''FPGAs and ASICs:''' Some custom accelerators might use fixed-point or block floating-point arithmetic for efficiency. They often treat the network in quantized terms. The ONNX quantization specification (QuantizeLinear/DequantizeLinear ops) exists for representing quantized models (INT8 etc.), which is another domain beyond floating-point, but it overlaps as a precision issue. In those systems, accumulators for INT8 × INT8 multiplies are usually 32-bit (to avoid overflow for dot products up to certain lengths). The trade-off there is between integer vs float representation, which can give big speedups on CPUs/DSAs via SIMD instructions (many CPUs have vector int8 multiply-add instructions now for inference).

* '''Memory bandwidth on hardware:''' Some accelerators include special high-bandwidth memory or cache specifically for lower precision. For example, an accelerator might fetch two FP16 values in one 32-bit memory transaction – effectively doubling memory bandwidth for half precision. On some hardware, using FP16 can even reduce memory ''latency'' if multiple values can be packed.

* '''Parallelism and Vectorization:''' Many hardware units process data in fixed vector widths (e.g., 256-bit registers). That could be 8 floats at once or 16 half-floats at once in the same 256 bits. If your code is in FP16, you might use the full vector width (16 values) whereas FP32 might only use half (8 values). This means FP16 can make better use of vector units. Similarly, matrix units often have fixed tile sizes, and smaller data types allow larger tiles or more tiles concurrently.

* '''Energy efficiency:''' Lower precision often means less energy per operation, which on large deployments (data centers or battery-powered devices) is a significant advantage. This isn't directly performance, but it's an important trade: you might choose a slightly lower precision to save power even if accuracy drops a tiny bit.

* '''Software overhead:''' Using mixed precision on hardware involves some overhead of casting and moving data between precisions. For instance, if you have to frequently cast FP16 to FP32 and back, that adds instructions and memory copies. A well-designed accelerator (like GPUs with unified memory for FP16/32 or special hardware casting) minimize this. Still, the runtime might insert extra operations (as we saw, ONNX Runtime CPU inserted many Cast nodes converting FP16 to FP32 and back<ref name="ref9"/>, which actually ''slowed down'' that FP16 model by 4x vs FP32 on a CPU, because the CPU had no native FP16). So ironically, on some hardware, using lower precision in ONNX might degrade performance if not supported (the CPU case). Always ensure the target hardware actually supports the low precision you plan to use; otherwise, you'll pay overhead for emulation.

To illustrate trade-off: Suppose you have a model that achieves 99% accuracy in FP32 and drops to 98.5% in FP16. If FP16 runs 2x faster, many would accept the 0.5% accuracy drop for double throughput. If int8 runs 4x faster but accuracy drops to 95%, that might be too much drop for some applications, but acceptable for others that value speed. It's application-dependent.

In critical applications, one might maintain higher precision accumulators or run certain validation steps in high precision to ensure nothing drifted. In research, sometimes a model is trained in mixed precision but validated in double to ensure any small numeric differences don’t influence the reported result (rare, but for scientific computing integration maybe).

In the ONNX world, performance-vs-accuracy is often managed via '''Execution Providers''' (EPs) that target specific hardware. For example, the TensorRT EP in ONNX Runtime can take an ONNX model and run it using NVIDIA's high-performance inference library, which will use FP16 or INT8 if allowed, to maximize speed. The user can configure the EP with precision flags. If the accuracy drop is too high, one might restrict it to FP32. If it's fine, allow FP16. Tools exist to measure the accuracy difference (like calibrating int8 quantization).

To conclude, the trade-offs revolve around picking the lowest precision that yields acceptable accuracy, and utilizing hardware features fully. ONNX being a high-level spec abstracts this, but it's ultimately up to the deployment to decide and sometimes up to the exporter to incorporate quantization. ONNX’s role is to be able to ''represent'' these choices (they have added quantized types, FP8 types, etc., to allow the graph to include those when needed).

== Best Practices and Design Guidelines ==

Given the above insights, here are some best practices and guidelines for dealing with numerical precision and accumulators in AI inference (and training) design:

=== 8.1. Choosing an Accumulator Precision ===

* '''Match precision to data length and range:''' If an operation accumulates a large number of terms, consider using a higher precision accumulator than the input data. For example, for summing >1000 values of type FP16, an FP32 accumulator is recommended to avoid large rounding error or overflow. If summing > million values of FP32, consider double (FP64) accumulation if high accuracy is needed (though this is rare in DL). ''Rule of thumb:'' if <math>N \cdot \varepsilon</math> is not tiny (say >1e-3), you may benefit from higher precision. For FP16 (''ε ≈ 1e-3''), that threshold N is a few hundred. For FP32 (''ε ≈ 1e-7''), N would be in the millions before 1e-3 error.

* '''Use higher precision in critical layers:''' Identify if certain layers of the model are more sensitive to precision. For instance, the output layer or a layer that feeds into a very sharp decision boundary might require more precision. If a small change there could flip the prediction, you don't want excessive numerical error. Using FP32 for logits and softmax is a common practice to maintain classification confidence stability.

* '''Follow hardware defaults, but override if needed:''' Many hardware accelerators have sensible defaults (like accumulate in FP32 for tensor ops). It's usually best to use those defaults as they were chosen to balance accuracy and speed. Only override if you have evidence of a problem. For example, if using ONNX Runtime on CPU, you discovered it upcasts to float for you – great. But if using some custom accelerator that does pure FP16, you might manually insert casts in the ONNX graph around certain ops to force higher precision on those ops.

* '''Maintain precision for gradient accumulation (training):''' This is more training-oriented, but worth noting: when summing gradients over batches or updating weights, use higher precision (FP32 or even 64) for those accumulations. This ensures training doesn't diverge due to numeric error. Many frameworks keep a "master copy" of weights in FP32 when training in FP16<ref name="ref2"/>. For inference, this translates to: any running statistics or counters in the model (like if you had a running average inside the model) should probably be in higher precision to avoid drift.

* '''Be mindful of intermediate cast costs:''' If you do decide to cast to higher precision for an operation in ONNX, remember it incurs memory copies and compute overhead. Make sure it's worth it. For example, if you cast a huge tensor to double, that could slow things drastically. Usually FP16->FP32 is fine (hardware handles it well), but FP32->FP64 might be expensive on GPUs.

=== 8.2. Safe Reduction Lengths and Heuristics ===

* '''Chunk long reductions:''' If you must do a very long sum in low precision (due to memory constraints or hardware), chunk the sum into parts, sum each to a higher precision local accumulator, then sum the partial sums. This is essentially pairwise summation. Many libraries use a fixed chunk size heuristic (e.g., sum in blocks of 256 in registers, then add those up). This bounds the error growth. If implementing manually, you can choose a chunk size such that chunk_length '' ε is small (like 256 '' 1e-3 = 0.256 for FP16, hmm maybe choose 100).

* '''Sort by magnitude if possible:''' In some cases (not typical in DL, but possible in post-processing), if you have numbers varying by orders of magnitude, summing from smallest to largest improves accuracy. It prevents small ones from being swallowed by big ones. This is more relevant if you output a sum of mixed-scale values (like some algorithm mixing big and small contributions). It’s not always feasible in real-time inference to sort, but keep in mind.

* '''Use compensation for critical sums:''' If you find a particular sum or difference in your inference computation is crucial (like computing a small residual from two close numbers), consider using Kahan summation or another compensated scheme just for that part. For example, if you had an ONNX custom function that needs to sum a lot of almost cancelling terms, you might implement it with compensation internally.

* '''Heuristics for FP8/low-bit:''' When using extremely low precision like FP8, adopt the recommended practices: use scaling (don't feed raw data into FP8, normalize it first), and perhaps only use FP8 on portions of the network that are empirically robust. The industry is still establishing heuristics, but one might be: use FP8 for matrix multiplications, but accumulate in FP16 or FP32; keep normalization in FP16; use FP16 for first and last layer maybe. This is evolving, so keep an eye on literature.

* '''Keep an eye on sum length in softmax and normalization:''' If you have extremely large class counts or group sizes (e.g., a softmax over 1e6 items, or layer norm over tens of thousands of features), realize this is a giant reduction. You may want to ensure that reduction is done in higher precision. ONNX Softmax by default will likely not handle a million-class case in FP16 well (the sum would overflow since 1e6 > 65504). So either do that on CPU in FP32 or break it down (could do hierarchical softmax, etc.). This is a corner case, but relevant for huge vocab language models. In practice, vocab ~50k is fine as we said.

* '''Test numeric differences:''' A practical tip: after quantizing or reducing precision, run some test inputs through both the high-precision model and low-precision model. Compare outputs (and maybe some intermediate signals if possible). If differences are consistently small relative to the application tolerance, you're good. If you see occasional large differences or instabilities (like outputs become NaN or Inf in low precision), that's a red flag. Use that testing to guide where you might need to increase precision or change algorithms.

=== 8.3. When Higher Precision Is Required ===

* '''Ill-conditioned calculations:''' If your model or pipeline performs any operation known to be ill-conditioned (matrix inversion, subtraction of nearly equal quantities, computing a tiny difference of large numbers), you likely need higher precision for that part. For example, if your ONNX model includes a custom op to solve a linear system, use double precision in that op if possible (or at least float with iterative refinement). In ML, this might occur in something like a physics-informed neural network or a differentiable simulation within the model.

* '''Strict accuracy requirements:''' Some applications (medical, aerospace, finance) might have strict tolerances. If an inference model’s output needs to be correct to 5 decimal places, you probably cannot use FP16, which has only 3-4 decimal digits precision. FP32 might be barely enough (~7 digits). FP64 might be warranted to guarantee that level of accuracy in worst case. Most deep learning applications (like image classification) don't demand that level of numeric precision in outputs – as long as the top class is correct, it doesn't matter if its probability was 0.901 vs 0.899. But if you're doing something like summing probabilities to make a risk estimate or doing a cumulative sum over many predictions, be careful to accumulate precisely.

* '''Large sums or products in post-processing:''' Sometimes the neural net is fine, but the ''post-processing'' of results might need precision. E.g., summing up probabilities of many events, or computing confidence intervals, etc. Do those calculations in double if needed outside the model.

* '''Preventing drift in long sequences:''' For sequential models (RNNs processing thousands of steps), rounding error can accumulate over steps. If each step has a small error, after 10000 steps it could add up. In such cases, using a higher precision for the state might reduce drift. Some practitioners use FP64 for the hidden state accumulation in very long sequence processing to maintain numerical stability (though this is not common due to speed loss). Alternatively, periodically renormalize or reset small errors.

* '''Simulations or differential equations:''' If your ONNX model involves simulating something (like a differential equation solver inside a model, or an ODE/RNN hybrid), those often need double precision to be stable for long durations. Recognize those situations and allocate precision accordingly.

* '''Verification and debugging:''' Use higher precision as a tool to verify correctness. For example, run your model in FP64 (if possible) and compare to FP32 to see if there are differences. If not, FP32 is fine. If yes, you might find a particular calculation losing too much info in FP32, implying maybe you should keep it in FP64 in production as well. This is rare in inference but can be part of rigorous validation.

In ONNX, using higher precision may involve casting to <code>tensor(double)</code> for certain ops. Not all runtimes support double on GPU though (some do via emulation or just slower path). ONNX CPU does support double for general ops. So be mindful of the target.

Finally, '''document your precision choices'''. If you deliver an ONNX model that uses mixed types (float16 inputs, float32 accumulations, int8 weights, etc.), ensure that is clear to whoever maintains or uses it. Precision issues can be sneaky, so clarity helps.

== Limitations and Open Problems ==

Despite advances in handling numerical precision, there remain limitations and open challenges in this domain:

* '''Lack of Precision Specification in ONNX:''' As noted, ONNX currently has no way to formally specify that an op ''should'' use a higher precision internally. This can lead to nondeterministic differences across platforms<ref name="ref12"/>. While proposals exist to add attributes for this<ref name="ref12"/>, it is not yet standard. Thus an ONNX model might behave slightly differently on two compliant backends, and there's no built-in mechanism to ensure identical results. This is a limitation for applications requiring cross-platform consistency. Solving it may involve extending ONNX (e.g., an attribute like <code>accumulatePrecision="FLOAT32"</code> for MatMul) or tightening the spec, but those come with backward compatibility and implementation burden concerns.

* '''Reproducibility and Determinism:''' Even with the same precision, parallel computation can introduce nondeterminism (due to different summation orders). Ensuring bit-identical results across runs (important for some sensitive domains) is non-trivial. Frameworks often provide "deterministic mode" which might use fixed ordering at cost of speed. In ONNX, there's no global flag for this; it's up to the runtime. More research could be done on algorithms that produce bit-wise identical results independent of parallel execution order (perhaps using techniques like pairwise summation or consistent partitioning). Stochastic rounding is another area: sometimes using probabilistic rounding can reduce bias in accumulated error, but it introduces randomness – reconciling that with reproducibility is tricky.

* '''Ultra-low Precision (FP8 and below):''' While FP8 is being explored, going to 4-bit floats or mixed schemes is an open challenge. The OpenAI community is investigating 4-bit (quantization-wise) for LLMs with some success, but a true 4-bit float (like E3M1) would be extremely limited. Perhaps hybrid schemes (like block floating point, log scales, etc.) will come. ONNX already has some support (they mention 4-bit in documentation placeholders<ref name="ref10"/>), but it's nascent. Research into training techniques, calibration, and new number formats (like '''posit numbers''' or tapering precision floats) is ongoing. If a new format proves useful, ONNX will need to adapt to represent it.

* '''Quantization Noise and Model Robustness:''' Reducing precision can sometimes expose models to issues like adversarial vulnerability or fragility – e.g., a slight perturbation might flip a decision if the margin was small and precision loss nudged it. Ensuring model robustness under quantization is an open problem. It might require re-training the model with quantization in the loop (quantization-aware training) or designing models that have large "margin" between classes to tolerate quantization noise.

* '''Dynamic range during training vs inference:''' In training, activations can have wider distributions (especially early in training). Models often learn to constrain their ranges (via activation functions saturating or normalization), which makes inference more stable in lower precision. But if one tries to train in FP8 from scratch, it's very hard. This gap between what's possible in a post-trained inference model vs training is an open area – how to train directly in low precision without significant accuracy loss. Solving that could simplify workflows (no need for high-precision training then quantize; just train low-prec end to end). Techniques like loss scaling, gradient clipping, etc., are partial solutions. The development of optimizers that are resilient to low precision is ongoing.

* '''Hardware variability:''' As new hardware emerges (with new numeric capabilities or quirks), ensuring that ONNX models run consistently is an ongoing task. For instance, one device might implement a fused op that reduces rounding error, another does it separately. Even something like convolution might use a different algorithm (FFT-based vs direct) which has different numeric properties. Open problem: could ONNX allow specifying numeric tolerances per op or preferred algorithms? That may be overkill for most, but for some HPC ML use cases, maybe desired.

* '''Testing and validation of numerics:''' There isn't a standardized set of "numeric compliance" tests for ONNX runtimes beyond simple correctness. Perhaps in the future, test suites might include challenging numeric scenarios (large sums, nearly singular matrices) to ensure a runtime handles them gracefully (maybe by switching precision). Right now, it's mostly on the user to test their model under expected conditions.

* '''Combining quantization with floating point:''' Many production models use a mix (e.g., int8 for some layers, FP16 for others). ONNX can represent this (with QuantizeLinear ops or MixedPrecision through casting), but tooling to automatically find the optimal mix is still developing. It's somewhat trial-and-error or using heuristics. An open question is how to automate precision assignment: given a model and a target hardware, how to automatically choose which layers can be int8, which should be FP16, etc., to meet an accuracy target. This crosses into NAS (neural architecture search) territory or automated quantization.

* '''Numerical debugging tools:''' When a model goes wrong due to precision, it's often hard to pinpoint where. Better tooling to analyze numerical error layer by layer could be useful. For instance, one could run a model in double and float and diff the intermediate results to see where the largest divergence happens. Doing this systematically for big nets is an open tooling challenge. Some frameworks have hooks for this, but not standardized.

In conclusion, while we've come far with using lower precision in AI without sacrificing much accuracy, the landscape is always evolving. The key open problem is how to push precision lower (for efficiency gains) without crossing the threshold where accuracy falls off a cliff. Each new generation of hardware and techniques (like FP8, novel quantization schemes) extends that boundary, but also brings new questions on how to manage the numerical errors. ONNX as an interchange will need to keep pace, possibly by adding more expressive power for these numeric considerations.

== See Also ==

* '''IEEE 754 Floating-Point''' – The standard defining binary16, binary32, binary64 formats and rounding behavior.
* '''Floating-Point Arithmetic''' – General topic on how floating-point math works, common issues (overflow, underflow, cancellation).
* '''Kahan Summation Algorithm''' – Algorithm for compensated summation to reduce numerical error<ref name="ref4"/>.
* '''BFloat16''' – A short-format float with 8-bit exponent, used in TPUs and some GPUs for training networks.
* '''Mixed Precision Training''' – Technique of using FP16 (or BF16) along with FP32 accumulators to accelerate training<ref name="ref2"/>.
* '''Quantization in Deep Learning''' – Converting networks to lower bit integer arithmetic for inference (INT8 quantization and calibration techniques).
* '''ONNX Operator Specification''' – Official docs for ONNX ops, which describe their mathematical function but not the required numeric precision (a point of potential extension in future).
* '''CUDA Tensor Cores & TF32''' – NVIDIA documentation on how tensor cores perform mixed precision and the TF32 format for accelerating FP32.
* '''Pairwise Summation''' – Summation algorithm with logN error growth<ref name="ref5"/>, used in many numerical libraries to improve accuracy of reductions.
* '''Softmax Stability''' – Articles on numerically stable softmax and log-sum-exp implementations to avoid overflow.
* '''Performance Tuning for Low Precision''' – Guides (e.g., NVIDIA or Intel whitepapers) on using low precision and the trade-offs involved in various hardware.

<references>
<ref name="ref1">Accelerate and simplify Scikit-learn model inference with ONNX Runtime - Microsoft Open Source Blog — opensource.microsoft.com — https://opensource.microsoft.com/blog/2020/12/17/accelerate-simplify-scikit-learn-model-inference-onnx-runtime</ref>
<ref name="ref2">Floating Point Precision: Understanding FP64, FP32, and FP16 in Large Language Models - DEV Community — dev.to — https://dev.to/lukehinds/floating-point-precision-understanding-fp64-fp32-and-fp16-in-large-language-models-3gk6</ref>
<ref name="ref3">Floating Point Precision and Its Limitations | by Umair Akbar | Medium — akbu.medium.com — https://akbu.medium.com/floating-point-precision-and-its-limitations-cfb7247d7789</ref>
<ref name="ref4">Kahan summation algorithm - Wikipedia — en.wikipedia.org — https://en.wikipedia.org/wiki/Kahan_summation_algorithm</ref>
<ref name="ref5">Pairwise summation - Wikipedia — en.wikipedia.org — https://en.wikipedia.org/wiki/Pairwise_summation</ref>
<ref name="ref6">Numerical behavior of NVIDIA tensor cores - PMC — pmc.ncbi.nlm.nih.gov — https://pmc.ncbi.nlm.nih.gov/articles/PMC7959640/</ref>
<ref name="ref7">Mixed precision and performance-accuracy tuning (neuron-cc) — AWS Neuron Documentation — awsdocs-neuron.readthedocs-hosted.com — https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/appnotes/neuron-cc/mixed-precision.html</ref>
<ref name="ref8">Create Float16 and Mixed Precision Models - ONNX Runtime — onnxruntime.ai — https://onnxruntime.ai/docs/performance/model-optimizations/float16.html</ref>
<ref name="ref9">[Performance] ONNX FP16 model is having performance bottle neck when compared to FP32 variant · Issue #25824 · microsoft/onnxruntime · GitHub — github.com — https://github.com/microsoft/onnxruntime/issues/25824</ref>
<ref name="ref10">Float stored in 8 bits - ONNX 1.21.0 documentation — onnx.ai — https://onnx.ai/onnx/technical/float8.html</ref>
<ref name="ref11">[2209.05433] FP8 Formats for Deep Learning — arxiv.org — https://arxiv.org/abs/2209.05433</ref>
<ref name="ref12">Should MatMul op allow specification of accumulation precision? · Issue #7072 · onnx/onnx · GitHub — github.com — https://github.com/onnx/onnx/issues/7072</ref>
<ref name="ref13">Quantize ONNX models | onnxruntime — onnxruntime.ai — https://onnxruntime.ai/docs/performance/model-optimizations/quantization.html</ref>
<ref name="ref14">Keep Attention Softmax FP32 during FP16/ZeRO Training · Issue #1485 · hpcaitech/ColossalAI · GitHub — github.com — https://github.com/hpcaitech/ColossalAI/issues/1485</ref>
</references>

=== 5.3. Backend and Hardware-Dependent Behavior ===

Because ONNX is implemented by many backends (ONNX Runtime, TensorRT, CoreML, PyTorch JIT, etc.), and these in turn run on varied hardware (CPU, GPU, accelerators), the numeric behavior can be backend-dependent. Some examples:

* '''CPU backends:''' Many CPUs do not have native float16 vector arithmetic (with a few exceptions in recent CPUs). ONNX Runtime’s default CPU execution provider will often upcast float16 to float32, compute, then downcast<ref name="ref9"/><ref name="ref9"/>. This ensures accuracy but means that if you thought you were getting a full float16 pipeline, you’re not – the CPU is essentially ignoring the low precision in compute. On the other hand, x86 CPUs have extended 80-bit precision in x87 registers (though less used now) and 64-bit SIMD registers for doubles; some compiler optimizations might carry intermediate results in higher precision. This could make CPU results slightly different from a GPU’s if, say, a sum is done in 80-bit on CPU versus 32-bit on GPU. Generally, modern compilers avoid such extended precision unless explicitly asked, to improve reproducibility.

* '''GPU backends:''' GPUs often perform summations in parallel using tree reductions. The order of operations can change depending on how threads are scheduled. This means even on the same GPU, summing an array in one launch versus another might not give bit-identical results if the internal parallel reduction algorithm differs (though it should be deterministically the same given the same library version and input, but if you change GPU model or BLAS library, it might differ). Also, GPUs have fused multiply-add (FMA) instructions that do <math>(a*b + c)</math> in one step with one rounding, which can produce slightly different results than doing the multiply and add separately (which would round twice). If an ONNX backend uses a fused kernel (like a GEMM library) it might yield different low-order bits than a simple loop.

* '''Vendor libraries:''' ONNX Runtime can use vendor BLAS or DNN libraries (MKL, cuDNN, TensorRT, etc.). These libraries often implement their own strategies for numerical stability. For example, NVIDIA’s cuDNN might use FP32 accumulation for FP16 inputs by default. TensorRT (NVIDIA’s inference engine) will sometimes automatically promote precisions or insert scaling to maintain stability<ref name="ref13"/>. On the other hand, if you explicitly request FP16 in TensorRT with no FP32 fallback, it might do the entire op in FP16 and you’d see more error. Some libraries offer knobs: TensorRT allows choosing FP32, FP16, or INT8 precision for each layer, and will use FP32 internally for safety in some cases unless forced otherwise.

* '''Hardware-specific quirks:''' As another example, NVIDIA’s '''TensorFloat-32 (TF32)''' on Ampere GPUs is a precision mode where matrix multiplications use 10-bit mantissa (so about FP16 precision for the multiply) but keep 8-bit exponent (range like FP32). By default, CUDA libraries use TF32 for FP32 matrix multiplies on Ampere to speed them up, unless you disable it. This means if you run an ONNX model with FP32 on an A100 GPU, your matmuls might actually be lower precision than FP32 (though higher than FP16)<ref name="ref6"/>. It’s an example of hardware choosing an intermediate precision for performance. From ONNX’s perspective, the output is still FP32 and (hopefully) within error tolerances, but a user might be surprised that it’s not a “true” FP32 computation internally. If exactness is needed, one has to turn off TF32.

* '''Non-determinism:''' Some backends (especially on GPUs) allow non-deterministic execution for performance. For instance, atomic adds in parallel reductions can produce results that vary run-to-run because the order of accumulation is race-condition-dependent. ONNX’s spec doesn’t cover this explicitly, but it’s something to consider. In ONNX Runtime, there are settings for deterministic computing, but by default it may use all available performance optimizations, even if results vary at the 1e-7 level.

In short, the numerical behavior under ONNX can depend on the combination of backend software and hardware capabilities. The lack of a strict spec for accumulation means implementations have freedom to use clever strategies (higher precision, reordering) to improve accuracy or speed. Usually this is positive (e.g. getting better accuracy than a naive approach). But it could also lead to slight inconsistencies. For example, you might find that exporting a PyTorch model to ONNX and running it with one engine yields outputs slightly different from PyTorch’s – not because ONNX is wrong, but because the summation or convolution was done differently (maybe PyTorch summed in FP32 as well, but maybe not in exactly the same way).

The key takeaway is that ONNX ''by design'' abstracts away these details, which is good for portability but means you need to '''trust the backend''' or verify its precision behavior for sensitive models. If necessary, you can enforce certain behavior by inserting ops (for example, if you want to ensure FP32 accumulation, you could insert a cast to FP32 for inputs of a MatMul and cast the result back to FP16 – this way the ONNX graph itself mandates FP32 compute in between). This of course sacrifices some performance or device-specific magic.

Looking forward, as mentioned, there are proposals to enrich ONNX’s specification regarding precision. This might include explicit flags for “exact summation” or “accumulate in int32” etc., which would make models more self-descriptive but also potentially less portable (if a backend doesn’t support that, what then?). For now, users should be aware of these backend differences. Most of the time, well-engineered ONNX runtimes yield results close enough that they don't affect the application (e.g., classification top-1 remains the same even if low-level sums differ by 1e-5). But for high-stakes numerical computations, testing and maybe adjusting the model or backend settings is prudent.

== Numerical Stability in Deep Learning Models ==

Deep learning computations involve many operations that, while theoretically well-defined, can be sensitive to numerical issues when implemented in finite precision. We'll discuss a few common scenarios in neural networks where numerical stability is a concern, and how precision and accumulation play a role.

=== 6.1. Dot Products and Large-Scale Reductions ===

Neural networks are full of dot products: every fully-connected layer computes <math>\mathbf{y} = W\mathbf{x}</math>, which is a series of inner products between weight vectors and the input. Convolutional layers compute weighted sums of local regions. Attention mechanisms compute large matrix products of query/key/value matrices. All these boil down to summing a lot of terms: <math>\sum_{k=1}^K a_k b_k</math>. When <math>K</math> is large (it could be in the thousands or more for a single neuron in a fully connected layer of a large model), the potential for floating-point error in that sum increases.

If using FP32, the error may still be negligible in many cases, but in lower precision like FP16, it can become significant. For example, consider a dot product of length 2048 in FP16. Each product is in FP16 (with ~3 decimal digits precision) and if accumulated in FP16, the worst-case error could be on the order of <math>2048 \times \varepsilon_{FP16} \approx 2048 \times 0.0009766 \approx 2</math> (i.e., you could lose a couple units in the third decimal place relative accuracy in worst case). Even in random error accumulation, you might see error on the order of <math>\sqrt{2048} \times 0.0009766 \approx 0.044</math> or 4.4% (though this is a rough estimate). That’s quite significant. Fortunately, as discussed, most hardware will accumulate such sums in a higher precision like FP32, dramatically reducing that error (~7 orders of magnitude smaller <math>\varepsilon</math>). But if it didn’t, the network’s outputs could get noisy. A fully-connected layer with many inputs might output slightly different results depending on summation order if done in low precision.

'''Large-scale reductions''' also occur in operations like global average pooling (summing all elements of a feature map) or computing the total loss by summing loss over batch samples. If one sums 100,000 values in FP32, the worst-case error could be on the order of 100,000 '' 1e-7 ≈ 1e-2 (1% uncertainty), though typical error would be much less. In FP16, summing 100k values is likely impossible without overflow: as noted, adding ~10^4 moderate numbers can overflow FP16<ref name="ref3"/>. So large reductions ''must* be handled carefully (either in chunks or in higher precision). Frameworks generally do: for example, when performing a sum reduction on GPU, libraries break it into parallel blocks that accumulate in registers (usually 32-bit registers, even if inputs are 16-bit).

Another aspect is that many dot products in neural nets involve values of varying magnitude. If the distribution of terms is such that some are much larger than others, the smaller terms might not contribute much if summed last. Ideally, one might sort by magnitude or use pairwise summation to improve that. Most BLAS libraries effectively do a pairwise strategy due to their parallel structure. So, while worst-case analyses are grim, practical implementations mitigate a lot of it.

However, certain layers can suffer if not handled properly. One famous example is in RNNs or LSTMs: if one uses FP16 naive summation for the gating mechanism, errors could accumulate over many time steps and cause drift or instability. In modern practice, recurrent layers are often run in FP16 with FP32 accumulators to avoid this.

In summary, '''dot products and large reductions need sufficient precision''' or robust algorithms. If you are implementing a custom kernel (say a custom ONNX op) and you have to sum a huge array, you should consider using a technique from Section 3 (pairwise summation, Kahan, or at least double accumulator if possible). Many deep learning computations, by virtue of hardware and library design, already do something like this under the hood. But it's always good to be aware, especially if pushing into lower precision territories.

=== 6.2. Cancellation, Dynamic Range, and Conditioning ===

We introduced ''cancellation'' earlier in the context of summation. In deep learning, cancellation can appear in a few places. One is in the computation of certain norms or variances. For instance, Batch Normalization computes mean and variance of activations. The variance formula <math>\mathrm{Var}(x) = E[x^2] - (E[x])^2</math> is subject to catastrophic cancellation if <math>E[x]</math> and <math>\sqrt{E[x^2]}</math> are close in magnitude (which can happen if the variance is small compared to the mean). If implemented naively in single precision, you might get a negative variance due to numerical error (which would then be clamped to zero or sqrt would yield NaN). Robust implementations compute variance in a single pass or increase precision. Indeed, in many frameworks, batchnorm accumulation is done in FP32 even if the activations are FP16, precisely to avoid numerical issues in computing mean and variance. ONNX itself doesn’t specify this, but an ONNX backend is likely to do batchnorm stats in higher precision for safety. The ''conditioning'' of the variance calculation is poor when variance is tiny relative to mean, so it's a spot where higher precision is needed.

Another example is in certain loss functions or regularizations. If you subtract two large numbers to get a small difference, you lose precision. Say you have two nearly identical predictions and you subtract them as part of some operation – if done in float16, you might get zero whereas float32 would have a slight difference. This could affect, say, a gradient if that difference was supposed to propagate information. Fortunately, neural nets often avoid directly subtracting large nearly equal numbers; but it can happen. In attention mechanisms, the ''score normalization'' (softmax) subtracts a max to avoid huge exponentials, which is good for range but if multiple scores are close to the max, you subtract numbers that are close. However, those subtractions are not catastrophic in the same way; they're intentional to reduce range and are typically fine in FP32 or even FP16.

'''Dynamic range''' issues are everywhere in deep learning. Activation functions like ReLU and exponential can produce very large or very small values. If subsequent computations don’t account for that, you get infinities or zeros. For example, consider a network output that produces a logit of 100 in FP16. <math>\exp(100)</math> in FP16 overflows to infinity (since FP16 max ~ 6e4 and <math>e^{100} \sim 3.7e43</math>). If you then do softmax, that one infinity will dominate the sum and you might get a probability of 1 for that class (or NaNs if not handled). The standard softmax implementation subtracts the max logit, so in this case it would subtract 100 from all logits, making the largest exponent <math>\exp(0)=1</math> and others tiny – this avoids overflow. But what about underflow? If you had a very negative logit, <math>\exp(-100)</math> in FP16 would underflow to 0. In softmax, that just contributes ~0 probability, which is fine (it effectively is zero anyway). Underflow tends to be less dangerous than overflow in inference — underflow usually just means “this probability/neuron is effectively 0,” whereas overflow can poison the results with NaNs or Infs.

'''Conditioning''' refers to how sensitive a function is to input perturbations. Neural networks themselves as functions can be poorly conditioned (e.g., tiny changes in weights cause big changes in output for some architectures), but typically they are designed to be reasonably well-behaved. Certain internal computations like matrix inversion (if one does a pseudoinverse or something for a layer) can be extremely ill-conditioned if the matrix is near singular. That again would need double precision perhaps. In ML, one common ill-conditioned operation is the '''softmax denominator''': <math>\sum_j e^{x_j}</math> can be very large and also very sensitive to changes in the largest <math>x_j</math>. However, the softmax is more stable if computed carefully (max trick). Another is '''normalization''': computing <math>\frac{x - \mu}{\sigma}</math> for layer norm or batch norm – if <math>\sigma</math> is very small, then the division blows up noise. In those cases, a small <math>\epsilon</math> is usually added to <math>\sigma^2</math> to prevent instability (this is a model-level fix, not a numeric one per se, but it acknowledges numeric limits).

In training, '''optimizers''' can suffer from precision issues. For example, the Adam optimizer keeps a moving average of squared gradients; in half precision, these small updates might flush to zero or not change at some point. Some research found that certain adaptive optimizers break in FP16 because the small differences are lost ("Why FP16 training breaks NAdam" etc.). The solution is often to keep the optimizer states in FP32 while doing the bulk of compute in FP16. This again is a form of mixed precision.

From an inference perspective, once the model is trained, the main concerns are avoiding overflow in activations and ensuring summations (like in softmax or reductions) are done with enough precision. The model’s weights are fixed, so we don't have to worry about cumulative error over iterations as in training, just the forward pass.

=== 6.3. Normalization Layers and Softmax ===

'''Normalization layers''' (BatchNorm, LayerNorm, etc.) and '''Softmax''' are specifically worth examining for numerical stability:

* '''Batch Normalization:''' During inference, batchnorm applies an affine transform using precomputed mean and variance (from training). It’s basically: <math>y = \frac{x - \mu}{\sqrt{\sigma^2 + \epsilon}} \gamma + \beta</math>. This is a stable operation as long as <math>\epsilon</math> is provided (to avoid division by zero). Since <math>\mu, \sigma^2</math> are constants in inference, it reduces to linear ops which are fine. The main numerical work (computing mean/variance) happened during training or model calibration, and frameworks usually did that in FP32. So batchnorm at inference time is not a source of instability (it’s just a scale and shift of the data).

* '''Layer Normalization / Group Normalization:''' These compute mean/var on the fly for each sample (and possibly channel group). In inference, they still need to compute these statistics for each forward pass. If implemented in low precision, the variance calculation can suffer cancellation as mentioned. However, it's likely that inference engines compute the mean and variance in FP32 even if inputs are FP16 (especially since these are per-sample operations, not huge batch reductions – it's feasible to do in higher precision). If someone implemented layer norm purely in FP16, they might see issues when the variance is very small. Usually there’s an <math>\epsilon</math> added which might dominate a tiny variance and thus mitigate catastrophic cancellation (the result might just output something mostly from <math>\epsilon</math>). In summary, normalization layers are usually fine as long as the implementation uses enough precision for the summation of squares.

* '''Softmax:''' Softmax is notorious if done improperly. The safe algorithm: subtract the maximum input from all inputs (to bring the largest exponent to 1.0), exponentiate, sum them up, divide each exponent by the sum. The max subtraction handles overflow in <math>\exp()</math>. But what about the sum? As discussed, if there are <math>N</math> elements in the softmax, the sum is at most <math>N</math> (when all inputs equal the max). In classification, <math>N</math> might be the number of classes. For ImageNet, <math>N=1000</math>, so worst sum ~1000 (which is fine in FP16, as 1000 << 65504). For an extreme case like a language model, <math>N</math> could be 50,000 (vocab size). Sum could be 50,000 in worst case (everyone equal probability). 50,000 is still below 65504, so actually an FP16 sum would ''not'' overflow in that scenario, and the rounding error on a sum of 50k identical terms would be maybe a few ULPs (which at 50k magnitude might be ~0.5 difference worst-case). So maybe softmax summation in FP16 is not catastrophic for output probabilities – you’d get essentially 1.0 after normalization instead of 0.99999 in some case. Usually fine.

:The bigger issue is if one or a few terms dominate the softmax. If one term is much larger, the others underflow to 0 after the max subtraction. That’s fine: you get a one-hot distribution essentially. If the model expected a softer distribution, that could be a problem. But if the logits were really that far apart, even FP32 would give nearly one-hot. So not really a precision issue, more of a model property.

:However, the '''stability during training''' is a big issue. The GitHub thread we saw (ColossalAI) mentioned training a 130B model where FP16 softmax in attention caused training to occasionally blow up<ref name="ref14"/><ref name="ref14"/>. They observed attention scores in some heads ranging up to 1e4 or as low as 1e-3<ref name="ref14"/>. 1e4 exponent in FP16 is definitely Inf (because <math>\exp(10^4)</math> is astronomically huge, but even exp(100) is inf in half). The method CogView introduced (Precision Bottleneck Relaxation) involved subtracting the max ''per head'' to reduce range<ref name="ref14"/>—which is basically the softmax max trick at a finer granularity or something similar. In the end, they found '''the simplest fix was to compute softmax in FP32''' for those layers<ref name="ref14"/>. They reported that it had negligible speed impact but significantly improved stability<ref name="ref14"/>. This highlights a general principle: certain operations like softmax (and sometimes matrix multiplications in attention) benefit from higher precision, and using FP32 in a few critical spots can save a mixed-precision training from crashing. In inference, using FP32 for softmax is also a safeguard if one is uncertain. It costs little because softmax is not a heavy compute compared to matrix multiplies.

:ONNX does have a Softmax operator; if the input is float16, a backend might do exactly what they described: internally cast to float32 for the softmax computation, then cast output back to float16. Some inference engines (like DeepSpeed’s inference or others) keep attention softmax in FP32 even if rest is FP16, for robustness<ref name="ref14"/>.

Apart from softmax, other places where exponentials occur are in loss functions like cross-entropy (which uses log-softmax internally), or in activation functions like Sigmoid and Tanh (which are basically scaled logistic functions). Sigmoid(<math>x)=1/(1+e^{-x})</math>. If <math>x</math> is moderately large positive, <math>e^{-x}</math> underflows to 0 in FP16 if <math>x>~11</math>. That gives a sigmoid output of 1.0 exactly. In FP32, <math>e^{-x}</math> for <math>x=11</math> is about <math>5e-5</math>, not underflow, so sigmoid ~0.99995. So FP16 will saturate some outputs to exactly 0 or 1 sooner than FP32 would. This can slightly change behaviors (e.g., a neuron might go to full saturation a bit early). Usually, this isn’t a huge deal, but it could make gradients zero earlier. In inference, we don’t care about gradients, just the output value. Getting exactly 1.0 vs 0.99995 probably has no practical difference for a classification output. But one should be aware that '''activation functions can saturate faster''' in lower precision due to limited range or precision. ReLU has no issue (it’s linear or 0), but exponential-based activations do. In practice, again, frameworks often simply compute those in the given precision and accept the slight difference. If it were a problem, one could do those in FP32 as well.

'''Summary:''' Numerical stability in inference is maintained by using robust computation methods: subtracting maxima in softmax/LogSumExp, adding epsilon in denominators for normalization, and using higher precision for summations or critical steps. Most of these are built into the algorithms. ONNX models assume these standard practices (they don't need to explicitly do them in the graph because the backend inherently will for ops like Softmax). It's wise when building or exporting models to ONNX to test them in the target precision and ensure there are no surprises (like an output becomes NaN or Inf due to precision). Usually if training was done in mixed precision, the model will have been vetted in that environment already.

== Performance vs. Accuracy Trade-offs ==

There is an inherent trade-off between numerical precision and computational performance (speed, memory usage) in AI systems. Choosing lower precision can dramatically increase throughput and reduce memory, but it may introduce arithmetic errors. Engineers must balance these considerations:

=== 7.1. Throughput, Memory Bandwidth, and Precision ===

'''Throughput (speed)''' is often higher for lower-precision operations. Modern hardware can execute more low-precision ops in parallel than high-precision ops. For example, an NVIDIA V100 GPU can perform 2× more half-precision FLOPs than single-precision per clock, and newer architectures with Tensor Cores can do 8× more (since a Tensor Core warp operation might perform 64 FP16 FMAs per cycle vs 16 FP32). Similarly, int8 operations can be even faster and more energy efficient. Lower precision numbers also use less memory bandwidth – fetching 16-bit data is twice as fast as 32-bit data from memory (assuming the same bus width), which can relieve memory bottlenecks. Many deep learning workloads are memory-bound, so halving the data size can nearly double throughput if compute is not the limiting factor. This is a huge incentive to use lower precision.

'''Memory and storage''': Using FP16 or int8 weights reduces model size, which is important for deploying large models to memory-limited environments (like edge devices or GPUs with limited VRAM). It also means you can fit larger batch sizes in memory, or more models on a chip, etc. There is also a cascade effect: smaller data means better cache utilization, which further improves performance.

However, '''accuracy''' suffers if precision is too low. We have discussed how rounding error and limited range can cause deviations in outputs. The question is: do those deviations matter for the application? Often for neural networks, a small amount of noise or error in intermediate calculations does not change the final prediction. Neural nets are somewhat error-tolerant due to their redundant distributed representations. This is why quantization and FP16 techniques can work at all. The final model accuracy (e.g. top-1 accuracy on ImageNet) might drop only 0.5% or less when using FP16 instead of FP32, or int8 instead of FP16, if done properly. For many use-cases, this loss is acceptable given the speed gains.

If one goes too low (like naive FP8 or int4 without special care), accuracy might drop severely – perhaps making the model unusable. Thus the trade-off: as precision decreases, performance increases but at some point accuracy degrades beyond an acceptable threshold. Engineers aim to find the lowest precision that still preserves accuracy.

There is also the consideration of '''accumulation strategy''' affecting performance: for instance, Kahan summation (compensated) is more accurate but ~4× slower than naive summation. If you need that accuracy, you pay a performance cost. In many AI inference scenarios, the slight extra accuracy from something like Kahan isn't needed, so it’s not used. But in a scenario where you sum millions of elements for a final result that is thresholded, maybe you do need it.

Another dimension is '''determinism vs speed'''. Enforcing a deterministic reduction order (for reproducibility) might require disabling some parallelism, which could slow things down. If exact repeatability is critical (like certain scientific or financial applications of neural nets), one might sacrifice some performance to ensure the summation happens the same way every time. ONNX allows both modes depending on the runtime.

From a memory perspective, if precision is higher than needed, you waste memory. For example, storing intermediate activations in FP32 when they could be FP16 uses 2× memory, which might mean lower batch size or more memory traffic. So there's strong motivation to keep things as low-precision as possible ''except'' where accuracy truly demands.

In summary, the trade-off often manifests as ''diminishing returns'': FP64 vs FP32 – huge slowdown for usually negligible accuracy gain in ML; FP32 vs FP16 – moderate speedup for usually minor accuracy hit; FP16 vs int8 – another big speedup but maybe more accuracy loss unless calibration is done; int8 vs int4 – speed gain but often big accuracy drop that might not be worth it for current models.

Engineers use techniques like '''mixed precision''' to target high precision only where needed. For example, run most of the model in FP16, but keep a few sensitive layers in FP32. This yields most of the performance benefits while ensuring critical parts are accurate. We already saw softmax as one example. Another might be final output layer: sometimes using FP32 for the logits and softmax can be safer.

Hardware is also trending to support mixed modes – like FP8 Tensor Cores accumulate in FP32, etc. So hardware tries to give accuracy cheaply.

=== 7.2. Hardware Accelerators and Tensor Cores ===

'''Hardware accelerators''' (GPUs, TPUs, FPGAs, ASICs like GraphCore IPU, etc.) have specific support for certain precisions. Leveraging these can provide massive speedups:

* '''NVIDIA Tensor Cores:''' These started in Volta (V100) with support for 4x4 matrix multiply-adds on FP16 inputs with FP32 accumulation<ref name="ref6"/>. Using Tensor Cores can give up to 8x throughput compared to using regular FP32 cores. But to use them, the data needs to be FP16 (or now BF16 or FP8 on newer GPUs) and aligned to certain dimensions (multiples of 8 etc. in matrix sizes, which frameworks handle by padding if needed). If an ONNX model is all FP32, by default it would not use Tensor Cores unless the runtime automatically converts some ops to mixed precision (which some do via a flag or environment variable enabling TF32 or FP16). The introduction of '''TensorFloat-32 (TF32)''' in Ampere GPUs is essentially a compromise: treat FP32 inputs as two 16-bit significand halves (10-bit mantissa actually) so that it can run on Tensor Cores, giving ~8x speed, while still outputting FP32. TF32 has about the precision of FP16 but range of FP32, and it was chosen because it typically doesn’t hurt convnet accuracy. ONNX Runtime or others on Ampere will use TF32 by default for conv and matmul unless disabled, giving a free speed boost at minimal accuracy cost. If exact FP32 is needed, users have to turn it off (sacrificing speed).

* '''TPU (Google):''' TPUs have native support for BF16 matrix ops with FP32 accumulators. They were designed specifically for the mixed precision approach in training. For inference, TPUs also support int8 and other quantized ops (e.g., via Edge TPU for lower-power). If exporting ONNX to run on a TPU (via some bridge), one might get automatic BF16 usage. BF16 has the advantage of no loss in range, so TPUs can more straightforwardly run models in BF16 without modifications.

* '''FPGAs and ASICs:''' Some custom accelerators might use fixed-point or block floating-point arithmetic for efficiency. They often treat the network in quantized terms. The ONNX quantization specification (QuantizeLinear/DequantizeLinear ops) exists for representing quantized models (INT8 etc.), which is another domain beyond floating-point, but it overlaps as a precision issue. In those systems, accumulators for INT8 × INT8 multiplies are usually 32-bit (to avoid overflow for dot products up to certain lengths). The trade-off there is between integer vs float representation, which can give big speedups on CPUs/DSAs via SIMD instructions (many CPUs have vector int8 multiply-add instructions now for inference).

* '''Memory bandwidth on hardware:''' Some accelerators include special high-bandwidth memory or cache specifically for lower precision. For example, an accelerator might fetch two FP16 values in one 32-bit memory transaction – effectively doubling memory bandwidth for half precision. On some hardware, using FP16 can even reduce memory ''latency'' if multiple values can be packed.

* '''Parallelism and Vectorization:''' Many hardware units process data in fixed vector widths (e.g., 256-bit registers). That could be 8 floats at once or 16 half-floats at once in the same 256 bits. If your code is in FP16, you might use the full vector width (16 values) whereas FP32 might only use half (8 values). This means FP16 can make better use of vector units. Similarly, matrix units often have fixed tile sizes, and smaller data types allow larger tiles or more tiles concurrently.

* '''Energy efficiency:''' Lower precision often means less energy per operation, which on large deployments (data centers or battery-powered devices) is a significant advantage. This isn't directly performance, but it's an important trade: you might choose a slightly lower precision to save power even if accuracy drops a tiny bit.

* '''Software overhead:''' Using mixed precision on hardware involves some overhead of casting and moving data between precisions. For instance, if you have to frequently cast FP16 to FP32 and back, that adds instructions and memory copies. A well-designed accelerator (like GPUs with unified memory for FP16/32 or special hardware casting) minimize this. Still, the runtime might insert extra operations (as we saw, ONNX Runtime CPU inserted many Cast nodes converting FP16 to FP32 and back<ref name="ref9"/>, which actually ''slowed down'' that FP16 model by 4x vs FP32 on a CPU, because the CPU had no native FP16). So ironically, on some hardware, using lower precision in ONNX might degrade performance if not supported (the CPU case). Always ensure the target hardware actually supports the low precision you plan to use; otherwise, you'll pay overhead for emulation.

To illustrate trade-off: Suppose you have a model that achieves 99% accuracy in FP32 and drops to 98.5% in FP16. If FP16 runs 2x faster, many would accept the 0.5% accuracy drop for double throughput. If int8 runs 4x faster but accuracy drops to 95%, that might be too much drop for some applications, but acceptable for others that value speed. It's application-dependent.

In critical applications, one might maintain higher precision accumulators or run certain validation steps in high precision to ensure nothing drifted. In research, sometimes a model is trained in mixed precision but validated in double to ensure any small numeric differences don’t influence the reported result (rare, but for scientific computing integration maybe).

In the ONNX world, performance-vs-accuracy is often managed via '''Execution Providers''' (EPs) that target specific hardware. For example, the TensorRT EP in ONNX Runtime can take an ONNX model and run it using NVIDIA's high-performance inference library, which will use FP16 or INT8 if allowed, to maximize speed. The user can configure the EP with precision flags. If the accuracy drop is too high, one might restrict it to FP32. If it's fine, allow FP16. Tools exist to measure the accuracy difference (like calibrating int8 quantization).

To conclude, the trade-offs revolve around picking the lowest precision that yields acceptable accuracy, and utilizing hardware features fully. ONNX being a high-level spec abstracts this, but it's ultimately up to the deployment to decide and sometimes up to the exporter to incorporate quantization. ONNX’s role is to be able to ''represent'' these choices (they have added quantized types, FP8 types, etc., to allow the graph to include those when needed).

== Best Practices and Design Guidelines ==

Given the above insights, here are some best practices and guidelines for dealing with numerical precision and accumulators in AI inference (and training) design:

=== 8.1. Choosing an Accumulator Precision ===

* '''Match precision to data length and range:''' If an operation accumulates a large number of terms, consider using a higher precision accumulator than the input data. For example, for summing >1000 values of type FP16, an FP32 accumulator is recommended to avoid large rounding error or overflow. If summing > million values of FP32, consider double (FP64) accumulation if high accuracy is needed (though this is rare in DL). ''Rule of thumb:'' if <math>N \cdot \varepsilon</math> is not tiny (say >1e-3), you may benefit from higher precision. For FP16 (<math>\varepsilon \approx 1e-3</math>), that threshold N is a few hundred. For FP32 (<math>\varepsilon \approx 1e-7</math>), N would be in the millions before 1e-3 error.

* '''Use higher precision in critical layers:''' Identify if certain layers of the model are more sensitive to precision. For instance, the output layer or a layer that feeds into a very sharp decision boundary might require more precision. If a small change there could flip the prediction, you don't want excessive numerical error. Using FP32 for logits and softmax is a common practice to maintain classification confidence stability.

* '''Follow hardware defaults, but override if needed:''' Many hardware accelerators have sensible defaults (like accumulate in FP32 for tensor ops). It's usually best to use those defaults as they were chosen to balance accuracy and speed. Only override if you have evidence of a problem. For example, if using ONNX Runtime on CPU, you discovered it upcasts to float for you – great. But if using some custom accelerator that does pure FP16, you might manually insert casts in the ONNX graph around certain ops to force higher precision on those ops.

* '''Maintain precision for gradient accumulation (training):''' This is more training-oriented, but worth noting: when summing gradients over batches or updating weights, use higher precision (FP32 or even 64) for those accumulations. This ensures training doesn't diverge due to numeric error. Many frameworks keep a "master copy" of weights in FP32 when training in FP16<ref name="ref2"/>. For inference, this translates to: any running statistics or counters in the model (like if you had a running average inside the model) should probably be in higher precision to avoid drift.

* '''Be mindful of intermediate cast costs:''' If you do decide to cast to higher precision for an operation in ONNX, remember it incurs memory copies and compute overhead. Make sure it's worth it. For example, if you cast a huge tensor to double, that could slow things drastically. Usually FP16->FP32 is fine (hardware handles it well), but FP32->FP64 might be expensive on GPUs.

=== 8.2. Safe Reduction Lengths and Heuristics ===

* '''Chunk long reductions:''' If you must do a very long sum in low precision (due to memory constraints or hardware), chunk the sum into parts, sum each to a higher precision local accumulator, then sum the partial sums. This is essentially pairwise summation. Many libraries use a fixed chunk size heuristic (e.g., sum in blocks of 256 in registers, then add those up). This bounds the error growth. If implementing manually, you can choose a chunk size such that chunk_length '' ε is small (like 256 '' 1e-3 = 0.256 for FP16, hmm maybe choose 100).

* '''Sort by magnitude if possible:''' In some cases (not typical in DL, but possible in post-processing), if you have numbers varying by orders of magnitude, summing from smallest to largest improves accuracy. It prevents small ones from being swallowed by big ones. This is more relevant if you output a sum of mixed-scale values (like some algorithm mixing big and small contributions). It’s not always feasible in real-time inference to sort, but keep in mind.

* '''Use compensation for critical sums:''' If you find a particular sum or difference in your inference computation is crucial (like computing a small residual from two close numbers), consider using Kahan summation or another compensated scheme just for that part. For example, if you had an ONNX custom function that needs to sum a lot of almost cancelling terms, you might implement it with compensation internally.

* '''Heuristics for FP8/low-bit:''' When using extremely low precision like FP8, adopt the recommended practices: use scaling (don't feed raw data into FP8, normalize it first), and perhaps only use FP8 on portions of the network that are empirically robust. The industry is still establishing heuristics, but one might be: use FP8 for matrix multiplications, but accumulate in FP16 or FP32; keep normalization in FP16; use FP16 for first and last layer maybe. This is evolving, so keep an eye on literature.

* '''Keep an eye on sum length in softmax and normalization:''' If you have extremely large class counts or group sizes (e.g., a softmax over 1e6 items, or layer norm over tens of thousands of features), realize this is a giant reduction. You may want to ensure that reduction is done in higher precision. ONNX Softmax by default will likely not handle a million-class case in FP16 well (the sum would overflow since 1e6 > 65504). So either do that on CPU in FP32 or break it down (could do hierarchical softmax, etc.). This is a corner case, but relevant for huge vocab language models. In practice, vocab ~50k is fine as we said.

* '''Test numeric differences:''' A practical tip: after quantizing or reducing precision, run some test inputs through both the high-precision model and low-precision model. Compare outputs (and maybe some intermediate signals if possible). If differences are consistently small relative to the application tolerance, you're good. If you see occasional large differences or instabilities (like outputs become NaN or Inf in low precision), that's a red flag. Use that testing to guide where you might need to increase precision or change algorithms.

=== 8.3. When Higher Precision Is Required ===

* '''Ill-conditioned calculations:''' If your model or pipeline performs any operation known to be ill-conditioned (matrix inversion, subtraction of nearly equal quantities, computing a tiny difference of large numbers), you likely need higher precision for that part. For example, if your ONNX model includes a custom op to solve a linear system, use double precision in that op if possible (or at least float with iterative refinement). In ML, this might occur in something like a physics-informed neural network or a differentiable simulation within the model.

* '''Strict accuracy requirements:''' Some applications (medical, aerospace, finance) might have strict tolerances. If an inference model’s output needs to be correct to 5 decimal places, you probably cannot use FP16, which has only 3-4 decimal digits precision. FP32 might be barely enough (~7 digits). FP64 might be warranted to guarantee that level of accuracy in worst case. Most deep learning applications (like image classification) don't demand that level of numeric precision in outputs – as long as the top class is correct, it doesn't matter if its probability was 0.901 vs 0.899. But if you're doing something like summing probabilities to make a risk estimate or doing a cumulative sum over many predictions, be careful to accumulate precisely.

* '''Large sums or products in post-processing:''' Sometimes the neural net is fine, but the ''post-processing'' of results might need precision. E.g., summing up probabilities of many events, or computing confidence intervals, etc. Do those calculations in double if needed outside the model.

* '''Preventing drift in long sequences:''' For sequential models (RNNs processing thousands of steps), rounding error can accumulate over steps. If each step has a small error, after 10000 steps it could add up. In such cases, using a higher precision for the state might reduce drift. Some practitioners use FP64 for the hidden state accumulation in very long sequence processing to maintain numerical stability (though this is not common due to speed loss). Alternatively, periodically renormalize or reset small errors.

* '''Simulations or differential equations:''' If your ONNX model involves simulating something (like a differential equation solver inside a model, or an ODE/RNN hybrid), those often need double precision to be stable for long durations. Recognize those situations and allocate precision accordingly.

* '''Verification and debugging:''' Use higher precision as a tool to verify correctness. For example, run your model in FP64 (if possible) and compare to FP32 to see if there are differences. If not, FP32 is fine. If yes, you might find a particular calculation losing too much info in FP32, implying maybe you should keep it in FP64 in production as well. This is rare in inference but can be part of rigorous validation.

In ONNX, using higher precision may involve casting to <code>tensor(double)</code> for certain ops. Not all runtimes support double on GPU though (some do via emulation or just slower path). ONNX CPU does support double for general ops. So be mindful of the target.

Finally, '''document your precision choices'''. If you deliver an ONNX model that uses mixed types (float16 inputs, float32 accumulations, int8 weights, etc.), ensure that is clear to whoever maintains or uses it. Precision issues can be sneaky, so clarity helps.

== Limitations and Open Problems ==

Despite advances in handling numerical precision, there remain limitations and open challenges in this domain:

* '''Lack of Precision Specification in ONNX:''' As noted, ONNX currently has no way to formally specify that an op ''should'' use a higher precision internally. This can lead to nondeterministic differences across platforms<ref name="ref12"/>. While proposals exist to add attributes for this<ref name="ref12"/>, it is not yet standard. Thus an ONNX model might behave slightly differently on two compliant backends, and there's no built-in mechanism to ensure identical results. This is a limitation for applications requiring cross-platform consistency. Solving it may involve extending ONNX (e.g., an attribute like <code>accumulatePrecision="FLOAT32"</code> for MatMul) or tightening the spec, but those come with backward compatibility and implementation burden concerns.

* '''Reproducibility and Determinism:''' Even with the same precision, parallel computation can introduce nondeterminism (due to different summation orders). Ensuring bit-identical results across runs (important for some sensitive domains) is non-trivial. Frameworks often provide "deterministic mode" which might use fixed ordering at cost of speed. In ONNX, there's no global flag for this; it's up to the runtime. More research could be done on algorithms that produce bit-wise identical results independent of parallel execution order (perhaps using techniques like pairwise summation or consistent partitioning). Stochastic rounding is another area: sometimes using probabilistic rounding can reduce bias in accumulated error, but it introduces randomness – reconciling that with reproducibility is tricky.

* '''Ultra-low Precision (FP8 and below):''' While FP8 is being explored, going to 4-bit floats or mixed schemes is an open challenge. The OpenAI community is investigating 4-bit (quantization-wise) for LLMs with some success, but a true 4-bit float (like E3M1) would be extremely limited. Perhaps hybrid schemes (like block floating point, log scales, etc.) will come. ONNX already has some support (they mention 4-bit in documentation placeholders<ref name="ref10"/>), but it's nascent. Research into training techniques, calibration, and new number formats (like '''posit numbers''' or tapering precision floats) is ongoing. If a new format proves useful, ONNX will need to adapt to represent it.

* '''Quantization Noise and Model Robustness:''' Reducing precision can sometimes expose models to issues like adversarial vulnerability or fragility – e.g., a slight perturbation might flip a decision if the margin was small and precision loss nudged it. Ensuring model robustness under quantization is an open problem. It might require re-training the model with quantization in the loop (quantization-aware training) or designing models that have large "margin" between classes to tolerate quantization noise.

* '''Dynamic range during training vs inference:''' In training, activations can have wider distributions (especially early in training). Models often learn to constrain their ranges (via activation functions saturating or normalization), which makes inference more stable in lower precision. But if one tries to train in FP8 from scratch, it's very hard. This gap between what's possible in a post-trained inference model vs training is an open area – how to train directly in low precision without significant accuracy loss. Solving that could simplify workflows (no need for high-precision training then quantize; just train low-prec end to end). Techniques like loss scaling, gradient clipping, etc., are partial solutions. The development of optimizers that are resilient to low precision is ongoing.

* '''Hardware variability:''' As new hardware emerges (with new numeric capabilities or quirks), ensuring that ONNX models run consistently is an ongoing task. For instance, one device might implement a fused op that reduces rounding error, another does it separately. Even something like convolution might use a different algorithm (FFT-based vs direct) which has different numeric properties. Open problem: could ONNX allow specifying numeric tolerances per op or preferred algorithms? That may be overkill for most, but for some HPC ML use cases, maybe desired.

* '''Testing and validation of numerics:''' There isn't a standardized set of "numeric compliance" tests for ONNX runtimes beyond simple correctness. Perhaps in the future, test suites might include challenging numeric scenarios (large sums, nearly singular matrices) to ensure a runtime handles them gracefully (maybe by switching precision). Right now, it's mostly on the user to test their model under expected conditions.

* '''Combining quantization with floating point:''' Many production models use a mix (e.g., int8 for some layers, FP16 for others). ONNX can represent this (with QuantizeLinear ops or MixedPrecision through casting), but tooling to automatically find the optimal mix is still developing. It's somewhat trial-and-error or using heuristics. An open question is how to automate precision assignment: given a model and a target hardware, how to automatically choose which layers can be int8, which should be FP16, etc., to meet an accuracy target. This crosses into NAS (neural architecture search) territory or automated quantization.

* '''Numerical debugging tools:''' When a model goes wrong due to precision, it's often hard to pinpoint where. Better tooling to analyze numerical error layer by layer could be useful. For instance, one could run a model in double and float and diff the intermediate results to see where the largest divergence happens. Doing this systematically for big nets is an open tooling challenge. Some frameworks have hooks for this, but not standardized.

In conclusion, while we've come far with using lower precision in AI without sacrificing much accuracy, the landscape is always evolving. The key open problem is how to push precision lower (for efficiency gains) without crossing the threshold where accuracy falls off a cliff. Each new generation of hardware and techniques (like FP8, novel quantization schemes) extends that boundary, but also brings new questions on how to manage the numerical errors. ONNX as an interchange will need to keep pace, possibly by adding more expressive power for these numeric considerations.

== See Also ==

* '''IEEE 754 Floating-Point''' – The standard defining binary16, binary32, binary64 formats and rounding behavior.
* '''Floating-Point Arithmetic''' – General topic on how floating-point math works, common issues (overflow, underflow, cancellation).
* '''Kahan Summation Algorithm''' – Algorithm for compensated summation to reduce numerical error<ref name="ref4"/>.
* '''BFloat16''' – A short-format float with 8-bit exponent, used in TPUs and some GPUs for training networks.
* '''Mixed Precision Training''' – Technique of using FP16 (or BF16) along with FP32 accumulators to accelerate training<ref name="ref2"/>.
* '''Quantization in Deep Learning''' – Converting networks to lower bit integer arithmetic for inference (INT8 quantization and calibration techniques).
* '''ONNX Operator Specification''' – Official docs for ONNX ops, which describe their mathematical function but not the required numeric precision (a point of potential extension in future).
* '''CUDA Tensor Cores & TF32''' – NVIDIA documentation on how tensor cores perform mixed precision and the TF32 format for accelerating FP32.
* '''Pairwise Summation''' – Summation algorithm with logN error growth<ref name="ref5"/>, used in many numerical libraries to improve accuracy of reductions.
* '''Softmax Stability''' – Articles on numerically stable softmax and log-sum-exp implementations to avoid overflow.
* '''Performance Tuning for Low Precision''' – Guides (e.g., NVIDIA or Intel whitepapers) on using low precision and the trade-offs involved in various hardware.

<references>
<ref name="ref1">Accelerate and simplify Scikit-learn model inference with ONNX Runtime - Microsoft Open Source Blog — opensource.microsoft.com — https://opensource.microsoft.com/blog/2020/12/17/accelerate-simplify-scikit-learn-model-inference-onnx-runtime</ref>
<ref name="ref2">Floating Point Precision: Understanding FP64, FP32, and FP16 in Large Language Models - DEV Community — dev.to — https://dev.to/lukehinds/floating-point-precision-understanding-fp64-fp32-and-fp16-in-large-language-models-3gk6</ref>
<ref name="ref3">Floating Point Precision and Its Limitations | by Umair Akbar | Medium — akbu.medium.com — https://akbu.medium.com/floating-point-precision-and-its-limitations-cfb7247d7789</ref>
<ref name="ref4">Kahan summation algorithm - Wikipedia — en.wikipedia.org — https://en.wikipedia.org/wiki/Kahan_summation_algorithm</ref>
<ref name="ref5">Pairwise summation - Wikipedia — en.wikipedia.org — https://en.wikipedia.org/wiki/Pairwise_summation</ref>
<ref name="ref6">Numerical behavior of NVIDIA tensor cores - PMC — pmc.ncbi.nlm.nih.gov — https://pmc.ncbi.nlm.nih.gov/articles/PMC7959640/</ref>
<ref name="ref7">Mixed precision and performance-accuracy tuning (neuron-cc) — AWS Neuron Documentation — awsdocs-neuron.readthedocs-hosted.com — https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/appnotes/neuron-cc/mixed-precision.html</ref>
<ref name="ref8">Create Float16 and Mixed Precision Models - ONNX Runtime — onnxruntime.ai — https://onnxruntime.ai/docs/performance/model-optimizations/float16.html</ref>
<ref name="ref9">[Performance] ONNX FP16 model is having performance bottle neck when compared to FP32 variant · Issue #25824 · microsoft/onnxruntime · GitHub — github.com — https://github.com/microsoft/onnxruntime/issues/25824</ref>
<ref name="ref10">Float stored in 8 bits - ONNX 1.21.0 documentation — onnx.ai — https://onnx.ai/onnx/technical/float8.html</ref>
<ref name="ref11">[2209.05433] FP8 Formats for Deep Learning — arxiv.org — https://arxiv.org/abs/2209.05433</ref>
<ref name="ref12">Should MatMul op allow specification of accumulation precision? · Issue #7072 · onnx/onnx · GitHub — github.com — https://github.com/onnx/onnx/issues/7072</ref>
<ref name="ref13">Quantize ONNX models | onnxruntime — onnxruntime.ai — https://onnxruntime.ai/docs/performance/model-optimizations/quantization.html</ref>
<ref name="ref14">Keep Attention Softmax FP32 during FP16/ZeRO Training · Issue #1485 · hpcaitech/ColossalAI · GitHub — github.com — https://github.com/hpcaitech/ColossalAI/issues/1485</ref>
</references>

Numerical Precision in ONNX and AI Inference

2026-01-30T15:41:16Z

Timo.stripf:

Numerical Precision in ONNX and AI Inference

2026-01-30T15:26:41Z

Timo.stripf:

Numerical Precision in ONNX and AI Inference

2026-01-30T15:25:22Z

Timo.stripf: