math: fix signed int left shift ub in sqrt

Submitted by Szabolcs Nagy on Oct. 13, 2019, 3:03 p.m.

Details

Message ID 20191013150319.GJ7832@port70.net
State New
Series "math: fix signed int left shift ub in sqrt"
Headers show

Commit Message

Szabolcs Nagy Oct. 13, 2019, 3:03 p.m.
old fdlibm bug, only affects soft float targets.

Patch hide | download patch | download mbox

From b877237b506e03fd3c92ad7b39adbac5e1b74441 Mon Sep 17 00:00:00 2001
From: Szabolcs Nagy <nsz@port70.net>
Date: Sun, 13 Oct 2019 14:54:31 +0000
Subject: [PATCH] math: fix signed int left shift ub in sqrt

Both sqrt and sqrtf shifted the signed exponent as signed int to adjust
the bit representation of the result. There are signed right shifts too
in the code but those are implementation defined and are expected to
compile to arithmetic shift on supported compilers and targets.
---
 src/math/sqrt.c  | 3 +--
 src/math/sqrtf.c | 3 +--
 2 files changed, 2 insertions(+), 4 deletions(-)

diff --git a/src/math/sqrt.c b/src/math/sqrt.c
index b2775673..f1f6d76c 100644
--- a/src/math/sqrt.c
+++ b/src/math/sqrt.c
@@ -179,7 +179,6 @@  double sqrt(double x)
 	ix1 = q1>>1;
 	if (q&1)
 		ix1 |= sign;
-	ix0 += m << 20;
-	INSERT_WORDS(z, ix0, ix1);
+	INSERT_WORDS(z, ix0 + ((uint32_t)m << 20), ix1);
 	return z;
 }
diff --git a/src/math/sqrtf.c b/src/math/sqrtf.c
index 28cb4ad3..d6ace38a 100644
--- a/src/math/sqrtf.c
+++ b/src/math/sqrtf.c
@@ -78,7 +78,6 @@  float sqrtf(float x)
 		}
 	}
 	ix = (q>>1) + 0x3f000000;
-	ix += m << 23;
-	SET_FLOAT_WORD(z, ix);
+	SET_FLOAT_WORD(z, ix + ((uint32_t)m << 23));
 	return z;
 }
-- 
2.23.0