From: Paul Ramsey <pramsey@cleverelephant.ca>
Date: Tue, 10 Oct 2017 18:04:56 +0000 (+0000)
Subject: Undefined behaviour in pointarray_to_encoded_polyline (2.4 branch)
X-Git-Tag: 2.4.1~11
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=1abf2081d47417e9a00566883bd2049a516673aa;p=postgis

Undefined behaviour in pointarray_to_encoded_polyline (2.4 branch)
(References #3891)



git-svn-id: http://svn.osgeo.org/postgis/branches/2.4@15955 b70326c6-7e19-0410-871a-916f4a2858ee
---

diff --git a/NEWS b/NEWS
index ca710c727..0e930c2b6 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,7 @@ YYYY/MM/DD
   - #3864, Performance improvements for b-tree geometry sorts
   - #3874, lw_dist2d_pt_arc division by zero
   - #3882, undefined behaviour in zigzag with negative inputs
+  - #3891, undefined behaviour in pointarray_to_encoded_polyline
 
 
 PostGIS 2.4.0
diff --git a/liblwgeom/lwout_encoded_polyline.c b/liblwgeom/lwout_encoded_polyline.c
index 8b1e617c7..050b4587c 100644
--- a/liblwgeom/lwout_encoded_polyline.c
+++ b/liblwgeom/lwout_encoded_polyline.c
@@ -73,7 +73,7 @@ char * pointarray_to_encoded_polyline(const POINTARRAY *pa, int precision)
 	stringbuffer_t *sb;
 	double scale = pow(10,precision);
 
-	/* Take the double value and multiply it by 1x10^percision, rounding the result */
+	/* Take the double value and multiply it by 1x10^precision, rounding the result */
 	prevPoint = getPoint2d_cp(pa, 0);
 	delta[0] = round(prevPoint->y*scale);
 	delta[1] = round(prevPoint->x*scale);
@@ -90,8 +90,8 @@ char * pointarray_to_encoded_polyline(const POINTARRAY *pa, int precision)
 	/* value to binary: a negative value must be calculated using its two's complement */
 	for (i=0; i<pa->npoints*2; i++)
 	{
-		/* Left-shift the binary value one bit */
-		delta[i] <<= 1;
+		/* Multiply by 2 for a signed left shift */
+		delta[i] *= 2;
 		/* if value is negative, invert this encoding */
 		if (delta[i] < 0) {
 			delta[i] = ~(delta[i]);