From: Matthew Fernandez <matthew.fernandez@gmail.com>
Date: Tue, 15 Nov 2022 04:36:15 +0000 (-0800)
Subject: neatogen: fix miscalculation of intermediate edge resources
X-Git-Tag: 7.0.2~3^2
X-Git-Url: https://granicus.if.org/sourcecode?a=commitdiff_plain;h=41ddbf28e4a0558c6360ec170d463861b0a748b5;p=graphviz

neatogen: fix miscalculation of intermediate edge resources

`genroute` was allocating an array for edge computation upfront. But what it was
not accounting for was that some of the functions it later calls _change_ the
`pn` value it used to determine how many array elements it should allocate.
Specifically, `Pshortestpath` can add new points to the polygon, thereby causing
the walk of the (now too short) array to write out of bounds.

Gitlab: fixes #42
Reported-by: mattjj
---

diff --git a/CHANGELOG.md b/CHANGELOG.md
index c6e513617..558b02688 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -26,6 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   7.0.2. Complete #2277, #2303
 - Several compilation errors when building Smyrna on macOS have been fixed. This
   was a regression in Graphviz 7.0.1.
+- A crash when using neato layout with large inferred per-node edge counts was
+  fixed. #42
 
 ## [7.0.1] – 2022-11-09
 
diff --git a/lib/neatogen/multispline.c b/lib/neatogen/multispline.c
index cd3f9dbab..061b756cb 100644
--- a/lib/neatogen/multispline.c
+++ b/lib/neatogen/multispline.c
@@ -790,7 +790,7 @@ static int genroute(tripoly_t * trip, int s, int t, edge_t * e, int doPolyline)
     Ppolyline_t pl, spl;
     int i, j;
     Ppolyline_t mmpl;
-    Pedge_t *medges = N_GNEW(trip->poly.pn, Pedge_t);
+    Pedge_t *medges = NULL;
     int pn;
     int mult = ED_count(e);
     node_t* head = aghead(e);
@@ -816,6 +816,7 @@ static int genroute(tripoly_t * trip, int s, int t, edge_t * e, int doPolyline)
 
     if ((mult == 1) || Concentrate) {
 	poly = trip->poly;
+	medges = N_GNEW(poly.pn, Pedge_t);
 	for (j = 0; j < poly.pn; j++) {
 	    medges[j].a = poly.ps[j];
 	    medges[j].b = poly.ps[(j + 1) % poly.pn];
@@ -867,6 +868,7 @@ static int genroute(tripoly_t * trip, int s, int t, edge_t * e, int doPolyline)
 	    make_polyline (mmpl, &spl);
 	}
 	else {
+	    medges = N_GNEW(poly.pn, Pedge_t);
 	    for (j = 0; j < poly.pn; j++) {
 		medges[j].a = poly.ps[j];
 		medges[j].b = poly.ps[(j + 1) % poly.pn];
diff --git a/tests/test_regression.py b/tests/test_regression.py
index 0e727f3a7..405f426b3 100644
--- a/tests/test_regression.py
+++ b/tests/test_regression.py
@@ -75,7 +75,6 @@ def test_14():
   dot("svg", input)
 
 @pytest.mark.skipif(which("neato") is None, reason="neato not available")
-@pytest.mark.xfail()
 def test_42():
   """
   check for a former crash in neatogen