I'm not sure if this question is more suited to mathoverflow.
Is there a systematic way to solve for the automorphism group of a finitely presented group with a solvable word problem? Preferably with software.
Here is my particular example:
$$G_n = \langle r_1, r_2, \dots , r_n \; \vert \;r_i^{n_i}, r_1 r_2 \dots r_n \rangle \quad \text{for some} \quad \{n_i\} \subset\mathbb N$$
This is a type of hyperbolic reflection group generated by a finite number of rotations. They are index $2$ subgroups of coxeter groups.
I've tried using the GAP System for Computational Discrete Algebra to find the automorphism group, but the method AutomorphismGroup doesn't seem to terminate. However, a rewriting system for all such groups can be developed using the kbmag package.
I'm actually interested in the outer automorphisms, as these are nontrivial. I expect the rank of $\mathrm{Out}(G_4)$ to be $2$, and I think I have the explicit form of the generators in this case.
However, I don't know how to check whether my suspected generators do in fact generate the whole outer automorphism group, and so I seek a way to compute $\mathrm{Out}(G_i)$ for given rotation orders $n_i$.