AA−1
δ=a′ij−aijaijeii
(A+eiδe⊤j)A−1=I+eiδe⊤jA−1
eiδe⊤j is the zero matrix, except the value of δ in the ij position. Can you see here how an appropriate rank one right multiplication with A−1 may give the desired new inverse? (Or equivalently, elementary column operations on A−1.)
Or if you like to do row operations instead, you can use
A−1(A+eiδe⊤j)=I+A−1eiδe⊤j
In the first case we have the identity with a row added. This is easy to do column operations upon to have the identity back again. Do these operations on A−1 and the result is the new inverse, as desired. The second case is the identity with a column added. In that case, you may do row operations instead. You may choose whichever is more convenient.