Probably the same way as Trawlers do ..sorry did.
Sonar and seabed patterns.
But there are also some VLF (very low frequency) systems in play I would think.
[Very good programme on BBC R4 this evening about the development of USS Nautilus and her voyage from the Pacific to the Atlantic via the North Pole in 1958. Who says the 1950's were dull!]
Archive hour on R4 tonight was interesting - Nautilus the 1st nuclear sub (per Twisterken). One of the voices on the prog. was the navigator.
One attempt at voyage under polar ice cap was abandoned after power failure knocked out gyro compasses. Navigator said that he was then solely reliant on magnetic compass, which "wasn't too happy" being encased in a steel hull. The probability of being able to bring the gyros back and get useful data from them was slim, so "it wasn't too long before I realised I couldn't determine our position", and we had to "reverse course".
Another attempt was abandoned after surfacing under the ice (twice?) and damaging both periscopes. The ship's engineer spent (10?) hours welding - in high winds & low temperatures - to get one of the scopes working.
Fascinating programme - well worth the a "listen again".
I suspect they may well use an Inertial Navigation System (INS) which was in use on commercial aircraft before GPS. Really cute, and doesn't rely on any external signals. Provided it knows where it starts from, any acceleration will be sensed by its gyros and maths does the rest.
Was talking once to a submariner, and asked him how they do it. From what I recall, the inertial nav system tends to "drift", so that the longer you go between fixes, the less certain you know where you are. So every now and then, they ping once with the echo sounder to check depth - but the problem with that is that it reveals their location to any other sub in the vicinity - so kept to a minimum. I think he said that they now have GPS on the periscope, which makes life easier. Before that, the periscope could also be used as a sextant for starsights.
So, in summary, they are capable of voyaging for days or weeks beneath the surface in nuclear reactors and increasingly haven't the faintest idea where they are.
I feel better for knowing that.
I used to have 2 ex-RN Nuclear Sub Navigation officers working for me, and if their attempts to direct me to Basingstoke railway station when they were desperate to catch a train are anything to go by they spend a lot of their time lost!
In WW2 they had problems from what I've read. Brits used non magnetic conning towers so as to give better chance for compass to work. However that compromised hull strength. Germans used steel Conning towers and often did not know where they were at all. Theres a story of a U boat surfacing in the Solent when he thought he was off the Channel Islands!
U boats wer bloody uncomfortable - very low silhouettes - so miserable to be on the bridge in rough weather -- but much more difficult to see better hulls etc.
And they managed to navigate too most of the time.