The browsers we use to surf the web are pretty stupid and to get some special characters, effects and functions, escape sequences are used. Basically they are a signal to the browser to do something different rather than just type out text on the screen.
Of course the trouble with standards, as they say, is that there are so many of them. When editing input, assembling stuff automatically at the server end or when the browser reads the page sometimes things get out of whack and the browser merrily just types what it saw. Result is you get to see the escape sequence rather than what was intended.
The '&' and the < and > characters are used in building web pages so when you want to display an ampersand you need to tell the browser with an escape sequence so they dreamed up '&Amp' for less than its '<' and greater than its '>'.
Its a bit techie but it goes back to the origins of HTML and the web and when computers were limited to an 8 bit data byte.
