The knowledge is still there, it’s just that the LLM has been instructed not to divulge it, and these instructions are often imperfect and can sometimes be circumvented accidentally or on purpose.

Yeah, given that LLMs are trained from the internet, they all eventually devolve into “Hitler did nothing wrong” memes and/or hardcore racism.