They only use words in context, which is their problem. It doesn't know what the words mean or what the context means; it's glorified autocomplete.
I guess it depends on what you mean by "information." Since all of the words it uses are meaningless to it (it doesn't understand anything of what it either is asked or says), I would say it has no information and knows nothing. At least, nothing more than a calculator knows when it returns 7 + 8 = 15. It doesn't know what those numbers mean or what it represents; it's simply returning the result of a computation.
So too LLMs responding to language.