If it was just one thing, that's fine. But it's more than just putting the verb at the end, it's:

• switching order of propositional phrases (uses post-position)
• no articles, it uses particles, which are also in postposition
• no difference between singular and plural, it comes from context

All of these together represent a complete shift in how you think about communicating ideas.

If I learn a romance or Germanic language coming from another romance or Germanic language, a lot of the grammar concepts apply and I can largely just translate my existing thoughts to the new language. With Japanese or Korean (very similar grammatically), I have to rewire everything, because the structure of the language is so different from my native language.

Japanese isn't particularly hard, per se, in an absolute sense. But it is hard from a relative sense coming from a western perspective.