The solar system is doing this

Which makes you go like this

And while it's possible to make your way towards the center, it's difficult

And this requires less work

The earth is going around the sun pretty fast, and you need to counteract that speed to get to the sun.

But when you’re leaving you can use the earths orbital velocity to give you a boost.

https://xkcd.com/681_large/

The sun creates a big gravity well. The earth happens to be quite close to the top of this well (because of the inverse square low). It takes about the same energy to go down and up the gravitational well.

if your object has just enough velocity to escape Earth's gravity, it'll begin orbiting the Sun, it won't fall directly towards it.

I'm going to oversimplify this : if you want your object to hit the sun after leaving the Earth, it has to have the velocity of the Earth (30 km/s) but in the opposite direction of it's orbit around the Sun.

you also should take into account the massive distance between us anl the Sun (150 million km). so you have to aim properly, otherwise your object is gonna enter a very elliptical orbit

It makes a lot more sense once you consider where you are starting from: a rock hurtling around the sun at breakneck speed (29.8 km/s). You can not really bullseye something when you are going almost 30 km per second sideways to it.

Orbital mechanics can be a bit counter intuitive.

To raise an orbit you need to speed up, but your overall orbital speed will decrease, since the distance grows the higher the orbit gets. So you'll need to speed up to reach escape velocity to escape Earth's gravitational pull, once you managed that, you enter the orbit around the sun. To lower an orbit you'll need to lower your speed, while your orbital velocity will increase, since the orbit gets smaller.

The "fuel" of a space craft is measured in Delta v, which describes the ability to change the velocity of the vehicle, the unit is m/s or km/s To reach a low earth orbit you need around 10.000 m/s Delta v, if I'm not mistaking another 9.000 m/s to escape velocity, then you're in a sun orbit, I don't know how much you need from there, but id say at least another 15-20km/s

dV also called delta V is the change in velocity required. Higher means more fuel is required and so making the rocket is harder. The earth orbits the sun at a very high speed and so to get to a planet much further inward requires slowing down a lot. You can look up escape velocity which is the velocity required to exit a gravitational body completely.

There’s been plenty of explanations already, but here’s a perspective I think can help:

Your original intuition is entirely correct for an object that appeared next to earth but which isn’t moving relative to the sun. It would fall straight in with very little trouble. If it’s moving a little sideways then it’d need to be nudged to make sure it didn’t miss the sun.

But the Earth is moving super fast sideways, so an object coming from Earth would need to be nudged a lot to not miss the sun.