Ohhhhh boy, I get to nerd out. OK, super short story; reading and chatting about The Expanse book series got me pointed towards the work of Alastair Reynolds. The early parts of his universes arch aren't really relevant for your purposes, but in the latter books, how humanity survives on lifeless rocks, is exactly what you're looking for. Plus, he's a astrophysicist doctor, iirc, and it is quite quite good hard Sci Fi.

Seveneves by Neal Stephenson is this.

Also Tau Zero by Poul Anderson, to a degree.

The Children of Time books by Adrian Tchaikovsky have a lot of those themes. Half of the first book is about an ark ship sent out to find a habitable planet because earth is dying. It spans hundreds of years as key crew members go in and out of hyper sleep. Relationships and political factions form and dissolve as the ageing ship continues its mission to find a new home.

The second book focuses on a terraforming crew that was sent to another star system to prepare a planet for humans. However, the planet's ecology is so alien it proves very difficult to gain a foothold.

Just a loose round up so far

Seveneves Neal Stephenson
Tau Zero Poul Anderson
Metro 2033 Dmitry Glukhovsky
The Children of Time Adrian Tchaikovsky
Lucifer's Hammer Larry Niven
Pushing Ice Alastair Reynolds
Record of a Spaceborn Few by Becky Chambers
Diaspora by Greg Egan
A Memory Called Empire by Arkady Martin
The 100 Kass Morgan
Interdependency trilogy by John Scalzi.
Silo series of books by Hugh Howey

The Expanse series is kinda like that. There are other planets, but most of the action takes place on ships, stations, and asteroids that have been converted into stations. It goes into depth about life in space, and everything from engineering to biology, sociology, politics, and theology.

The Bobiverse series is about a person who dies and wakes up as an AI that must replicate itself across the stars, while humanity ends on Earth.

Read Dune if you didn't read it it goes deep in to ecology and terraforming of Arrakis, Fremen surviving on it,water relations in environment...

Another inspiration for you may be Scavengers Reign - animated series about surviving on lush planet that is really inhospitable for humans.

The Interdependency series by John Scalzi portrays a society where some number of star systems, containing only one habitable planet which is at the very far reaches of the wormhole network, are connected together by wormholes. The society is called "the Interdependency" because every orbital habitat, dome and underground city is hugely dependent on trade with other habitats... without robust transfer of goods and raw materials EVERYONE would die... and this DOESN'T prevent stupid, short sighted, greedy humans from gambling with the stability of it all for their own personal economic and political gain. Fun books. Like most Scalzi, it's not too deep. But it's lots of fun.

If you also accept TV series, Battlestar Galactica may interest you.

Doesn't quite fit the bill as there's a planet eventually but Children of Time by Tchaikovsky is excellent and half the book follows a generation ship. The other half follows a successive evolution of uplifted spiders. It's reasonably hard sci-fi not Martian levels of detail about the science but very well written and enjoyable. Could be worth a go for some inspiration.

Children of Time series goes over this a little bit, especially in the first book. Colonists end up waking up early due to a malfunction and end up falling into a devolving tribalistic race to the bottom on their journey to the planet.

EDIT: As for "hard" scifi, while I wouldn't say this series is at the same level as The Martian or maybe The Expanse, it is pretty good with trying to keep things real, especially with regards to the human threads of the story.

Not quite what you're after but I absolutely love Diaspora by Greg Egan.

It's a different take on the same issues you're asking about (not at first, but it's not really a spoiler to say that it explores them whether or not it's as necessary as your examples state), a take that leans more into different forms of existence rather than supporting our current existence in a different environment (but touches on aspects of that too, kind of). It's mega-multi-generational while also not being that at all, depending on perspective.

The Mars series by Kim Stanley Robinson definitely fits the bill. The Ministry of the Future does too but it is more about the coming climate change disaster.

Someone what mentioned Kim Stanley Robinson's Red Mars trilogy, and that is really good, but his book Aurora is almost exactly what you are describing.

Highly recommend.

It's a very non traditional story structure (at least to a western reader) but The Three Body Problem series has a lot of plot revolving around the lack of inhabitable worlds.

Lucifer's Hammer by Larry Niven is a fantastic book that might be near what you are looking for. It's about an asteroid impact on Earth, this removes a lot of the population and infrastructure and the story focuses on a few different groups of people as they make do with what they can find or scavenge, and then the resource battling that goes on between groups.

A story line I remember well is on a group that found an abandoned neighborhood and were astonished to find that it still had running water from the nearby local dam/reservoir. They lived here for quite a while in their relative luxury until it just stopped working one day. A burst pipe in some other neighborhood had slowly drained the dam faster than they would have used it up.

Anyway, it's a great book because it feels so realistic as to what would really happen and the struggles people would actually be going through.

Children of Time is nearly exactly what you're looking for. The whole series doesn't follow nicely with what you're looking for but the focus remains on that aspect of things for lack of wanting to spoil anything. If nothing else read the first book, it's exceptional.

You might want to check out Record of a Spaceborn Few by Becky Chambers. The book is about the people of the Exodus Fleet, a group of multi-generation ships that left Earth years ago. Even though the fleet eventually found other planets for them to live on, many are content to continue living out in space. It's a neat little slice of life book about this community doing their part to keep these ships going.

All tomorrow's by c.m koseman may be interesting to you. It's a short story that examines the state of humanity several billion years in the future after they have evolved to be unrecognizable. Some civilizations thrived and became better, many devolved and live tortured existances. Quite a few lose the ability to speak or lose intelligence in general.

Saving this thread for good book suggestions.

“The Dark Beyond the Stars” by Frank Robinson might fit for you. It’s set on a generation ship that can’t find a good landing spot.

People already mention the Mars series by Orson Scott Card, the Expanse series by Corey, and Seveneves by Stephenson, which are all fantastic and all fit your request well. Two others you might consider are:

• The Moon is a Harsh Mistress by Heinlein. Very old school classic that features a moon colony fighting with earth.
• Beggars in Spain by Kress. Most of it is on a near future earth, but the last hunk of it involves a segment of people relocating to a space station.

Tau Zero is essentially where eventually within a few months no hospitable worlds exist. This is due to a spacecraft being out of control and reaching relativistic speeds.

Completely different angle towards the question but Metro 2033 (and sequels) might be a good source of inspiration. Not space themed but there might be some elements that can be a source of inspiration.

It's about a post nuclear war Moscow where to survive humanity as set up a series of interlink communities in the underground metro tunnels. The book talks a lot about the daily life in the stations. One is known for growing mushrooms used in tea. One was burnt down leading the rest of the system to strictly control fires. Another gained a reputation as a capital like station because it's entrance was next to a university and government building.

Not a true hard sci fi book (has things like irradiated mutants) but a lot of thought went into the logistics of living in the metro.

Inverted World by Christopher Priest is kinda this.

Might not be quite "hard" enough, but perhaps try the Interdepency trilogy by John Scalzi.

Maybe not "hard" enough for you (eg. it has absibles) but Becky Chambers's Record of a Spaceborn Few is about life on a fleet of generation ships.

You might get some value from Stephen Baxter's Manifold series; while they aren't central to what you're asking for they definitely deal with humanity coping with a hostile universe. Heinlein wrote a couple of short stories set on a generation ship, although I can't bring the titles to mind. Accelerando takes place in many manufactured settings, rarely earth like (past the first major part). Brin and Benford's Heart of the Comet may also point in the direction you want.

I'm going to go the other way and recommend The Fifth Season, which is technically a fantasy trilogy but which has won both the Hugo and Nebula awards, because (as if that wasn't a spoiler) it's got a ton of sci Fi in it.

It's basically about people on a planet that keeps dying. They've had to deal with so many apocalyptic events that prepping for the next one defines the entirety of their civilization. If you want a window into the psychology of a society constantly on the verge of destruction, I can't think of a better series.