top 50 comments

sorted by: hot top controversial new old
[–] 62 points 10 months ago (3 children)

As a sysadmin that's pretty much my worst nightmare. I really feel for those trying to pick the pieces back up again after that.

  • source
  • hideshow 6 child comments
  • [–] 76 points 10 months ago (1 child)

    I feel your pain, but meanwhile I'm laughing. No backups, no fire suppression system, what the fuck are these clowns doing.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 59 points 10 months ago (4 children)

    Suppressing evidence? Looks like this was a deliberate oversight to later erase data.

    Probably data retention safety was overruled earlier by one person.

    So, I think the designed erase worked as designed .

  • source
  • parent
  • hideshow 8 child comments
  • [–] 20 points 10 months ago (1 child)

    Pretty big conspiracy theory for a situation where incompetence doesn't seem out of the ordinary.

    SysAdmins saying "We need X, we need Y", someone who signs the cheques saying "But it works fine! Why would there be a fire? We already spent so much on those UPSs"

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 10 months ago (1 child)

    I don’t know a lot at this time. The cool thing is this will be discussed for years , and any who follow this will know a lot more.

    I’m just saying if it looks like a duck, quacks like a duck it may not be a squirrel

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 10 months ago (3 children)

    What you described isn't "looks like a duck, quacks like a duck".

    This is a fire in a government (publicly-funded) location in an industry that's often overlooked and underfunded... in a traditionally conservative country (with a Liberal leader for the last few months) that is part of a very top-down hierarchical structure. Which would suggest that the complaints of a systems administrator are likely to be ignored. And that's assuming that the systems administrator is competent, which isn't a guarantee in and of itself.

    I'm seeing what looks like a duck, I'm hearing the quacks. The conspiracy theory is that it's a squirrel in a duck costume that learned how to quack so people would toss it bread.

  • source
  • parent
  • hideshow 3 child comments
  • load more comments (3 replies)
  • [–] 3 points 10 months ago

    Hanlon's razor mate. I know it's entirely possible that it's a conspiracy, but it'd entirely possible (and more likely) that it's just incompetence and lack of communication due to inefficient and lackluster processes as well as too many levels of leadership.

  • source
  • parent
  • [–] 2 points 10 months ago* (1 child)

    You know for a fact that the people doing the largest share of the recovery effort have nothing to do with the decision to have no backups.

    ...but with the way social/work hierarchies work in SK, it was probably never brought up.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 10 months ago

    You know for a fact that the people doing the largest share of the recovery effort have nothing to do with the decision to have no backups.

    Exactly. How they got there is no consolation to those dealing with it now.

  • source
  • parent
  • [–] 56 points 10 months ago (1 child)

    Wow. this is one of the biggest instances of IT incompetence that I've heard of in recent years. Hosting a server farm without remote backups? Sound like the London Magnetic Tape Incident.

    The LMTI: One employee was sent to the other end of London with the magnetic backup tapes every day. He got money to take a taxi, but saved it and took the tube. His favourite seat was right above one of the motors, where he sat the bag with the tapes on the floor for the journey. The tapes were just stored at the destination, and not checked in any way. Guess what they learned the first time they had to rely on those tapes?

  • source
  • hideshow 2 child comments
  • [–] 39 points 10 months ago (1 child)

    Korean government's

    South.

    (Just in case anybody else isn't a clairvoyant either)

  • source
  • hideshow 2 child comments
  • [–] 41 points 10 months ago (2 children)

    I'll be honest, I assumed it was the South because I would be honestly shocked if the North ever reported anything bad happening to itself.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 10 months ago (1 child)

    “Cloud storage” in the North is just a drawer full of usb sticks confiscated from people smuggling K-dramas in from the South.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 10 months ago*

    The DPRK has their own intranet called kwangmyong, which needs its own storage and servers. Citizens use it regularly, access to the broader global internet however is more controlled and limited to specific jobs and positions as far as I know.

  • source
  • parent
  • [–] 31 points 10 months ago

    Hmm a system that stored government documents caught fire and they have no backups? Hmm, this carries the same energy of registry offices catching fire "spontaneously" in the past.

  • source
  • [–] 29 points 10 months ago (2 children)

    Always have two backups in different places than the original. If not, the least you can do is have one backup copy. How does the government don't have such thing?

  • source
  • hideshow 4 child comments
  • [–] 5 points 10 months ago* (2 children)

    Government people get jobs by schmoozing and making deals, not by merit or skill.

    Pournelle's Law always seems relevant.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 10 months ago

    I have one backup but keep legacy things so that in a massive disaster I still have archives that have some of my important long term type documents. So figure one up to date backup and in a disaster I have stuff from last year.

  • source
  • parent
  • [–] 23 points 10 months ago (1 child)

    This is a clear example where 'the cloud' really is a physical place.

  • source
  • hideshow 2 child comments
  • [–] 2 points 10 months ago (1 child)

    What else would it be? What do people think the cloud is?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 10 months ago (1 child)

    For a long while, I had hoped it was at least 6 physical places, with various redundancies. A few billion small-ish servers at internet network hubs.

    That or the magical floating bits that go over hackers heads in the movies. Those also look like the cloud. Not very secure, but quite convenient.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 20 points 10 months ago

    Perhaps contained evidence against the former PM who tried to do a military coup and this is the way of getting rid of it? Perhaps protecting co-conspirators if not the top guy.

  • source
  • [–] 20 points 10 months ago (1 child)

    At least they saved the Internet box. Otherwise, we'll have to bother the elders of the Internet again.

  • source
  • hideshow 2 child comments
  • [–] 14 points 10 months ago

    Yeah who needs proper DR. it's not like disasters ever happen.

  • source
  • [–] 8 points 10 months ago (2 children)

    Disaster Recovery Concepts:

    • Recovery time objective (RTO): maximum time to restore system function.
    • Recovery point objective (RPO): maximum age of data needed to resume operations.
    • Recovery consistency objective (RCO): how many inconsistent entries are allowed in recovered data.
    • Sierra Madre objective (SMO): We don't need no stinkin' backups.
  • source
  • hideshow 3 child comments
  • load more comments (1 reply)
    [–] 7 points 10 months ago (1 child)

    Too slow storage to back up? What a stupid, false reason. I assume nobody works at night. Do something else than full backups and you at least have something. A simple differential update replication would have saved them here.

  • source
  • hideshow 2 child comments
  • [–] 3 points 10 months ago

    Or do it by priority. Files that change often or are very important are copied fully often, maybe daily. A differential update of all files could follow daily or who knows weekly. Its the government, they should have money to add more storage, so that shouldn't be the problem. At least some strategy to manage slow speed, instead not having ANY backup its the dumbest thing I've read from governments in a while.

  • source
  • parent
  • [–] 6 points 10 months ago

    Even though they historically used to have a great backup system for their historians and government records.

    see link: https://en.wikipedia.org/wiki/Veritable_Records_of_the_Joseon_Dynasty

  • source
  • [–] 4 points 10 months ago

    Weird how there is a rising MAGA party in KR and an aligned former pm who attempted a coup. What a weird coincidence.

  • source
  • [–] 4 points 10 months ago

    Kein Backup, kein Mitleid!

  • source
  • [–] 2 points 10 months ago*

    It's too bad it wasn't a rain cloud or it would have destroyed the fire and not the other way around

  • source
  • [–] 2 points 10 months ago

    Always backup!!! From the fire, I mean, place your safety first

  • source
  • [–] 2 points 7 months ago

    saw a sticker recently that said "there is no cloud, just other people's computers"

  • source
  • [–] 1 point 10 months ago

    So this is the "maintenance" the gov services are currently having.

  • source
  • [–] 1 point 6 months ago

    Great talk that touches on this from CCC

    https://www.youtube.com/watch?v=0dMzlw-qSZI

    TLDR it was to destroy evidence the day before a government investigation.

  • source
  • load more comments
    view more: next ›