1
 
 

@ray@lemmy.ml Got it done, I'm first of the mods here and will be learning a little Lemmy over the next few weeks.

While everything is up in the air with the reddit changes I'll be very busy working on replacing the historical pushshift API without reddits bastardizations should a PS version come back.

In the mean time you should all mirror this data ensuring its survival, do what you do best and HOARD!!

https://the-eye.eu/redarcs/

2
 
 

cross-posted from: https://lemmy.ml/post/49741495

After the recent post about thousands of chiptune music from cracking teams in mp3 format, I wanted to share an already well known collection of Commodore 64 music in their original tracking format. To play these SID format music files, an audio player that supports the format is required.

You only need to download a single complete package, that is in size 80 MB for 7z format or 97 MB for RAR.


61157 SIDS

About HVSC

The High Voltage SID Collection (HVSC) is a freeware hobby project which organises Commodore 64 music (also known as SID music) into an archive for both musicians and fans alike. The work on the collection is done completely in the Team and contributors' spare time and is proudly one of the largest and most accurate computer music collections known.

3
2.5 vs 3.5 HDD (sh.itjust.works)
submitted 1 month ago* (last edited 1 month ago) by to c/datahoarder@lemmy.ml
 
 

I'm looking to get a new internal hard drive for a specific use case:

  • it's for a desktop, meaning I can fit either
  • i need 2TB, no more and no less, also meaning I can get either
  • it's for my media collection, mostly movies
  • it's not gonna be read-only, but I wouldn't write too often (I would torrent, but all the program's metadata would be on another drive)
  • most likely gonna be formatted with btrfs (subject to change, open to suggestion)

Should I get the 2.5 inch or the 3.5 one?

Thanks in advance.

4
 
 

Collection of pre-patched Super Mario World Romhacks and Mods from SWMDB.me 202107 By Storage Manager (Repack)

Uploaded by Thingsiplay

Originally uploaded by Storage Manager at https://archive.org/details/super-mario-world-hacks (defunced now). This is a repackage of the same files. The original collection contains over 29 thousand files, only compressed as individual files. This means that Internet Archive's internal "on-the-fly-zip" doesn't work and each file has to be downloaded individually. I downloaded all files and packaged them up into multiple archives based on their category directory they were sorted into. Also the original source no longer exists, therefore this is a reupload for archival purpose.

Legacy information from original upload:

Complete: Entire collection of original upload. Each category is archived separately for easy download. The original 14 GiB full set of 29k compressed Roms. Every single Rom file is compressed as 7z.

Reduced: A script clean_smwdb.sh tries to detect multiple versions of a Rom and keeps only the newest entry of it. Furthermore bad files and unneeded categories (such as Jokes and Trash) are deleted. This reduces the total number of files from 29k to "only" 8k uncompressed Roms. Also while each category is still packaged for batch downloads, every single Rom file in it are already extracted (unlike the Complete packages).

5
 
 

YouTube is still one of the major points of centralization on the internet, so I've been brainstorming ways around the problem.

From the readme:


Torrent-Tube is a set of tools to help decentralize YouTube videos, by moving them to torrents, which can be shared by many people. It includes:

  • A Torrent-Tube search site which searches the Torrents-csv search engine to see if the given YouTube video already exists, and is being seeded.
    • It does this by extracting the YouTube [VIDEO_ID] from a link, which you can also do manually if you like (IE, the text after watch?v=...).
  • A script to download, and create torrent files from YouTube videos, with a uniform naming style and format, taken from TheFrenchGhosty's YouTube-DL-Scripts.
  • You will need to upload these torrent files yourself to a service (details below), and seed them.

Torrent-Tube Search

In the future, it may be possible to create a browser plugin that checks a video link that you're currently watching for existing torrents.

Create torrent script

Requirements

Instructions

Copy a YouTube video URL.

# Clone this repo
git clone https://github.com/dessalines/torrent-tube

# Run the script
./create_torrent.sh [YOUTUBE_URL]

The video will download, and is saved in the videos folder. The torrent file is saved in the torrents folder.

Add the torrent to your torrent app, such as qbittorrent.

6
 
 

geteilt von: https://feddit.org/post/31252728

Dear Fedi friends, I need your help.

If by any chance any of you have saved the video of #WSocial CEO Anna Zeiter's presentation at the Monaco AI conference (uploaded to YouTube on May 21) I would be ETERNALLY grateful.

OG URL: https://youtu.be/xDNGHaISqs4

The 35 minute video included priceless quotes like: "Yes we have a business model, it's not a philanthropy, we are not a B corp, we actually want to make money."

Following the publication of my exposé, the video has been removed from YouTube!

7
 
 

I have found a few on 4chan. A rotating realistic chicken burger with a transparent background for example.

It seems to be mostly the same 20 or so GIFs in regular circulation.

Ideally I'd like to avoid individual downloads or signing up for an account. I thought someone here may have stumbled across a page on the Internet Archive or something.

8
 
 

1. Purpose

  1. Internal Disk - OpenZFS/ext4/NTFS ? for all OS (Win,Lin,mac)
  2. External Disk - Is exFAT OK ? Else OpenZFS/ext4 ? for all OS
  3. Backup Disk - guy says exFAT is bad for backup as it corrupts. so ***ext4/***OpenZFS ?
  4. SOHO Server/NAS - ZFS

2. OpenZFS vs BtrFS?

3.1 Is ZFS on Win/mac mature enough ?

3.2 If not Is Ext4 on Win/mac good ?

9
 
 

https://archive.org/details/nes-romhack-collection_202103-repacked

Total size of all 24.258 .nes Roms (plus a few other files) unpacked: 5.5gb


A huge collection of NES Rom Hacks.

Originally uploaded by MiroMirian at https://archive.org/details/nes-romhack-collection_202103 . This is a repackage of the same files. The original collection contains over 24 thousand files, without compression. This means that Internet Archive's internal "on-the-fly-zip" doesn't work and each file has to be downloaded individually. I downloaded all files and packaged them up in .7z archives. This reduces the download size drastically and also makes it possible to download all of them in one go.

Complete: Entire collection in a single archive. The original full set.

Split: Divided the collection by letter groups into multiple archives. Also removed all non .nes Rom files for additional cleanup. Otherwise contains the same Roms found in the Complete package.

10
 
 

Example

  1. sometimes foobar2000 randomly changes some settings and things break, I don't want to create duplicates (to save space) instead I want to track changes in it.
  2. Same for chromium.

I say select because I don't want the imaging solution

11
 
 

These software have option to config directory using .ignore file

  1. Kopia - .kopiaignore
  2. Restic (hard to use)
  • --exclude-file
  • Ignore File: create a custom text file e.g., excludes.txt
  • .exclude_me supports --exclude-if-present, which lets you skip any directory containing a specific file, such as .exclude_me.
  • Uses gitignore-style syntax
  • wrapper like Restatic - crawls directories before Restic starts and looks for .restaticignore files. It merges all those local files into one big exclude list and passes it to Restic automatically.
  1. Duplicacy .duplicacy

https://github.com/gilbertchen/duplicacy/wiki/set


Rclone (Only ignores no option to config)

  1. --exclude-if-present .rcloneignore (any name)
  2. Global --exclude-from list.txt

Why is it only recommended for "Large-scale cloud copies" and "Google Drive/Dropbox/S3"?


Borg (Only ignores no option to config) .borgignore


FreeFileSync (Maybe global only).ffs_gui or .ffs_batch


How to set up like this with features -

  1. Forever full backups/DeDuplication - Option Delete changes older than x. Where full-backup is taken once then incremental-s are merged. I have heard terms like, snapshot and CBT but don't understand it.
  2. Save a "Ghost" for excluded data ie only Filename, Metadata and Folder-Structure.
  3. "File Change Tracker" to see summary of what files are moved/deleted/renamed between 2 backups. Like Kopia's https://kopia.io/docs/reference/command-line/common/diff/
  4. "File History" where I see previous version of files on the main disk. Like Kopia's kopia snapshot list <filename>

To Backup

  • External and Internal Disk (files) to backup,
  • all separately backup-ed to the same backup disk, .

Old Post: https://www.reddit.com/r/DataHoarder/comments/1rzmg67/discussion/_backup/_solution/_comparison/

Duplicacy has a -nobackup-file

12
 
 

publication croisée depuis : https://lemmy.dbzer0.com/post/67080379

Hello, since it's complicated to index DHTs, I figured it'd be more efficient to build an index of fingerprints from real data once.

So I've been collecting releases hashes for this index. It can be used for various purposes:

  • check the integrity of your own files (bit rot is a real thing)
  • identify BTv2 torrent files that contain specific files (a database of torrent files is required)
  • locate alive IPFS swarms to join more easily (no need to read all your data multiple times to recompute various CIDs yourself)

The collection contains around 1K releases and weights 40MB. I've prioritized scene Bluray rips of movies (1080p / 2160p). No infohash will be included, as these are not reproducible enough.

I'm using a basic script to add a new release (filename must match the official release name). I'm using others to discover scene releases in a filesystem; retrieve official release names from files using the srrdb api (crc32 search); collect torrents from Prowlarr and H&R them (although I'd prefer crowd-source directly from the community!).

The index is stored on git to allow collaboration. It is hosted using Radicale to avoid centralization and reduce hosting pressures.

If you are interested, join and add your own hashes to the collection in Radicle patches! (see instructions in the README)

Let me know what you think, suggest improvements or discuss similar projects you know about!

13
 
 

cross-posted from: https://lemmy.ml/post/46121046

so i've been searching for a long time for an image tagging software, there are some tools out there, some are electron based so no thanks, then there's Hydrus which is actually very similar to what this does, but the GUI is horrible and the installation is way more complicated. I created a docker compose file and I was running Blombooru in a few minutes. So if anyone was looking for something like this to organize their system, please give it a try because it is really good!

14
 
 

Crossposted from https://thebrainbin.org/m/datahoarder@lemmy.world/t/1576555

After the original post, I ordered the generic reader I mentioned and it arrived today.

Testing around, and it can generate encrypted ISO files just fine with Linux Mint's default Disks program. No ddrescue needed!

Also apparently one of the discs I have had the key shipped with the reader revoked. But using the issue as an opportunity to test MakeMKV, I created a decrypted backup with it and tried opening the backup with VLC.

This decrypted backup seems to behave just like the encrypted ISOs I had been creating, thankfully, though further tests may be needed as I am still very unfamiliar with this revocation situation.

But all in all, I'm happy I can finally do my backups properly. In retrospect, the PS3 is too moody and slow even when the hardware is healthy, making backing stuff up with it a rather miserable process. Better something fast, more predictable, and that isn't tied to FAT32 with gambiarras applied to work with NTFS.

And on a side note, I also tested the reader with a DVD for the same purpose and it worked fine, so I guess it'll be working overtime. e.e"

15
 
 
  1. Kopia .kopiaignore
  2. Duplicacy .duplicacy
  3. Borg .borgignore
  4. FreeFileSync .ffs_ignore

How to set up like this with features -

  1. Forever full backups/DeDuplication
  2. Option Delete changes older than x.
  3. How to Only backup Select Data, like only personal data,
  4. Save a "Ghost" for other data (internet data not personal), which is only Filename, Metadata and Folder-Structure.
  5. "File Change Tracker" to see summary of what files are moved/deleted/renamed.
  6. "File History" where I see previous version of files.
  7. Config from inside folders for disks (not OS) with e.g..backupconfigfile containing e.g. backup=1, or have select file backuped/not-backed-up.

To Backup

  • External and Internal Disk (files) and OSes to backup,
  • Backup select data from Disks,
  • all separately backup-ed to the same backup disk, .

Old Post: https://lemmy.ml/post/44707979

16
 
 

Is it a viable option?

I've gotten a few discs, and up to some years prior, I'd use the PS3 to back them up. However, as its HDD is starting to fail (took it long enough e.e''), I'd been considering getting a bluray reader for dumping the ISOs, possibly a Wabcom 5-in-1 but still evaluating that part.

But as I am on Linux Mint, and I don't mind the ISOs being encrypted (iirc I'd use FOSS keys and they'd work fine), would ddrescue be enough for that, or would I need to use some other program, perhaps even some dedicated/paid one?

Also bonus question: I also got CDs and DVDs to backup. Would I be able to do it for them too?

Thanks in advance!

17
 
 

https://archive.org/details/sms_mods_and_romhacks_collection_20260409_patched

My personal collection of Master System Romhacks, in an already patched and ready to play ROM format. Most games are patched by myself, but not all are tested. Each .sms file comes with a text description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

  • 110 Romhacks across 53 different games (or across 52 games, depending on how you process data and count).
  • Download one package size: 9.7 MB
  • Unpacked size: 44 MB

flat structure: mastersystem_mods_and_romhacks_collection_20260409_patched_flat.7z

    mastersystem_mods_and_romhacks_collection_20260409/
        Alex Kidd in Miracle World_Snappy Snorg and the Seven Silver Stones v1.4.sms
        Alex Kidd in Miracle World_Snappy Snorg and the Seven Silver Stones v1.4.txt

or sub structure: mastersystem_mods_and_romhacks_collection_20260409_patched_sub.7z

    Master System Mods and Romhacks Collection 2026-04-09/
        Documents/
            Alex Kidd in Miracle World/
                Snappy Snorg and the Seven Silver Stones v1.4.txt
        Games/
            Alex Kidd in Miracle World/
                Snappy Snorg and the Seven Silver Stones v1.4.sms

Both contain same files, just different file structure.

18
 
 

If you archive emails to S3 (via AWS SES inbound or other pipelines), QuickMailBites lets you browse them with a proper native email client.

No need to write scripts to pull emails from S3 anymore — just configure the bucket and folder prefix.

19
 
 

It's redirecting to a new site completely, does anyone have a copy of the old source?

20
 
 

https://archive.org/details/n64_mods_and_romhacks_collection_20260404_patched

My personal collection of Nintendo 64 Romhacks, in an already patched and ready to play ROM format. Most games are patched by myself, but not all are tested. Each .z64 file comes with a text description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

  • 207 Romhacks across 38 different games (or across 31 games, depending on how you process data and count).
  • Download one package size: 2.3 GB
  • Unpacked size: 5.7 GB

flat structure: nintendo64_mods_and_romhacks_collection_20260404_patched_flat.7z

         nintendo64_mods_and_romhacks_collection_20260404/
            Super Mario 64_Mario Builder 64 v1.1.txt
            Super Mario 64_Mario Builder 64 v1.1.z64

or sub structure: nintendo64_mods_and_romhacks_collection_20260404_patched_sub.7z

            Nintendo 64 Mods and Romhacks Collection 2026-04-04/
                Documents/
                    Super Mario 64/
                        Mario Builder 64 v1.1.txt
                Games/
                    Super Mario 64/
                        Mario Builder 64 v1.1.z64

Both contain same files, just different file structure.

Nintendo 64 emulator compatibility with Romhacks is a bit wonky. Sometimes they work out of the box and sometimes they require specific settings, and other times they just don't work on my setup.

I play Nintendo 64 games with Mupen64Plus-Next core on RetroArch. Two distinct configuration files are included. These Romhacks are quickly tested for compatibility and categorized into one of these setups. The configuration is not required and are included as a reference. There is no guarantee that the included Romhacks will work.

21
 
 

Gaming Historian didn't post in 3 years and this is his goodbye video today: https://youtu.be/nV_Aww8_6wQ And his last gift about an unfinished video of him, he releases said documents: https://archive.org/details/universal-v-nintendo-court-documents

Gaming Historian has some of the highest quality gaming related documentations. I highly recommend watching past episodes.

22
 
 

Windows/Linux to Android - on PC i will create a folder of data to be synced created via SymbLinks

Android to Windows/LAN - Only sync select data preferably via WiFi

23
 
 

https://archive.org/details/md_mods_and_romhacks_collection_20260320_patched

My personal collection of Mega Drive / Genesis Romhacks, in an already patched and ready to play ROM format. Most games are patched by myself, but not all are tested. Each .md file comes with a text description, copied from the places where I downloaded the Romhacks (but sometimes also from README files, random blogs and other websites too).

  • 421 Romhacks across 166 different games (or across 163 games, depending on how you process data and count).
  • Download one package size: 182 MB
  • Unpacked size: 815 MB

flat structure: megadrive_mods_and_romhacks_collection_20260320_patched_flat.7z

     megadrive_mods_and_romhacks_collection_20260320/
        Sonic_Character Pak v1.0.md
        Sonic_Character Pak v1.0.txt

or sub structure: megadrive_mods_and_romhacks_collection_20260320_patched_sub.7z

        Mega Drive Mods and Romhacks Collection 2026-03-20/
            Documents/
                Sonic/
                    Character Pak v1.0.txt
            Games/
                Sonic/
                    Character Pak v1.0.md

Both contain same files, just different file structure.

24
submitted 4 months ago* (last edited 3 months ago) by to c/datahoarder@lemmy.ml
 
 

How to set up like this with features -

  1. Forever full backups/DeDuplication
  2. Option Delete changes older than x.
  3. How to Only backup Select Data, like only personal data,
  4. Save a "Ghost" for other data (internet data not personal) which is only Filename, Metadata and Folder-Structure.
  5. "File Change Tracker" to see summary of what files are moved/deleted/renamed.
  6. "File History" where I see previous version of files.
  7. Config from inside folders for disks (not OS) with e.g..backupconfigfile containing e.g. backup=1, or have select file backuped/not-backed-up.

To Backup

  • External and Internal Disk (files) and OSes to backup,
  • Backup select data from Disks,
  • all separately backup-ed to the same backup disk, .
Software FOSS Enterprise OS Encrypted GUI MultiMachine Dedup Snapshots Scalable Schedule Image Lesson
Restic Frontends - Resticprofile, 2 , Backrest (garethgeorge), restic-browser) [zerobyte (nicotsx)-Video, Automation, UI, schedule, manage, Monitor] 🟢 🟢 🟢 🟢 🟢 (Seems like is best it's Old and trusted)
urbackup (seems Powerful, some people love it some say it's not reliable) [Backup/Imaging] 🟢 🟢 🟢 Server/Client, ChangeBlockTracker , Lesson , https://christitus.com/urbackup/
Duplicati 🟢 🟢 🟢 Data issues
Freefilesync.org 🟢 🟢
Minarca 🟢 🟢
plakar.io 🟢 🟢
syncBKUP (Jim-JMCD) 🟢
Bacula 🟢 🟢 🟢 🟢 🟢 🟢 🟢 🟢 🟢 Lesson
Bareos (Bacula Fork) 🟢 🟢 🟢 🟢 🟢
Kopia 🟢 🟢 🟢 🟢
vykar 🟢 🟢 🟢 🟢 🟢 Rust, YAML config, Support for S3, Custom REST, SFTP Storage. Inspired by BorgBackup, Borgmatic, Restic, Rustic.
Pika 🟢 ❌Windows https://www.youtube.com/watch?v=W30wzKVwCHo
Borg (borgbackupserver) 🟢 ⚠️Windows)(cygwin/WSL) ⚠️macOS 🟢 🟢
Duplicacy github source-available 🟢 🟢
BackInTime (rsync frontent for backups) ❌Windows
blinkdisk 🟢 🟢
Veeam (Free) 🟢 ⚠️macOS 🟢
Backblaze - -
zfs_autobackup 🟢
eXdupe (rrrlasse) 🟢
zpaqfranz (fcorbelli) 🟢
VaultSync (ATAC-Helicopter) 🟢
https://bvckup2.com/ Maybe Not
https://www.nakivo.com/ Free

Dead but FOSS

https://github.com/zmanda/amanda

https://en.wikipedia.org/wiki/List/_of/_backup/_software

Freemium

Software OS
uranium Windows
SyncBack Free Windows

Backup and Imaging

Software Foss
ShadowMaker Free
Paragon Backup
MSP360
Macrium Free
Acronis Free

Disk Imaging

Software Foss Imaging Backup OS
Veeam Agent for Microsoft Windows
Rescuezilla (clonezilla) 🟢 🟢

Sync Software

Foss OS
Rsync https://linux.die.net/man/1/rsync 🟢 All
RClone - ( https://www.youtube.com/watch?v=QKCIi-NxJEo ) 🟢 All
ByteSync 🟢 All
FreeFileSync 🟢 All
SyncBackFree Windows
Syncthing 🟢 All

Cloud

https://www.reddit.com/r/Backup/wiki/index/cloud/_backup/_services/backblaze/

Info

Incremental backup method, where I only make full backups once ,

Question - as i understand from "grandfather,father,son" method - that Full backups are still necessary when using snapshots/incremental backups, why is that ?

"deduplication" is of 2 types

Data/Block de-duplication - tech to reduce amount storage required. This breaks files upon into chunks and creates a DB. Data de-duplication is influenced heavily by data type. Data de-duplication is a waste of time with compressed multimedia and encrypted data. Data de-duplication is often incorporated into backup products and can exist in independently, its built into some file system types (Windows server and some Linux).

File deduplication - Removing identical files. by comparing the whole file. Homelabers and home users are more worried about this than commercial environments. In commercial environments users and projects are usually allocated a quota storage space and its up to them how they want to manage it.

Incrementals Forever and Synthetic Full Backup - incremental may/not have de-duplication. They can also use Change Block Tracking (CBT) to save a lot of backup time.

"Forever Full" are simply a variation on synthetic full backups with data deduplication and CBT being optional.

Data deduplication occurs when you:

  • Have multiple copies of the same data across multiple machines e.g. the operating system files of the computers you are backing up.
  • Data that has not changed since you last successive backup. This includes files that have partially changed, only new unique data is added to the dedupe database/repository. Old Post about methodology - https://lemmy.ml/post/44433232
25
 
 
  1. I like Incremental backup methodology, but it needs frequent full backups (as i understand from "grandfather,father,son" method), How to have version control where i only create full-backup only once ?

  2. and I can choose to Delete changes older than 1 month.

  3. How to Only backup Select Data, like only personal data,

  4. and a "Ghost" for other data. Ghost is only filename and it's metadata (also folder structure). Data selected for Ghost is from internet which can be downloaded.

Related

  1. "file change tracker" to see summary of what files are moved/deleted/renamed.

  2. "File History" where I see previous version of files.

Software ?

  1. Seems like https://restic.net/ is best as it's enterprise trusted works on all OS,

How do I set it up like I described in the Original Post.

I have a external disk and internal HDD, I want to backup select data, to a 3rd disk, both backup to the same disk, both separately backup-ed.

view more: next ›