Restore files on headleass server

[OiPenguin]

Coming into this late, so I didn't read through the long post details, just skipped over the summary of it.

I don't see that as a problem with RAID itself (just another factor), but rather that you used an rsync option that deleted the files, once it confirmed that is backed them up (one-by-one)...

So you have "a backup" of them correct? Or was there a problem with the files rsync copied over? They could just be restored.

If you didn't have good copy of the files, then use "testdisk" or "photorec" to recover them.

I'm afraid I don't have a proper backup.
- The most outdated confirmed backup is 2000 (start) to 2013. Meaning 10 years is lost if this is the only one I'm able to recover.
- Another backup is a server with a (most likely) faulty power supply. My guess is that this is a backup from 2000 to approximately 2017, however I've yet to find out. The box has two 1TB disks set up as raid1. I've got an External sata hard drive case, but was unable to mount the disks on my Ubuntu-laptop, probably because the disks are part of a pair of Raid1 disks. Can I use testdisk to mount Raid1 disks in Ubuntu or do I need a further procedure? (I'm also searching for a new power supply, which I assume will be an easier solution.)
- The disks of the main server has been pulled out and delivered to a professional for assessment and a price quote.

Yours,

Lars

[dragonfly41]

After hopefully the recovery shop pulls your data out of the fire I would sit back and think carefully about your future architecture. Read here .. https://www.reddit.com/r/editors/com...th_it_in_2023/

Do you need the hassle of RAID boxes? You can achieve redundancy in different ways and allowing you to take routine backups. That is why I suggested the StarTech route.

The old RAID setup will be in some container. What is it?

Plan ahead where you will keep backups. Spread them in multiple locations. Remember the "deadman's handle" protocol. What happens if you fall under a bus. So recovery instructions must be documented too. Passwords?

Your server (spec?) can have an array of multiple plugin caddies, not necessarily bound into your server tower casing.

Remember that if you try PhotoRec you will have a bunch of renamed files. Further detective work needed to find the content. That is why I am taking a conscious step to place my assets into containers to aid last resort recovery.

There is a forensic tool I have just found which creates a forensic tree of assets. It s named ftree. It creats a json file of all filses in a location. Search HACK42 and DART. But this will not recover your data.

In conclusion give more information about the hardware containers of your RAID since others might have ideas. As written in link above RAID does not guard against human failure. Suggested use of GUI (Grsync, Lucky Backup) instead of command line but even there have a verification script that the profile is sane and follows best practice.

Having a local professional shop is your best hope.

Burn n+1 redundancy into your plans.

[dragonfly41]

Any progress with the recovery shop? I am returning here since I remembered this recovery tool with good references .. although caveat I have never used it.

https://www.grc.com/sr/testimonials.htm

Contact old hand Steve Gibson and explain your plight.

[OiPenguin]

Any progress with the recovery shop? I am returning here since I remembered this recovery tool with good references .. although caveat I have never used it.

https://www.grc.com/sr/testimonials.htm

Contact old hand Steve Gibson and explain your plight.

Thank you. That looks like a very interesting program, although the website looks like its developed 20 years ago. I'm reading about the program now and I've not gone through all the information yet, but I have two questions which I've been able to answer:
- Can I make a bootable USB of this (Windows?) program from my Ubuntu machine?
- Can this program recover from disks set up with Raid?

You wrote contact old Steve Gibson. Are you suggesting that I actually contact him or rather that I purchase the software?

Regarding progress. The shop which has the disks updated me yesterday. After 3-4 days the disks are 65% analysed so I'll hear from them in the start of next week.

[dragonfly41]

I was thinking that you might tap into his sphere of knowledge, possibly by trying to email him or via his forum. His research goes back to the days of spinning disks but has been updated to look at SSD performance. If you follow some of the YouTube chats his experience is interesting.
But .. reflecting overnight .. I'm not sure if that software can help with a human failure in inadvertently deleting files with rsync. It might help other scenarios (cited often) where SSD's give up the ghost early in life. So we have two scenarios: hardware failure and human failure. Keep up the research.
Good luck with the recovery. And of course you are planning now backup strategy.

[POSTSCRIPT]
But again .. I've just read advice from @TheFu here post #6 on that suggestion so I will withdraw the idea.

[OiPenguin]

An update.

My disks are still with the local shop. I'm waiting for the result of the analysis and don't know why it takes so long time, nor the price of recovery.

In the meantime I've received a quote from another company. They charge $220 per disk for analyses. I've got three disks (the want to analyse the SSD with the OS on as well) which means about $660 for the analyses of three disks. Plus a further $2500 - 4500 for reconstruction of the data. When I enquired about the steep price the explanation was that Raid disks are more complicated.

Hence, if anyone has a suggestion as to how to to analysis and recovery on my own, please let me know. It is a viable option to invest some money, in hardware and software, to do this giving the extreme price estimate I've been given.

[dragonfly41]

It seems that we are in the last chance saloon.

The first "local shop" is taking too long to assess. I would have asked up front for (a) references and (b) an estimate.

I think that you will have to take the risk to explore forensic recovery process yourself and learn from the experience (whether it succeeds or fails).

The next step is perhaps to try recovery through TestDisk and PhotoRec. If any images are recovered they will need to be categorised using labelling methods but we can consider that hurdle later if TestDisk/PhotoRec succeeds.

You will need a fresh drive, at least equal in size to your RAID, into which you might save recovered files. So get to the point where you have:

TestDisk/PhotoRec installed on LiveUSB (not RAID) and running on a LiveUSB.
Your RAID (or preferably a clone of your RAID) connected
A fresh drive waiting in sidewings to receive any recovered data.)




You wrote earlier:

- I'm not sure which type of Raid I'm using and don't know how to find out. Anyway, as I understand from other threads, Raid protects against disk failure, not errors like I've made. The files (or the reference to the files) will be deleted on all partitions almost instantly.

It is vital that you understand type of RAID, ask the local shop which worked on it .. but gave you no results. Not hopeful with that route.

Some of recovery applications assume that you cannot access your RAID. But you made the error in deletlng so it is not hardware failure.

Here s one product which requires Windows to run.

https://hetmanrecovery.com/download.htm?product_type=4

Note: at this site a docking bay is suggested.

https://hetmanrecovery.com/recovery_...-from-raid.htm

I use StarTech dual docking bay for multi drives (but have no experience with RAID recovery).

The first suggestion, although you gain advice from forum, is to subscribe to your own AI assistant. The one I recommend is PHIND.com. The reason is that links to various references are given whereas other AI assistants only offer text replies.

This will cost you £10 per month subscription which you can close later after taking advice.

Remember that the "prompts" or "queries" for each session must be crystal clear.


Examples follow:

What approach is recommended for a sole developer attempting to recover important family photos deleted in error. There is no hardware failure only human failure. Can deleted data held in RAID in Ubuntu be recovered? What tools are needed for self recovery since professional recovery shops are too expensive.

If deciding to use Photorec it is understood that there will be no filenames attached to any recovered images. Are there methods for trying to relabel such recovered images? Perhaps an automation script allowing images to be viewed by an AI agent to caegorises into buckets. What AI scanning tools are suggested?

Can AI scanning methods be pre-trained with existing samples of photos?

Here is one earlier session I received:

"For recovering data deleted in error on a Ubuntu system, several leading data forensic tools are available, each with its unique features and capabilities. Here's an expanded list of tools, including those you're already aware of, along with their specific use cases and requirements for RAID data recovery:"

"TestDisk: A versatile tool that can recover lost partitions, undelete files, and fix partition tables. It's widely used in forensics due to its user-friendly interface and powerful recovery algorithms. TestDisk supports various file systems, making it a comprehensive solution for Linux forensics".

"Photorec: Part of the TestDisk suite, Photorec is known for its ability to recover files without their original filenames. It's particularly useful for recovering media files".

"Foremost: A forensic data recovery tool pre-installed in Kali Linux but can be installed on any Linux distro. It can recover data in multiple media file formats and from images created with the dd command. Foremost is designed for forensic data recovery, making it suitable for uncovering embedded or hidden data elements from images".

"R-Undelete: Offers IntelligentScan technology for data recovery, including MBR and GPT recovery, and boot sector recovery on major file systems. It's capable of recovering files on ext, FAT, and NTFS systems".

"SafeCopy: This tool is effective for recovering data from damaged and bad sectors on a hard drive. It performs low-level IO operations to extract maximum possible data and can fix errors in data on CDs, DVDs, and even floppy drives".

"Scalpel: A powerful open-source file carving tool that extracts deleted files based on their file signatures and headers. Scalpel supports various file systems, making it versatile for Linux file recovery".
"
The Sleuth Kit: A collection of open-source command-line tools for digital investigations, including utilities like Autopsy and Photorec. It supports various file systems, providing wide coverage for Linux forensics".

"ddrescue: A GNU data recovery utility designed to retrieve data from damaged or corrupted storage devices. It operates in a progressive rescue mode, recovering easier-to-read data first before tackling more complex segments. ddrescue provides an interactive map and logging feature to track the recovery progress ".

"HACK42: While not detailed in the provided sources, HACK42 is a tool designed for data recovery and analysis, often used in forensic investigations. It's known for its ability to recover data from various storage devices and file systems".

"Autopsy: Part of The Sleuth Kit, Autopsy is a graphical interface to the disk forensics tools. It's used for disk imaging, file carving, and indexing, making it a comprehensive tool for digital forensics".

"DART (Digital Advanced Response Toolkit): A digital forensics toolkit that includes a variety of tools for disk imaging, memory analysis, and network forensics. It's widely used in law enforcement and cyber forensics for its comprehensive set of forensic tools".

"R-Linux: A data recovery tool that can recover files from ext2, ext3, ext4, ReiserFS, JFS, XFS, and Btrfs file systems. It's designed to recover files from Linux systems and is known for its ability to recover files even when the file system is severely damaged".


"For RAID data recovery, special requirements and tools may be necessary due to the complexity of RAID configurations. Tools like ddrescue and Foremost can be particularly useful for RAID data recovery, as they can operate on images of RAID arrays, allowing for recovery attempts even when the RAID configuration is damaged or the RAID controller is unresponsive. Additionally, understanding the specific RAID configuration (RAID 0, RAID 1, RAID 5, RAID 6, etc.) and the file system used can help in selecting the most appropriate recovery tool".

"When selecting a data recovery tool, consider factors such as ease of use, compatibility with your system, the specific file systems you need to recover from, and whether the tool offers a demo or trial version. This will help ensure that the tool meets your needs and can effectively recover your deleted data".

[dragonfly41]

i should add a footnote. Since you will be using LiveUSB to be your "home forensic lab" by installing tools such s TestDisk/PhotoRec, ddrescue, R-Linux, and so on .. it should have reasonable size and importantly persistent. If not persistent you will have to reinstall these extra forensic apps every time you boot up. So search how to make persistent LiveUSB such as using MkUSB. And use USB 3.0 device and USB 3.0 port for speed.



Also glean information by studying multiple sites such as here (this one is in U.S.),

https://www.salvagedata.com/raid-data-recovery/

Note that data recovery vendors allow trials of their products to test if data can be recovered . without actually running the entire recovery. Avoid vendors who expect payment irrespective of whether recovery succeeds or not.

[dragonfly41]

Although this writer has zero experience with RAID devices the broad model developing in mind (avoiding third party forensic data recovery specialists) is to create a mentor/mentee framework.


The idea is:
Subject (mentee) creates a local persistent LiveUSB expressly for forensic data recovery experiments on local drives.
Helper (mentor) creates a repository of forensic tools to download into mentee's LiveUSB in a series of eLearning processes (starting with PhotoRec).
One advice offered earlier by TheFu is that write capability to the target RAID must be disabled at earliest stage to avoid the further risk of overwriting data blocks to be recovered. But there is a cost penalty. they are not low costs.
Some of the data undelete products require Window 10 and so the nettle must be grasped to install Windows 10 on the projected LiveUSB which in turn accesses the RAID.
Write blocker research:
https://www.opentext.com/products-an...000001nZXMPIA4
https://www.opentext.com/products/listing#t
https://www.opentext.com/products/ta...rensic-bridges
https://www.opentext.com/products/ta...idges#overview
Tableau Forensic Bridge (write blocker for USB).
https://security.opentext.com/tablea...re/details/t8u
https://security.opentext.com/video/...nd-Setup-Video
https://www.forensicfocus.com/forums...s-not-enabled/
https://www.forensicfocus.com/
https://ubuntuforums.org/showthread....+RAID+recovery
In further research into this discussion (since I do not use RAID) I learned about mdadm.
sudo apt install mdadm
man mdadm
Next I searched this entire Ubuntu forum with keywords:
mdadm RAID recovery
One of many threads
https://ubuntuforums.org/showthread....+RAID+recovery
read sage advice from TheFu.
Also i read about "write blockers". but it is debatable whether such an investment is worthwhile.
My current thinking is that you should use a local LiveUSB which can be primed from remote repo to experiment with different tools.
Some require Windows so a LiveUSB (Ubuntu) with Windows virtual machine might be worth building.

[dragonfly41]

Any progress with PhotoRec?
Setting aside the flood of references given earlier, another idea is for you to approach a local university in your country experienced in data forensics, talk to a professor in forensics and request to assign the recovery task to a forensics graduate as a case study.

https://www.noroff.no/en/studies/uni...ital-forensics