Hedge – The copy verification software just got a little smarter

Screen Shot 2018 04 24 at 10.01.47 AM

Hedge has become very popular with filmmakers who want an easy to use, reliable and cost-effective copy/verification program. I have been using Hedge ever since it was first released back in early 2016. Well, now they have a new update (Hedge 18.2) that features Duplicate Detection. Duplicate Detection checks if a file already exists in the Destination Folder, and if it does, Hedge skips it.

What is Hedge?

If you aren’t familiar with Hedge, it’s a program that allows the user to make multiple backup copies of their camera media and have those copies fully verified. The program uses a very simple user interface that is easy to use. You just select your camera media, and then drag and drop the drives you want to copy to over to the destinations window. There you can easily set the destination folder or create a new folder. So now that you know what Hedge is, let’s have a look at some of the new features that are in version 18.2.

Copy into existing folders

Until this latest version Hedge cloned data, as in, the destination folder was always an exact copy of the source, no exceptions. With cloning, it was not possible to add data to existing folders — each copy needed to be discrete. Now with Duplicate Detection, this is no longer the case. This is a feature I wish had of been included in Hedge from day 1. Often when I am shooting I may well stop for a break and then back up what I have currently shot. Once this is done I then put that same media card back in my camera and continue shooting. I do this for two reasons. One the card isn’t full and I want to keep using it, and two, I don’t like erasing a media card unless I absolutely have to. The problem with previous versions of Hedge is that it had no idea that all I wanted to do is copy the new files on that card and not recopy everything. Recopying files that are already backed up is a waste of time, particularly if you have lots of material.

It’s best to think of duplicate detection in the same way you would do an incremental backup. All Hedge is doing now is to copy only those files that don’t already exist in that destination. For a copy verification program just seeing if a file isn’t already there and then copying it is fraught with danger.  So when is a file identical?

Hedge uses five steps to determine this, in this exact order:

  • Does the folder structure exist?
  • Is there a file with the same name?
  • Is its size identical?
  • Do the creation dates match?
  • Do the modification dates match?

If one of these checks doesn’t match, the file is copied with an incrementer in the file name — as Finder and Explorer do.

Can you pause and restart a backup?

Unfortunately, one of the features I would like to have seen added to Hedge still hasn’t been implemented. Not being able to pause and then restart a backup can be problematic, to say the least. If you are backing up material in the field you may well find scenarios where you have to suddenly pack up your computer and move, or you may be low on laptop power. In these instances, you may well want to pause the backup you are doing and then be able to restart it again. The good news is that Hedge is working on adding this feature in a future version.

When incremental backups don’t work

Duplicate detection is very useful for clip and frame based cameras, but what happens if you are using a camera that creates a single metadata file, ie. a file that gets updated with each new clip? Well, there really isn’t an easy solution. For these type of cameras, you really should erase that media after each offload.

For example, if you do multiple offloads of the same Sony SxS card into the same folder, it will result in multiple MEDIAPRO.XML files: MEDIAPRO (1).XML, MEDIAPRO (2).XML, etc. If your NLE uses this file, you’d need to replace the original with the last copy, and then rename it.

So does it actually work?

Just because a program says it does something, doesn’t mean you should just take that companies word as gospel. I decided to test out the Duplicate Detection function to see just how well it worked. I tried copying material from an SD card, then putting that card back in the camera and then recording some more clips. When I went to copy/verify the card again, Hedge only copied the new files.

So far so good, but just to be sure I tried doing the same thing with a CFast 2.0 card.  I took a CFast 2.0 card that had a bunch of files already on it and then used Hedge to back it up. I then took that same CFast 2.0 card, put it back in the camera, and then shot some more material. Once I was finished I took that same card and went to copy it again in Hedge.

Here is where I ran into a problem. Once I backed up the CFast 2.0 and then added additional material to that same card, Hedge wouldn’t let me just copy those new files. All I kept getting were error messages. I asked Hedge about why this was happening and they told me that “It’s due to read-only drives that also have read-only folders. Hedge mirrors these permissions, which is in the way of Duplicate Detection.

Not only did they get back to me very quickly, but they also immediately started working on a solution to fix the problem. Within a day Hedge had released another update  (18.2.2) that completely solved the problem, Having a company that not only responds to your problem but then creates a fix for it is one of the main reasons I continue to use Hedge. I wish more companies took customer service this seriously.

Hash Verification

In Layman’s terms, file verification is the process of using an algorithm for verifying the integrity of a computer file. This can be done by comparing two files bit-by-bit, but that means it requires two copies of the same file, and the problem with this method is that it may miss systematic corruptions that may occur to both files. A more popular approach is to also store checksums (hashes) of files, also known as message digests, for later comparison.

Hedge does checksum verification, and a lot more too: it also cross-references all destinations and repairs any issues that are found. If you try deleting or editing a copied file while the transfer is still in progress, and you’ll see it reappear in its original state. This wouldn’t be possible without doing a re-read of the destinations. According to Hedge’s analytics engine,  a repair is done about once in 985K files, statistically speaking. There’s no correlation between file size, type or storage type; it’s entropy.
So if Hedge is doing checksum verification like most other copy/verification programs, why is it so much faster? Hedge is fast because it doesn’t do source verification. There is only a single read per copy; Hedge doesn’t verify if the source read is corrupted. For those who require or want source verification, it will be implemented into the next iteration of Hedge (18.3).  Choosing to use source verification will slow down the copy/verification process considerably, but it’s a necessity for some users.

Connect for Windows

For Hedge Windows users, Connectivity has been added. This works in exactly the same way as it does on the Mac version. Connectivity allows you to receive notifications of when your backups have been completed.

  • Generate a code, use that to connect your iOS device
  • Share the code with others in production
  • For security reasons, you can opt to auto-reset the code on quit.

Speed Toggle 🔄

This might be a small thing, but being able to toggle between an estimated completion time or a write speed is a nice feature. This is particularly handy for seeing if you have a hardware bottleneck.

Why you should use copy verification software

Protecting, backing up and verifying the video files that you shoot is often a step that a lot of people don’t take seriously enough. Too many times I have seen instances where crucial files were lost because someone failed to make a backup or didn’t correctly copy the media in the first place. Certain cameras can generate very complex media folder structures and if they aren’t all copied correctly you can run into a situation where an editor or client can’t access any of the footage. Often the camera operator has already erased or re-formatted the cards and moved onto their next job.

A lot of shooters I know just copy their camera media to hard drives without verifying it. They do this by just dragging and dropping files from a media card straight to a hard drive. Just straight copying files through the use of Finder or Explorer can be problematic, to say the least. This can be a costly mistake and when a client is paying for your services, you can’t afford for anything to go wrong when it comes to delivering the material you’ve shot. Verification is an important step because it looks at the original camera media and makes sure that every single file structure and the file is exactly replicated to wherever you are copying it to.

I mean why would you spend a small fortune on equipment and then be blasé about backing up the material you capture. At the end of the day, that material on those media cards is the fruits of your labour. You should treat your media, the backing up and protecting of that media like your life depends on it. Not taking copy/verification seriously is just lazy and disrespectful to your craft.

 Get Hedge 18.2 now

This latest Hedge update is free for anyone that bought or renewed Hedge less than a year ago. If your free updates expired, you can renew your Hedge license, or do it in-app.  If you do it in-app before your year of updates is over, Hedge will give you a 30% discount.

For those who don’t already own a copy of Hedge, you can download a free trial to test it out. The normal pricing options are as follows:

Project License $25 USD
30 days of Hedge, for the occasional job or short gig. Counts towards a full license, if bought within a year.

Full License $129 USD
Gives you activation of Hedge on all your computers (you can only run Hedge on just one computer at a time)

Site License $999 USD
Use Hedge on every computer in your company – with a single license key.

You can get Hedge today, by updating in-app, or by downloading a fresh copy here: hedge.video/download/hedge 

 

Subscribe to our newsletter