Backup Software

At one time I did backups manually by periodically copying directories 
from my computer to a backup hard drive.  The last year or two I've been 
using the Memeo Instant backup that came with an external Seagate drive. 
  I'm finding out it is not what I thought it was.

First, it doesn't backup the entire drive.  It only backs up things that 
*it* thinks you want to backup.  There is no way to tell it what to 
backup, or so I thought until I found I could diddle the hidden bit on 
the user directory to get that to backup.  But this means *everything* 
under user is backed up which is *too* much.

They just offered me a "special" on upgrading for only $40 I think 
compared to the usual $60.  I think this version gets around the issue 
of controlling what is backed up.

I'm wondering what better backup software is out there for Windows and 
possibly Linux too.  I've got a Raspberry Pi which I may start using as 
a server and will need to be backed up.

-- 

Rick

rickman wrote:

> At one time I did backups manually by periodically copying directories 
> from my computer to a backup hard drive.  The last year or two I've been 
> using the Memeo Instant backup that came with an external Seagate drive. 
>   I'm finding out it is not what I thought it was.
>
> First, it doesn't backup the entire drive.  It only backs up things that 
> *it* thinks you want to backup.  There is no way to tell it what to 
> backup, or so I thought until I found I could diddle the hidden bit on 
> the user directory to get that to backup.  But this means *everything* 
> under user is backed up which is *too* much.
>
> They just offered me a "special" on upgrading for only $40 I think 
> compared to the usual $60.  I think this version gets around the issue 
> of controlling what is backed up.
>
> I'm wondering what better backup software is out there for Windows and 
> possibly Linux too.  I've got a Raspberry Pi which I may start using as 
> a server and will need to be backed up.
>

Under Linux I use duplicity.  I wrote a small wrapper script around it,
and have cron run that script nightly.  It was pretty straightforward,
but I can't speak to how it works on Windows.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com

Email address domain is currently out of order.  See above to fix.

rickman <gnuarm@gmail.com> writes:
> I'm wondering what better backup software is out there for Windows and
> possibly Linux too.  I've got a Raspberry Pi which I may start using
> as a server and will need to be backed up.

No idea about Windows.  For Linux there's all kinds of programs but in a
simple setup like that, it's probably easiest to just use rsync, to
another machine or to a USB memory stick that you leave plugged into the
Pi.  Also since the RPi uses an SD card for storage, if your laptop has
an SD slot and you get a USB SD card reader, you can copy the card to
another one to have an image copy.  Finally if you develop on the RPi,
you can set up most source control these days to push your commits to a
remote machine.

In article <n4canb$s6v$1@dont-email.me>, gnuarm@gmail.com says...
> 
> At one time I did backups manually by periodically copying directories 
> from my computer to a backup hard drive.  The last year or two I've been 
> using the Memeo Instant backup that came with an external Seagate drive. 
>   I'm finding out it is not what I thought it was.
> 
> First, it doesn't backup the entire drive.  It only backs up things that 
> *it* thinks you want to backup.  There is no way to tell it what to 
> backup, or so I thought until I found I could diddle the hidden bit on 
> the user directory to get that to backup.  But this means *everything* 
> under user is backed up which is *too* much.

So many of these types of backup software are only useful for recovering
documents inside Users tree and so many I have seen keep backup 
catalogues on the SOURCE harddrive, let alone allow incremental backups
without ensuring the base full backup exists.

So if hard drive buys the ranch they are fairly ineffective for recovery

> They just offered me a "special" on upgrading for only $40 I think 
> compared to the usual $60.  I think this version gets around the issue 
> of controlling what is backed up.
> 
> I'm wondering what better backup software is out there for Windows and 
> possibly Linux too.  I've got a Raspberry Pi which I may start using as 
> a server and will need to be backed up.

Firstly what levels of backup do you need?
What frequency of backup (or put another way how much data can you 
afford to lose)?

What amount of time from bare drive to up and running do you want?

In a few places I deal with with Windows servers (due to other software
requirements), strategies include 

1/ Make DVD of Windows Recovery Environment, this allows Windows backup 
   images, system restore and a few other tools like chkdsk.

2/ If no OS DVD with system take some form of image backup of the 
   'recovery' (nuke to Factory configuration) partition.

3/ Run suitable Windows Task Scheduler scripts to run Windows backup
   Either as image, selected or excluded directories/files

On two sites due to they have fairly constant throughput on data that
cannot be reproduced easily (including x-rays of some patients daily),
they have agreed that a loss of one days data can be mostly recovered 
from other records as well. There are two servers so items can be sorted
to switch over within an hour.

Various scripts run overnight when they are closed

1/  Servers run MIRROR RAID (level 1) as minimum

2/  File ROBOCOPY of two specialist databases (About 30GB) to backup 
    partition, servers and external hard drive

3/  Image of whole boot and main partition to backup partition. Using 
    Windows Backup.

    Later copied to backup server and external hard drive. 

4/  Robocopy of special software partition containing sources of printer
    drivers, software, scripts, system documentation

5/  Scripts monitor errors and emails a log file with PASS or FAIL and
    ejects USB hard drive on final completion.

    PASS goes to users to swap external drive daily

    FAIL goes to users and me so I know when someone forgot to change 
    drives, or major error occured.

    Swapped out drive taken OFF-site (they have 7 drives on each site)

In 3 years a few hard drives failures, however we were able to be 
recovered from and resynced without loss.

You probably don't need anything so severe but I would recommend the 3 
steps I mentioned earlier with new image after new software install or 
windows updates. Going from old image and waiting for all updates can 
take a LONG time.

I personally don't like incremental backups, if you have too much data
then you should be looking st rsync utilities to other physical 
machine(s).

A lot of incremental backups get screwed because the original full 
backup is 1 year or older and not all increment steps are usable or 
known where to exist. Most incremental recovery goes funny when you 
even the slightest problem with one of the increment steps. Hence
daily images of which the databases are the largest part.

Robocopy allows only changes on files that are newer, and has 
multiple threading support with ability to MIRROR trees between two 
drives, so no extra files exist. Robocopy can copy all the file metadata
as well (security etc).

-- 
Paul Carpenter          | paul@pcserviceselectronics.co.uk
<http://www.pcserviceselectronics.co.uk/>    PC Services
<http://www.pcserviceselectronics.co.uk/pi/>  Raspberry Pi Add-ons
<http://www.pcserviceselectronics.co.uk/fonts/> Timing Diagram Font
<http://www.badweb.org.uk/> For those web sites you hate

rickman wrote:
> At one time I did backups manually by periodically copying directories
> from my computer to a backup hard drive.  The last year or two I've been
> using the Memeo Instant backup that came with an external Seagate drive.
>   I'm finding out it is not what I thought it was.
>
> First, it doesn't backup the entire drive.  It only backs up things that
> *it* thinks you want to backup.  There is no way to tell it what to
> backup, or so I thought until I found I could diddle the hidden bit on
> the user directory to get that to backup.  But this means *everything*
> under user is backed up which is *too* much.
>
> They just offered me a "special" on upgrading for only $40 I think
> compared to the usual $60.  I think this version gets around the issue
> of controlling what is backed up.
>
> I'm wondering what better backup software is out there for Windows and
> possibly Linux too.  I've got a Raspberry Pi which I may start using as
> a server and will need to be backed up.
>


I wrote my own in Tcl. It compares the file size and the file mod time
to make a copy/no copy decision.

I've extended it to create a directory named by date
stamp ( could also be time stamp) to store the version on
the backup media that is being covered up.

-- 
Les Cargill

On 12/10/2015 5:20 PM, Paul wrote:
> In article <n4canb$s6v$1@dont-email.me>, gnuarm@gmail.com says...
>>
>> At one time I did backups manually by periodically copying directories
>> from my computer to a backup hard drive.  The last year or two I've been
>> using the Memeo Instant backup that came with an external Seagate drive.
>>    I'm finding out it is not what I thought it was.
>>
>> First, it doesn't backup the entire drive.  It only backs up things that
>> *it* thinks you want to backup.  There is no way to tell it what to
>> backup, or so I thought until I found I could diddle the hidden bit on
>> the user directory to get that to backup.  But this means *everything*
>> under user is backed up which is *too* much.
>
> So many of these types of backup software are only useful for recovering
> documents inside Users tree and so many I have seen keep backup
> catalogues on the SOURCE harddrive, let alone allow incremental backups
> without ensuring the base full backup exists.
>
> So if hard drive buys the ranch they are fairly ineffective for recovery

Recovery shouldn't be a problem as the backup is just the same directory 
structure and files.  I can go into it with Windows Explorer and copy 
any file I wish.


>> They just offered me a "special" on upgrading for only $40 I think
>> compared to the usual $60.  I think this version gets around the issue
>> of controlling what is backed up.
>>
>> I'm wondering what better backup software is out there for Windows and
>> possibly Linux too.  I've got a Raspberry Pi which I may start using as
>> a server and will need to be backed up.
>
> Firstly what levels of backup do you need?
> What frequency of backup (or put another way how much data can you
> afford to lose)?

This backup is just of files, not the hard drive as an entity.  It runs 
anytime the backup drive is plugged in keeping it constantly up to date.


> What amount of time from bare drive to up and running do you want?
>
> In a few places I deal with with Windows servers (due to other software
> requirements), strategies include
>
> 1/ Make DVD of Windows Recovery Environment, this allows Windows backup
>     images, system restore and a few other tools like chkdsk.
>
> 2/ If no OS DVD with system take some form of image backup of the
>     'recovery' (nuke to Factory configuration) partition.
>
> 3/ Run suitable Windows Task Scheduler scripts to run Windows backup
>     Either as image, selected or excluded directories/files
>
> On two sites due to they have fairly constant throughput on data that
> cannot be reproduced easily (including x-rays of some patients daily),
> they have agreed that a loss of one days data can be mostly recovered
> from other records as well. There are two servers so items can be sorted
> to switch over within an hour.
>
> Various scripts run overnight when they are closed
>
> 1/  Servers run MIRROR RAID (level 1) as minimum
>
> 2/  File ROBOCOPY of two specialist databases (About 30GB) to backup
>      partition, servers and external hard drive
>
> 3/  Image of whole boot and main partition to backup partition. Using
>      Windows Backup.
>
>      Later copied to backup server and external hard drive.
>
> 4/  Robocopy of special software partition containing sources of printer
>      drivers, software, scripts, system documentation
>
> 5/  Scripts monitor errors and emails a log file with PASS or FAIL and
>      ejects USB hard drive on final completion.
>
>      PASS goes to users to swap external drive daily
>
>      FAIL goes to users and me so I know when someone forgot to change
>      drives, or major error occured.
>
>      Swapped out drive taken OFF-site (they have 7 drives on each site)
>
> In 3 years a few hard drives failures, however we were able to be
> recovered from and resynced without loss.
>
> You probably don't need anything so severe but I would recommend the 3
> steps I mentioned earlier with new image after new software install or
> windows updates. Going from old image and waiting for all updates can
> take a LONG time.
>
> I personally don't like incremental backups, if you have too much data
> then you should be looking st rsync utilities to other physical
> machine(s).
>
> A lot of incremental backups get screwed because the original full
> backup is 1 year or older and not all increment steps are usable or
> known where to exist. Most incremental recovery goes funny when you
> even the slightest problem with one of the increment steps. Hence
> daily images of which the databases are the largest part.
>
> Robocopy allows only changes on files that are newer, and has
> multiple threading support with ability to MIRROR trees between two
> drives, so no extra files exist. Robocopy can copy all the file metadata
> as well (security etc).
>


-- 

Rick

On 10/12/15 16:56, rickman wrote:
> I'm wondering what better backup software is out there for Windows and possibly
> Linux too.  I've got a Raspberry Pi which I may start using as a server and will
> need to be backed up.


For linux, look at unison and unison-gtk. It is customisable,
shows you what it will do when you say "go", and makes
pessimistic/safe assumptions about things going wrong.

In article <n4d7be$i27$2@dont-email.me>, gnuarm@gmail.com says...
> 
> On 12/10/2015 5:20 PM, Paul wrote:
> > In article <n4canb$s6v$1@dont-email.me>, gnuarm@gmail.com says...
> >>
> >> At one time I did backups manually by periodically copying directories
> >> from my computer to a backup hard drive.  The last year or two I've been
> >> using the Memeo Instant backup that came with an external Seagate drive.
> >>    I'm finding out it is not what I thought it was.
> >>
> >> First, it doesn't backup the entire drive.  It only backs up things that
> >> *it* thinks you want to backup.  There is no way to tell it what to
> >> backup, or so I thought until I found I could diddle the hidden bit on
> >> the user directory to get that to backup.  But this means *everything*
> >> under user is backed up which is *too* much.
> >
> > So many of these types of backup software are only useful for recovering
> > documents inside Users tree and so many I have seen keep backup
> > catalogues on the SOURCE harddrive, let alone allow incremental backups
> > without ensuring the base full backup exists.
> >
> > So if hard drive buys the ranch they are fairly ineffective for recovery
> 
> Recovery shouldn't be a problem as the backup is just the same directory 
> structure and files.  I can go into it with Windows Explorer and copy 
> any file I wish.

Then scrip[ts with ROBOCOPY is your friend

-- 
Paul Carpenter          | paul@pcserviceselectronics.co.uk
<http://www.pcserviceselectronics.co.uk/>    PC Services
<http://www.pcserviceselectronics.co.uk/pi/>  Raspberry Pi Add-ons
<http://www.pcserviceselectronics.co.uk/fonts/> Timing Diagram Font
<http://www.badweb.org.uk/> For those web sites you hate

On Thu, 10 Dec 2015 17:55:34 +0000, Rob Gaddi wrote:

> rickman wrote:
> 
>> At one time I did backups manually by periodically copying directories 
>> from my computer to a backup hard drive.  The last year or two I've been 
>> using the Memeo Instant backup that came with an external Seagate drive. 
>>   I'm finding out it is not what I thought it was.
>>
>> First, it doesn't backup the entire drive.  It only backs up things that 
>> *it* thinks you want to backup.  There is no way to tell it what to 
>> backup, or so I thought until I found I could diddle the hidden bit on 
>> the user directory to get that to backup.  But this means *everything* 
>> under user is backed up which is *too* much.
>>
>> They just offered me a "special" on upgrading for only $40 I think 
>> compared to the usual $60.  I think this version gets around the issue 
>> of controlling what is backed up.
>>
>> I'm wondering what better backup software is out there for Windows and 
>> possibly Linux too.  I've got a Raspberry Pi which I may start using as 
>> a server and will need to be backed up.
>>
> 
> Under Linux I use duplicity.  I wrote a small wrapper script around it,
> and have cron run that script nightly.  It was pretty straightforward,
> but I can't speak to how it works on Windows.

Rick- I use duplicity via cron job as well.  You might be interested to know
 * it can encrypt backups, so you can securely backup to the 'cloud'; and
 * there are now scripts that hide (simplify) some of duplicity's complexity
      {I haven't used these - started using duplicity before these were
       available, no reason not to continue...}

On 12/10/2015 11:56 AM, rickman wrote:
> At one time I did backups manually by periodically copying directories
> from my computer to a backup hard drive.  The last year or two I've been
> using the Memeo Instant backup that came with an external Seagate drive.
>   I'm finding out it is not what I thought it was.
>
> First, it doesn't backup the entire drive.  It only backs up things that
> *it* thinks you want to backup.  There is no way to tell it what to
> backup, or so I thought until I found I could diddle the hidden bit on
> the user directory to get that to backup.  But this means *everything*
> under user is backed up which is *too* much.
>
> They just offered me a "special" on upgrading for only $40 I think
> compared to the usual $60.  I think this version gets around the issue
> of controlling what is backed up.
>
> I'm wondering what better backup software is out there for Windows and
> possibly Linux too.  I've got a Raspberry Pi which I may start using as
> a server and will need to be backed up.
>

I keep stuff in a relatively watertight directory structure, and once a 
week I make backup DVDs of the stuff I wouldn't want to lose.  (Takes 
three to five DVDs, depending on what I've touched that week.  About a 
buck's worth, plus half an hour.

That makes me relatively secure from ransomware and so forth, especially 
since all the important stuff is under version control.

Unfortunately, hard drives and anything USB are increasingly insecure.

Cheers

Phil Hobbs



-- 
Dr Philip C D Hobbs
Principal Consultant
ElectroOptical Innovations LLC
Optics, Electro-optics, Photonics, Analog Electronics

160 North State Road #203
Briarcliff Manor NY 10510

hobbs at electrooptical dot net
http://electrooptical.net