Secondary store

Don Y <this@is.not.me.com> wrote:
> But they will suffer from the same endurance issue that any
> NON-volatile semiconductor memory exhibits.  AFAICT, there are no
> SD-card or USB "RAM" devices (and, a USB hard disk just adds
> another "box" to the mix).

For about 7 years my router was a Netgear WGT634U.  That was the usual
consumer specs: 200MHz 32-bit MIPS CPU, 16MB flash 32MB RAM.  Only it wasn't
running the Netgear firmware, it was running full Debian/mips from a USB
stick (1GB or 2GB, being reasonably big back then).

Debian in 32MB is fairly constrained, especially when running various
servers as I was, which meant it was hammering the USB stick all day every
day using it as swap.  I replaced the USB stick once or twice as a
precaution, but never had an issue with wearout.  The old sticks went into
my junk drawer, where they still are and still work.

So if you need a 'swap' device, use a USB flash stick.  Don't use SD, CF or
(worst) micro SD - there isn't enough space for buffer memory so performance
is worse than USB.  Just replace it every few years and you should be fine.

Theo

Hi Theo,

On 8/26/2014 5:46 PM, Theo Markettos wrote:
> Don Y<this@is.not.me.com>  wrote:
>> But they will suffer from the same endurance issue that any
>> NON-volatile semiconductor memory exhibits.  AFAICT, there are no
>> SD-card or USB "RAM" devices (and, a USB hard disk just adds
>> another "box" to the mix).
>
> For about 7 years my router was a Netgear WGT634U.  That was the usual
> consumer specs: 200MHz 32-bit MIPS CPU, 16MB flash 32MB RAM.  Only it wasn't
> running the Netgear firmware, it was running full Debian/mips from a USB
> stick (1GB or 2GB, being reasonably big back then).
>
> Debian in 32MB is fairly constrained, especially when running various
> servers as I was, which meant it was hammering the USB stick all day every
> day using it as swap.  I replaced the USB stick once or twice as a
> precaution, but never had an issue with wearout.  The old sticks went into
> my junk drawer, where they still are and still work.

Did you ever look to see what peak swap usage was?

How speedy is "thrashing" via USB?  I.e., will I lose all my MIPS gains
to a USB bottleneck?  (the newer box -- now running with a 2.5" drive
cobbled inside -- is ~112MB/800MHz)

> So if you need a 'swap' device, use a USB flash stick.  Don't use SD, CF or
> (worst) micro SD - there isn't enough space for buffer memory so performance
> is worse than USB.  Just replace it every few years and you should be fine.

I may try that!  There are provisions for two internal USB "sticks".
I had originally thought of using a USB wireless adapter and trying
a USB "R/O" image for the OS/executables -- leaving the disk in place
to gather metrics about how it uses swap.

But, I could skip the wireless adapter and add a second thumb drive
to act as "swap device" -- so the system is never at risk (if the
system image wasn't accessible, booting to recover/repair would
be problematic!)

And, if this works, replace the disk with a CF card (for the OS
and executables), a thumb drive for swap and then add the wireless
adapter back in...

On 27/08/14 02:46, Theo Markettos wrote:
> Don Y <this@is.not.me.com> wrote:
>> But they will suffer from the same endurance issue that any
>> NON-volatile semiconductor memory exhibits.  AFAICT, there are no
>> SD-card or USB "RAM" devices (and, a USB hard disk just adds
>> another "box" to the mix).
> 
> For about 7 years my router was a Netgear WGT634U.  That was the usual
> consumer specs: 200MHz 32-bit MIPS CPU, 16MB flash 32MB RAM.  Only it wasn't
> running the Netgear firmware, it was running full Debian/mips from a USB
> stick (1GB or 2GB, being reasonably big back then).
> 
> Debian in 32MB is fairly constrained, especially when running various
> servers as I was, which meant it was hammering the USB stick all day every
> day using it as swap.  I replaced the USB stick once or twice as a
> precaution, but never had an issue with wearout.  The old sticks went into
> my junk drawer, where they still are and still work.
> 
> So if you need a 'swap' device, use a USB flash stick.  Don't use SD, CF or
> (worst) micro SD - there isn't enough space for buffer memory so performance
> is worse than USB.  Just replace it every few years and you should be fine.
> 
> Theo
> 

That's an interesting first-hand account - I would not have liked to use
a USB flash stick or other small card for swap.  (Real SSD's, especially
reasonable quality devices, are fine for swap.)

It's an odd choice to run a full Debian on such a system - typically
distros like OpenWRT are more popular on such hardware.  Even 7 years
ago it had packages for many useful servers, and you would get the
server software itself in the 16 MB flash - leaving the USB for data
and, if necessary, swap.

Don Y <this@is.not.me.com> wrote:
> Hi Theo,
> 
> On 8/26/2014 5:46 PM, Theo Markettos wrote:
> Did you ever look to see what peak swap usage was?

I think I set it up with 32MB DRAM and 256MB swap.  I think typical usage
was ~128MB of swap was used, though of course lots of things are paged out
until they get used (eg servers idling until somebody connects to their
socket).  I didn't measure swap bandwidth/traffic.

> How speedy is "thrashing" via USB?  I.e., will I lose all my MIPS gains
> to a USB bottleneck?  (the newer box -- now running with a 2.5" drive
> cobbled inside -- is ~112MB/800MHz)

It's not quick, but I had a 200MHz router so I didn't expect it to be quick
anyway.  What helped a lot was to buy a flash stick with decent random write
performance: at the time such were marked Vista ReadyBoost compatible. 
These days they tend to emphasise bulk speeds, but if you can find one with
a CrystalDiskMark benchmark (Amazon listings are good for this) it helps.
For example, something like this, though the drive mentioned is a bit
expensive today:
http://www.altechnative.net/2012/02/07/morebetter-internal-storage-on-the-toshiba-ac100-part-2/
I don't know if today buying 'USB3' sticks is helpful in getting the faster
ones - I must do some proper tests at some point.

Theo

David Brown <david.brown@hesbynett.no> wrote:
> It's an odd choice to run a full Debian on such a system - typically
> distros like OpenWRT are more popular on such hardware.  Even 7 years
> ago it had packages for many useful servers, and you would get the
> server software itself in the 16 MB flash - leaving the USB for data
> and, if necessary, swap.

This was 2005.  OpenWRT wasn't as advanced at that point - it particular
there weren't as many packages, and the setup was substantially different. 
Plus 16MB flash isn't enough to fit much in - I ran a full Asterisk server
on it for instance (fine apart from not enough CPU to do transcoding).  I
really dislike busybox and I don't want to be interacting with it on a daily
basis (it's got better now).

The router (and its flash) was still functional when I retired it 2 years
ago: at short notice I needed a SSH-accessible Linux router for an
inaccessible hostile remote location, so I reflashed it with modern OpenWRT
and last I heard it was working fine.

Theo

On Wednesday, August 27, 2014 4:26:08 AM UTC-7, Theo Markettos wrote:
> David Brown <david.brown@hesbynett.no> wrote:
>  It's an odd choice to run a full Debian on such a system - typically distros like OpenWRT are more popular on such hardware.  Even 7 years ago it had packages for many useful servers, and you would get the server software itself in the 16 MB flash - leaving the USB for data and, if necessary, swap.
> 
> This was 2005.  OpenWRT wasn't as advanced at that point - it particular there weren't as many packages, and the setup was substantially different. Plus 16MB flash isn't enough to fit much in - I ran a full Asterisk server on it for instance (fine apart from not enough CPU to do transcoding).  I really dislike busybox and I don't want to be interacting with it on a daily basis (it's got better now).
> 
> The router (and its flash) was still functional when I retired it 2 years ago: at short notice I needed a SSH-accessible Linux router for an inaccessible hostile remote location, so I reflashed it with modern OpenWRT and last I heard it was working fine.

Why deal with old stuffs, when newer hardwares are dirt cheap.

My GSM/Wifi router runs on a 16G microSD via USB.  With 1G system memory on 1.6GHz dual core Atom, speed is not a problem. With at least 8G log space, it should outlast me as well.

Should I feel guilty about wasting resources?  But it draws not much more power than COTS router, perhaps 5W average.  And the netbook cost me less than $50, becasuse of a cracked LCD screen.

BTW, there is a 250G hard drive i am not using, in order to save power.  It's always powered down.

I also have another Window 7 laptop and Linux Mint laptop.  I got tired of dual booting, so just use two laptops at the same time.

Hi Theo,

On 8/27/2014 4:06 AM, Theo Markettos wrote:
> Don Y<this@is.not.me.com>  wrote:
>> On 8/26/2014 5:46 PM, Theo Markettos wrote:
>> Did you ever look to see what peak swap usage was?
>
> I think I set it up with 32MB DRAM and 256MB swap.  I think typical usage
> was ~128MB of swap was used, though of course lots of things are paged out
> until they get used (eg servers idling until somebody connects to their
> socket).  I didn't measure swap bandwidth/traffic.

Understood.  On my previous box, I knew it idled at about 40M of
swap used.  But, I also noticed (as I was "pilfering" components
from it on the way to the tip) that it had 256M of DRAM (the "new"
box has a bit less than 110M).  So, I will reexamine swap stats
on the new box to see how much hurt therein lies...

One of the "problems" with having a disk available is that it is
sorely tempting to migrate other services/utilities into the box
that could be handled elsewhere.  So, part of this exercise will be
disciplining myself to pull the services that don't need to be
24/7/365 *off* the box and onto something more appropriate.

>> How speedy is "thrashing" via USB?  I.e., will I lose all my MIPS gains
>> to a USB bottleneck?  (the newer box -- now running with a 2.5" drive
>> cobbled inside -- is ~112MB/800MHz)
>
> It's not quick, but I had a 200MHz router so I didn't expect it to be quick
> anyway.  What helped a lot was to buy a flash stick with decent random write
> performance: at the time such were marked Vista ReadyBoost compatible.
> These days they tend to emphasise bulk speeds, but if you can find one with
> a CrystalDiskMark benchmark (Amazon listings are good for this) it helps.
> For example, something like this, though the drive mentioned is a bit
> expensive today:
> http://www.altechnative.net/2012/02/07/morebetter-internal-storage-on-the-toshiba-ac100-part-2/
> I don't know if today buying 'USB3' sticks is helpful in getting the faster
> ones - I must do some proper tests at some point.

OK, I'll have to do some research.  I think moving the system/data
off the rotating medium is a good first step (see below).  Then, I
can experiment with different swap media (disk/USB/CF).

I ended up cramming a 2.5" into the box (by relocating a tiny
speaker that was otherwise interfering).

<http://www.mediafire.com/view/m7dn3odw2mw32n2/1.JPG>
<http://www.mediafire.com/view/8zssof6cx4mbut9/2.JPG>
<http://www.mediafire.com/view/gna5v2h1l43kp0z/3.JPG>

As you can see, it's about the same footprint as a Linksys
router (but far more "boxy")

Note thumb drive I've installed *inside* the case.  I'll play
with moving the system onto that drive -- leaving the disk for
swap.  Then, adding another thumb drive to replace the rotating
media *as* swap (I figure all the wear will be confined to that
"disposable" drive, then).

Meanwhile, I'll scrounge a 1.8" disk as a backup plan -- as
swap, only.

Hi Edward,

On 8/27/2014 9:08 AM, edward.ming.lee@gmail.com wrote:
> On Wednesday, August 27, 2014 4:26:08 AM UTC-7, Theo Markettos
> wrote:

> Why deal with old stuffs, when newer hardwares are dirt cheap.

Why deal with NEW stuff when *newer* stuff will be even CHEAPER?  :>

What's your suggestion?  Discard the old?  "Ah, it's Jan 1 -- time
for a new cell phone!"

One of the places I volunteered at accepted donated items that they
converted to cash (by refurbishing/reselling/recycling) and then
using the cash to fund "aid" operations.  We processed 3.5 tons
every week.  And, most people were either unaware of our existence
or unwilling to be bothered with hauling their kit *to* us!

Recycling is horribly inefficient -- *if* you actually have folks
actively participating and service agencies that can use all of
the recycled material!

Plastic is trash.  Folks won't even TRY to recycle it -- no "return"
there!

Sheet metal (e.g., computer cases) are a penny or two per pound
(depending on whether they are "clean" or still have bits of
plastic, etc. attached that must be processed/removed/discarded.

A nice copper (or aluminum) heatsink is golden!  "Motherboards"
are maybe a couple of dollars per pound.

I.e., take your brand new computer and recycle it -- and you will
probably get about $10 out of it!

Contrast that with what it costs to purchase -- even "manufacturer's
COST".  The difference represents the inefficiency in the recycling
process BEST CASE (as often raw materials are cheaper to *extract*
than *recover*!)

 From a technical viewpoint, chances are you'll have more documentation
for older kit than newer items -- especially as most manufacturers
don't publish anything technically useful (so, your kit needs to be
"available" long enough for someone to become interested in exploring
it in detail and publishing their own observations).

There is also personal satisfaction at having *done* something (no
matter how trivial) instead of just "buying a solution".  And, the
leveraging the knowledge gained to produce other solutions with
different constraints (e.g., when I have to migrate these services
to even fewer resources).

> My GSM/Wifi router runs on a 16G microSD via USB.  With 1G system
> memory on 1.6GHz dual core Atom, speed is not a problem. With at
> least 8G log space, it should outlast me as well.
>
> Should I feel guilty about wasting resources?  But it draws not much
> more power than COTS router, perhaps 5W average.  And the netbook
> cost me less than $50, becasuse of a cracked LCD screen.
>
> BTW, there is a 250G hard drive i am not using, in order to save
> power.  It's always powered down.
>
> I also have another Window 7 laptop and Linux Mint laptop.  I got
> tired of dual booting, so just use two laptops at the same time.

> > Why deal with old stuffs, when newer hardwares are dirt cheap.
> 
> Why deal with NEW stuff when *newer* stuff will be even CHEAPER?  :>

These 5 to 6 years old netbooks are useless otherwise, but they are better than your 10+ years stuff.  If i don't use them, they will be trashed anyway.  First, they are made unusable with Vista, but they are great for Linux router. The 10" screen and resolutions are too small for real work.

In the case of my Gateway, the external case was so weak that plenty of them got cracked LCD.  So, you can pick them up cheap for around $50.

> 
> What's your suggestion?  Discard the old?  "Ah, it's Jan 1 -- time for a new cell phone!"

The better of two evils: recycle newer trash first.

On 2014-08-27, edward.ming.lee@gmail.com <edward.ming.lee@gmail.com> wrote:
>
> These 5 to 6 years old netbooks are useless otherwise, but they are better
> than your 10+ years stuff.  If i don't use them, they will be trashed anyway. 
> First, they are made unusable with Vista, but they are great for Linux router.
> The 10" screen and resolutions are too small for real work.
>

Actually, for me, those netbooks are the ideal size and weight to be
able to drop into a backpack and pull out again on the train.

They are small enough to not take up a major part of a 15 litre backpack
and they are light enough not to be really noticable while walking
around a city (although I wouldn't carry one on a long distance
day walk. :-))

I run Scientific Linux 5.x on mine (an Acer Aspire 1).

Simon.

-- 
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world