Feasible to implement a router on a system on a chip?

Hi,

Is it feasible, or even possible to implement a complete router on a
system on a chip (i.e. on one FPGA)? The FPGA would be one of the
higher-end FPGAs that exist.

The router should be able to handle BGP and OSPF (i.e. traditional
router functionality).

For example, which of the following routers would be feasible/possible
to implment on a system on a chip (the router should be able to
forward the packets at full speed according to the interface
speed) ? :

-A router with four 1 Gbps Ethernet interfaces?

-A router with ten 1 Gbps Ethernet interfaces?

-A router with four 10 Gbps Ethernet interfaces?

-A router with ten 10 Gbps Ethernet interfaces?

Or does it sound completely undoable?

"dspfun" <dspfun@hotmail.com> wrote in message 
news:2d805671-9b4e-46f2-a3da-06d6d351d670@i12g2000prf.googlegroups.com...
> Hi,
>
> Is it feasible, or even possible to implement a complete router on a
> system on a chip (i.e. on one FPGA)? The FPGA would be one of the
> higher-end FPGAs that exist.
>
> The router should be able to handle BGP and OSPF (i.e. traditional
> router functionality).
>
> For example, which of the following routers would be feasible/possible
> to implment on a system on a chip (the router should be able to
> forward the packets at full speed according to the interface
> speed) ? :
>
> -A router with four 1 Gbps Ethernet interfaces?
>
> -A router with ten 1 Gbps Ethernet interfaces?
>
> -A router with four 10 Gbps Ethernet interfaces?
>
> -A router with ten 10 Gbps Ethernet interfaces?
>
> Or does it sound completely undoable?

Google for single chip router turns some interesting results.

Scott

On Mar 27, 8:19 am, dspfun <dsp...@hotmail.com> wrote:

> Is it feasible, or even possible to implement a complete router on a
> system on a chip (i.e. on one FPGA)? The FPGA would be one of the
> higher-end FPGAs that exist.

The logic, probably yes.

But the data buffering and even firmware are likely to require
external memories implemented in technologies more suited to large
scale memory than that used for the FPGA fabric - ie, DRAM of some
sort, FLASH of some sort.

You may also need external PHY's.

Take a look at typical consumer router designs - you have a processor
incorporating the MACs (and often wireless as well), then add external
SDRAM, FLASH, and a multi-port PHY or swith IC.  Only the processor
ASIC is a really good candidtate for FPGA functional replacement.

dspfun wrote:

> Is it feasible, or even possible to implement a complete router on a
> system on a chip (i.e. on one FPGA)?

While I'm new to FPGA's, I'm not new to routers.  I don't think this is 
possible.  There's a reason Cisco routers are large and expensive, and 
it's not because there's a single chip inside. :)  (of course it's 
because their NAME is on it!!)  Cisco, or one of the other companies, 
would have distilled it down to one chip, if they could.  I don't 
disagree that FPGA's and/or ASICs are used in the designs, they most 
certainly are -- I just think you need more supporting components 
(processor, big memories, interface chips, etc)

> The router should be able to handle BGP and OSPF (i.e. traditional
> router functionality).

Depends on the class of router.  SOHO routers most certainly don't 
support BGP or OSPF --- normally those routing protocols are supported 
in mid-to-high end routers.  BGP tables can get very big, with full 
internet tables being 200k+ routes.  Calling BGP and OSPF traditional 
features might be assuming a little much.

> For example, which of the following routers would be feasible/possible
> to implment on a system on a chip (the router should be able to
> forward the packets at full speed according to the interface
> speed) ? :

Most routers have limited processing power --- even large routers.  They 
normally get rated in terms of packets per second.  The thing most 
vendors play around with is this: how big are the packets that we're 
talking about? 64 bytes? 128 bytes? full 1500-ish byte packets?  This 
gets complicated when you start talking about access-lists, firewalls, 
NAT, etc because depending on what deep-packet inspection/modification 
is going on can REALLY slow things down.

Many router vendors lie like dogs when it comes to performance.  I'd be 
surprised if there are routers out there that can truly handle multiple 
single gig ports with any type of reasonable throughput.  Outside of 
Internet 2, there aren't many needs for such products.

Most gigabit routing solutions end up being LOCAL routing, between 
multiple local LANs, and then Layer-3 switches end up picking up the 
task.  But then, you lose many capabilities in terms of filtering, etc 
--- which of course would limit the speed again.

> -A router with four 1 Gbps Ethernet interfaces?

No.

> -A router with ten 1 Gbps Ethernet interfaces?

No.

> -A router with four 10 Gbps Ethernet interfaces?

Definitely no.

> -A router with ten 10 Gbps Ethernet interfaces?

Are you crazy??

> Or does it sound completely undoable?

Yup. Undeniably undoable.

A couple 10mbps ethernet ports in a SOHO application? Maybe.

Keith

On Mar 27, 6:37 pm, Keith M <keit...@yanktheobvious.verizon.net>
wrote:
> dspfun wrote:
> > Is it feasible, or even possible to implement a complete router on a
> > system on a chip (i.e. on one FPGA)?
>
...
>
> Keith


I discovered that the MIPS architecture is widely licensed and used
frequently in common embedded applications.  For example, the DI-525
router, hardware version C, uses a System on a Chip with the MIPS
instruction set, FLASH, and RAM.  Lots of flash and ram, like 16Mb RAM
and 4Mb Flash (various posts cite different numbers).

Buy one on eBay (make sure it is Rev C) and open it up.  There are
only 4 or 5 chips.  One is a switch, one is the SoC.  I speculate that
a switch could easily be brought into FPGA.

aubrey wrote:

> I discovered that the MIPS architecture is widely licensed and used
> frequently in common embedded applications.  For example, the DI-525
> router, hardware version C, uses a System on a Chip with the MIPS
> instruction set, FLASH, and RAM.  Lots of flash and ram, like 16Mb RAM
> and 4Mb Flash (various posts cite different numbers).

I've often thought about buying some of these devices just to have a 
working platform.  If you buy all that stuff separately, it would cost 
you a fortune.  Those people hacking those devices are pretty crazy in 
their reverse engineering skills.

Yeah 16mb of ram and 4mb of flash is a lot of memory.  However, if you 
compare it to today's serious routers(anything even remotely close to 
the OP's idea), it's not nearly enough.  I just recently fitted a Cisco 
with 64MB flash and 256MB DRAM.  It doesn't have the most complete 
feature set either...... and if you look at Cisco's memory roadmap, it 
will probably need upgraded by 2009.  And this was in a router designed 
to handle T3 speeds. 45mbps.

Keith

Keith M wrote:
> aubrey wrote:
> 
>> I discovered that the MIPS architecture is widely licensed and used
>> frequently in common embedded applications.  For example, the DI-525
>> router, hardware version C, uses a System on a Chip with the MIPS
>> instruction set, FLASH, and RAM.  Lots of flash and ram, like 16Mb RAM
>> and 4Mb Flash (various posts cite different numbers).
> 
> I've often thought about buying some of these devices just to have a 
> working platform.  If you buy all that stuff separately, it would cost 
> you a fortune.  Those people hacking those devices are pretty crazy in 
> their reverse engineering skills.
> 
> Yeah 16mb of ram and 4mb of flash is a lot of memory.  However, if you 
> compare it to today's serious routers(anything even remotely close to 
> the OP's idea), it's not nearly enough.  I just recently fitted a Cisco 
> with 64MB flash and 256MB DRAM.  It doesn't have the most complete 
> feature set either...... and if you look at Cisco's memory roadmap, it 
> will probably need upgraded by 2009.  And this was in a router designed 
> to handle T3 speeds. 45mbps.
> 

I can see how you might have use for more RAM - storing bigger routing 
tables, ARP caches, connection trackings, and so on, as well as doing 
some packet buffering.  But I have difficulty seeing how you would fill 
256 MB with this sort of thing unless you are switching a lot of 10 GB 
lines with serious congestions - your aim in the router is to pass 
packets through without storing them, unless it is absolutely necessary 
(such as because of differences in line speeds).  I certainly can't 
imagine what you'd want with 64 MB flash - even if you run a 
non-specialised kernel such as Linux on your router, the kernel, all the 
networking, routing and filtering code, and the basic configuration 
tools will fit in about 2 MB.  Add another 2 MB for a fancy web 
interface if you want.

On Mar 28, 9:19 am, David Brown <da...@westcontrol.removethisbit.com>
wrote:
> Keith M wrote:
> > aubrey wrote:
>
> >> I discovered that the MIPS architecture is widely licensed and used
> >> frequently in common embedded applications.  For example, the DI-525
> >> router, hardware version C, uses a System on a Chip with the MIPS
> >> instruction set, FLASH, and RAM.  Lots of flash and ram, like 16Mb RAM
> >> and 4Mb Flash (various posts cite different numbers).
>
> > I've often thought about buying some of these devices just to have a
> > working platform.  If you buy all that stuff separately, it would cost
> > you a fortune.  Those people hacking those devices are pretty crazy in
> > their reverse engineering skills.
>
> > Yeah 16mb of ram and 4mb of flash is a lot of memory.  However, if you
> > compare it to today's serious routers(anything even remotely close to
> > the OP's idea), it's not nearly enough.  I just recently fitted a Cisco
> > with 64MB flash and 256MB DRAM.  It doesn't have the most complete
> > feature set either...... and if you look at Cisco's memory roadmap, it
> > will probably need upgraded by 2009.  And this was in a router designed
> > to handle T3 speeds. 45mbps.
>
> I can see how you might have use for more RAM - storing bigger routing
> tables, ARP caches, connection trackings, and so on, as well as doing
> some packet buffering.  But I have difficulty seeing how you would fill
> 256 MB with this sort of thing unless you are switching a lot of 10 GB
> lines with serious congestions - your aim in the router is to pass
> packets through without storing them, unless it is absolutely necessary
> (such as because of differences in line speeds).  I certainly can't
> imagine what you'd want with 64 MB flash - even if you run a
> non-specialised kernel such as Linux on your router, the kernel, all the
> networking, routing and filtering code, and the basic configuration
> tools will fit in about 2 MB.  Add another 2 MB for a fancy web
> interface if you want.

Thank you all for your input!

The design I had in mind would be a system on a chip plus high-end
external memories. It would look something like this:

In the SOC (i.e. the FPGA):
Two PPC440 CPUs handling BGP and OSPF (i.e. control plane). The fast
path/data plane is all handled in the FPGA-code, i.e. packet
forwarding/routing.

The external memories would consist of something like 64 MB Flash, 128
MB QDRII+ SRAM and 1 GB of DDR3 SDRAM (maybe need to adjust some of
the sizes).

A 1 Gbps interface means forwarding 64-byte packets at a rate of 2
MPPS. Four 1 Gbps interface -> 4x2 MPPS = 8 MPPS packet forwarding.

A 10 Gbps interface means forwarding 64-byte packets at a rate of 20
MPPS. Four 10 Gbps interface -> 4x20 MPPS = 80 MPPS packet forwarding.

So, how many MPPS (64-byte packets) can a FPGA forward?

As far as I have understood a high-end FPGA with fast external
memories should be able to forward packets at a rate of 20 MPPS.

Having four 10 Gbps interfaces means that the FPGA would have to
forward packets at a rate of 4x20 MPPS= 80 MPPS, is that not possible?

What would you estimate the highest possible packet forwarding rate
(MPPS) to for a single high-end FPGA?

David Brown wrote:

> I certainly can't 
> imagine what you'd want with 64 MB flash - even if you run a 
> non-specialised kernel such as Linux on your router, the kernel, all the 
> networking, routing and filtering code, and the basic configuration 
> tools will fit in about 2 MB.  Add another 2 MB for a fancy web 
> interface if you want.

Cisco's executable images are up to 20mb a pop, and that firmware is 
stored in the flash.  I forget whether 20mb is compressed or not -- it 
might be.  Cisco is the big dog with lotsa features, so they might be an 
extreme case.

I tried like heck this morning to find the memory timeline/roadmap on 
Cisco's site, but no can do.  However the Cisco 3845 datasheet

http://www.cisco.com/en/US/prod/collateral/routers/ps5855/product_data_sheet0900aecd8016a8e8.html

and see under memory

� Default-64 MB Compact Flash; 256 MB DDR SDRAM

I honestly don't know what the breakdown of why that much is required 
--- but can tell you from practical experience that 64/256mb is 
considered standard for low-midrange (Note I'm specifically excluding 
SOHO routers) full-featured routers.  The Cisco 3845 maxes out around T3 
speeds.  The larger routers have larger requirements --- but note that 
memory doesn't scale with bandwidth........ you don't necessarily need 
more memory to support higher throughput.

However, I can tell you that having support for many different types of 
modules, many different WAN protocols, VPN support, firewall support, 
VOIP support, etc etc etc Executable image size and memory requirements 
can skyrocket.  Memory requirements are going to hinge exactly on what 
features you intend on supporting.

Keith

dspfun wrote:
> ...
> The design I had in mind would be a system on a chip plus high-end
> external memories. It would look something like this:
>
> In the SOC (i.e. the FPGA):
> Two PPC440 CPUs handling BGP and OSPF (i.e. control plane). The fast
> path/data plane is all handled in the FPGA-code, i.e. packet
> forwarding/routing.
> ...

You may want to investigate the products Freescale have on offer.
You sound like you are trying to reinvent some of their PPC based
QUICK and sort of parts.

Dimiter

------------------------------------------------------
Dimiter Popoff               Transgalactic Instruments

http://www.tgi-sci.com
------------------------------------------------------
http://www.flickr.com/photos/didi_tgi/sets/72157600228621276/

dspfun wrote:
> On Mar 28, 9:19 am, David Brown <da...@westcontrol.removethisbit.com>
> wrote:
> > Keith M wrote:
> > > aubrey wrote:
> >
> > >> I discovered that the MIPS architecture is widely licensed and used
> > >> frequently in common embedded applications.  For example, the DI-525
> > >> router, hardware version C, uses a System on a Chip with the MIPS
> > >> instruction set, FLASH, and RAM.  Lots of flash and ram, like 16Mb RAM
> > >> and 4Mb Flash (various posts cite different numbers).
> >
> > > I've often thought about buying some of these devices just to have a
> > > working platform.  If you buy all that stuff separately, it would cost
> > > you a fortune.  Those people hacking those devices are pretty crazy in
> > > their reverse engineering skills.
> >
> > > Yeah 16mb of ram and 4mb of flash is a lot of memory.  However, if you
> > > compare it to today's serious routers(anything even remotely close to
> > > the OP's idea), it's not nearly enough.  I just recently fitted a Cisco
> > > with 64MB flash and 256MB DRAM.  It doesn't have the most complete
> > > feature set either...... and if you look at Cisco's memory roadmap, it
> > > will probably need upgraded by 2009.  And this was in a router designed
> > > to handle T3 speeds. 45mbps.
> >
> > I can see how you might have use for more RAM - storing bigger routing
> > tables, ARP caches, connection trackings, and so on, as well as doing
> > some packet buffering.  But I have difficulty seeing how you would fill
> > 256 MB with this sort of thing unless you are switching a lot of 10 GB
> > lines with serious congestions - your aim in the router is to pass
> > packets through without storing them, unless it is absolutely necessary
> > (such as because of differences in line speeds).  I certainly can't
> > imagine what you'd want with 64 MB flash - even if you run a
> > non-specialised kernel such as Linux on your router, the kernel, all the
> > networking, routing and filtering code, and the basic configuration
> > tools will fit in about 2 MB.  Add another 2 MB for a fancy web
> > interface if you want.
>
> Thank you all for your input!
>
> The design I had in mind would be a system on a chip plus high-end
> external memories. It would look something like this:
>
> In the SOC (i.e. the FPGA):
> Two PPC440 CPUs handling BGP and OSPF (i.e. control plane). The fast
> path/data plane is all handled in the FPGA-code, i.e. packet
> forwarding/routing.
>
> The external memories would consist of something like 64 MB Flash, 128
> MB QDRII+ SRAM and 1 GB of DDR3 SDRAM (maybe need to adjust some of
> the sizes).
>
> A 1 Gbps interface means forwarding 64-byte packets at a rate of 2
> MPPS. Four 1 Gbps interface -> 4x2 MPPS = 8 MPPS packet forwarding.
>
> A 10 Gbps interface means forwarding 64-byte packets at a rate of 20
> MPPS. Four 10 Gbps interface -> 4x20 MPPS = 80 MPPS packet forwarding.
>
> So, how many MPPS (64-byte packets) can a FPGA forward?
>
> As far as I have understood a high-end FPGA with fast external
> memories should be able to forward packets at a rate of 20 MPPS.
>
> Having four 10 Gbps interfaces means that the FPGA would have to
> forward packets at a rate of 4x20 MPPS= 80 MPPS, is that not possible?
>
> What would you estimate the highest possible packet forwarding rate
> (MPPS) to for a single high-end FPGA?