In FTTH networks, fibre cables are run all the way down from the service provider and terminated directly into the consumer’s home. In HFC networks, although the backhaul comprises of fibre cables, they’re terminated further away from homes, and coaxial cables take over this last mile instead.

I first noticed the new cable installations nearly a week ago. Closer inspection of the markings revealed that they were indeed fibre cables, and the huge rolls of them lying around on the lobby suggested large scale deployment. Conversing with the technicians who were responsible for them confirmed that.

A few days later, these units begin to appear on the landing of every pair of home, which I can only surmise to be FTTH termination equipment.

Although these cables do not seem to be carrying any form of data at this point in time, they are perhaps the most important groundwork ever done for Singapore’s next generation broadband network. Last mile equipment is usually the hardest and most expensive problem to tackle, but it seems to me that we have solved that already.

I’ve ceased using commercial routers aimed targeted at the home segment for a number of years now. What made those routers unfeasible was the growing bandwidth available to home user and the popularity of peer-to-peer, meshed, file sharing – where a large number of connections would be made simultaneously to grab various parts of the same file from different users. The earlier home routers with their low memory and processing capability couldn’t handle the load, and enterprise routers were, and still are, out of the reach of my financial capability.

That was when I turned to building my own. I started refurbishing an old computer with additional network cards. The system with I replaced today ran on an Intel Pentium III 450 MHz processor with 256 MB of ram, and had been for a number of years.

At the simplest level, any computer with more than one Ethernet adapter can be used as a router. On, Linux, routing can be done through iptables, on Windows, either through Internet Connection Sharing or Routing and Remote Access on consumer and server variants of the operation system respectively. However, there are Linux-based distributions which are designed with the sole purpose of turning a PC into a dedicated router which can rival commercial offerings in terms of performance and features. One such example and the one that I’m using is m0n0wall.

Lately, my old setup has been giving me issues, which I attributed to a failing power supply. Replacing it wasn’t the best of idea since the system was old, really old. This was the opportunity to get rid of and replace the system. A new one wouldn’t cost very much, and the much lower power draw of the Atom was welcomed too. It was the perfect system for the task.

I did some shopping, and although some manufacturers, such as Gigabyte, do offer Atom boards with dual GbE adapters, none of those models made it to the local market. In fact, they’re relatively few Atom models here in Singapore. I had to settle for the Asus AT3GC-I, which sports a dual core Intel Atom 330 processor with a single GbE port, resulting in me having to purchase an additional network card, filling up the only PCI expansion slot available. The 2 GB of RAM that I bought was a huge overkill, but the stores were only carrying 1 GB and 2 GB DDR2 memory, and the cost between the two was a mere $2. Add in a mini-ITX case, and I was almost ready to go.

Since m0n0wall takes up only a mere 10 MB of space, I decided to skip the hard disk. Instead, I opted for a 4GB USB thumb drive as the primary storage medium. It was the lowest capacity one I could find.

Assembly was a breeze, and was the easiest one I’ve done to date. The motherboard fit right into the case, and since the only peripheral I have is one expansion card, it was all very straightforward. No issues with lengthy graphics card that wouldn’t fit into the casing, nor a billion front panel chassis connectors to deal with.

For m0n0wall installation, I downloaded physdiskwrite 0.5.2 and the generic-pc-1.3.img from http://m0n0.ch/wall/downloads.php. Opening the command prompt on my Windows 7 PC with Administrator rights, I inserted my newly bought thumb drive in, formatted it, and began writing the m0n0wall image to it by executing the following.

physdiskwrite –u generic-pc-1.3.img

I was displayed a list of the drives available and selected my thumb drive as the destination. One-sixth of a minute later or under, it was done. I plugged the thumb drive into my Atom system, did a few changes in the BIOS to set it as the default boot medium, and booted. The picture below shows the initial boot.

I spent the next half hour or so copying over my configuration from my previous set-up, and the hour after that rearranging some furniture and re-doing cabling. The beauty of m0n0wall is that it can be used right out of the box if you have no need for more advanced features such as traffic shaping. There really is little configuration.

I still maintain that the system is an overkill. Under load, memory and CPU usage hardly crosses 5%. However, a lesser powered system, such as one of Soekris Engineering’s offerings, would have cost just as much or even more. I’m extremely happy with my new setup. Before I end the article, here’s another pictures, from the rear, where you can clearly see the 4GB thumb drive which would be permanently plugged in as the boot medium.

Back on my desktop, I was greeted with a previously unseen error as I navigated to the site.

I verified that Apache was running with a ps aux | grep apache, and indeed it was. I checked that I was able to access the page from machine on the same subnet. At this point, I suspected it was an issue with the Vyatta router, although its rather puzzling since all traffic initiated from the 192.168.5.0 network, internet bound or otherwise seemed to be working fine.

Carrying out tcpdump on the Ubuntu server, it seems that the first SYN packet from 192.168.1.50 was received, and judging from the subsequent similar (they all have same sequence numbers) SYN-ACK packets that were sent out from the server, it was clear that server never heard from 192.168.1.50 any further. One of the following could’ve happened:

1. The SYN-ACK bound for 192.168.1.50 never arrived at its destination.
2. The SYN-ACK packets arrived at 192.168.1.50, but the ACK respond required to complete the handshake never made it back to the server.

Carrying out a tcpdump dump similarly on eth0 of Vyatta, I was able to verify that the packets were following both ways, which took the fault of the router of the question. All was well with it.

I don’t know why I didn’t look at this earlier, but when you install a sparkling new piece of equipment and it something doesn’t work, you would automatically lay blame to that equipment instead of examining the complete picture closely, exactly like the old rhetoric “it used to work until I <insert something that you recently just did here>”. I ran Wireshark, which is a GUI version of a packet capturing tool much like tcpdump on my desktop itself. What showed next was the reason behind all my problems.

Wireshark showed that my IPCop router was sending me an ICMP redirect for every packet I sent out destined for 192.168.5.3. It was then that it occurred to me that since the Vyatta router had an interface in the same network as my computer, IPCop was telling me that that shorter route to the 192.168.5.0 network was to send it directly to the gateway at 192.168.1.150 instead of sending it to the default gateway specified on the desktop of 192.168.1.1 aka IPCop and then have it rerouted.

It turns out that on Windows if the Windows Firewall is turned on, ICMP redirects are dropped by default. As a routing mechanism, ICMP redirects have no security built in and hence, are vulnerable to IP spoofing attacks. So although an ICMP redirect was sent to my computer, it was silently dropped and the message never got through.

To get around this problem, the firewall could simply be enabled to accept ICMP redirects. Despite the security issues with it, it is rather safe on a small LAN where every computer comes under my control and ICMP requests are not forwarded by the exterior facing router into the network. Alternatively, an entry could be added in the Windows routing table.

route -p add 192.168.5.0 mask 255.255.255.0 192.168.1.150

It turns out to be a simple and small issue, but it took the better part of an afternoon to solve. Its all about the learning experience I guess, that’s what the network was setup for in the first place.

I’ve a few objectives I wish to attain with this setup. Mainly,

• Learning Windows administration in depth.
• Testing Vyatta and hope to eventually use it as my primary router.
• Learning to implement more complex network setups with various routing protocols. I’ll eventually add more routers to my existing virtualized topology. Now I’m just getting the basics off the ground.
• Learning some basic Linux administration. I’m adding Ubuntu 8.04 to the pile of virtualized OS soon, I currently have it installed on my desktop via Wubi.
• Learning and deploying IPv6 as a minor objective after all the above has been satisfied.

My main home router is an old P3 450 MHz machine that’s finding new purpose in life and now runs IPCop. The router for my virtualized network runs Vyatta.

Taking advantage of Microsoft’s trial program for their server and developer products, I downloaded a copy of Windows Server 2008. The 60 day evaluation period can be extended to 240 days, and doing so is even blessed by Microsoft. One reinstallation every 240 days sound fair enough for me. This time I really really am going sit down and teach myself Active Directory. Yes, really.

In order for the newly added network to be recognized, an entry in IPCop’s routing table needed to be added. Being a SOHO router distribution, it doesn’t support any routing protocols so a static route had to be manually added. As a Linux newbie, the whole process took way longer because of a silly mistake that I made, missing out the gateway portion of the route add command. From my past experiences working on Cisco routers, adding a static path required you only to specify the destination network address, mask and either the next hop router address or outgoing interface. I hastily entered the same, specifying the outgoing interface without realizing that if a gateway is between the two networks, I’ve to use the next hop address syntax instead of just specifying the outgoing interface.

Correct:

route add -net 192.168.5.0 netmask 255.255.255.0 gw 192.168.1.150

Wrong:

route add -net 192.168.5.0 netmask 255.255.255.0 eth0

This resulted in a scenario of traffic being able to reach from 192.168.1.1 to 192.168.5.1, but not to the rest of the 192.168.5.0 network, which puzzled me greatly until I realized my folly.

Another silly was the result of cloning VMWare machines. The fact that cloned machines would have similar MAC addresses never crossed my mind, and that caused all kinds of havoc on its own. It was fixed by changing the MAC address in the machine configuration file (.vmx).

Aside from those issues, setting up the network was relatively easy. Vyatta took minimal configuration out of the box.

Setting up the interfaces:

set interfaces ethernet eth0 address 192.168.1.150/24
set interfaces ethernet eth1 address 192.168.5.1/24

Default routing to allow internet access:

set protocols static route 0.0.0.0/0 next-hop 192.168.1.1

Enable the ssh service:

set service ssh

In Vyatta, changes are not committed immediately after typing in every command until you explicitly tell it to, so:

commit

Done for now, more updates on this small project of mine as they come along. Oh, and I’m proud of the network diagram I drew, it’s the cleanest and neatest one I’ve ever done. Previously they were hand drawn or MS Paint jobs, this one’s done in Powerpoint.