View source for XUELK-TN-001: Configuring SBC Lynx as industrial router

{{InfoBoxTop}}
{{AppliesToAXELULite}}
{{AppliesToAXEL ULite TN}}
{{AppliesToSBCLynx}}
{{InfoBoxBottom}}
== History ==
{| class="wikitable" border="1"
!Version
!Date
!Notes
|-
|1.0.0
|August 2016
|First public release
|-
|{{oldid|5301|1.1.0}}
|August 2016
|Added section about persistent rules
|-
|1.1.1
|August 2016
|Added sysctl persistent settings
|-
|1.1.2
|May 2017
|Fix sysctl parameter
|-
|}

==Introduction==
Thanks to the dual Ethernet interface, SBC Lynx allows to implement non-trivial network configurations. As an example of this flexibility, this article shows how to configure SBC Lynx to implement a Linux-powered router that manages data packet forwarding between two different LANs. This task can be performed in parallel with the other application-specific activities (typically field bus communications, monitoring, control etc.). This solution allows to reduce significantly overall infrastructure costs in many industrial environments where Ethernet networking is popular nowadays.

==Network topology==
The following image shows a simplified block diagram of the network topology that has been used for testing this configuration.


[[File:SBCLynx-router-IP-forwarding.png|thumb|center|600px|Simplified block diagram of the network topology]]


Two LANs have been used:
* main LAN (192.168.0.0/24)
* secondary LAN (192.168.11.0/24).

The following devices are connected to these networks:
*a PC connected to the main LAN (IP address = 192.168.0.28)
*main LAN switch
*SBC Lynx equipped with two Ethernet interfaces
**primary interface (eth0) connected to main LAN (IP address = 192.168.0.209)
**secondary interface (eth1) connected to secondary LAN (IP address = 192.168.11.209) [1]
*secondary LAN managed switch (IP address = 192.168.11.239)
*WiFi access point connected to secondary LAN (IP address = 192.168.11.241)

Secondary LAN managed switch and access point integrate a web server, accessible at port 80. Two IP forwarding rules have been be set up in order to make web servers accessible at 192.168.0.209:80 and 192.168.0.209:8080:
*192.168.0.209:80 <-> 192.168.11.241:80
*192.168.0.209:8080 <-> 192.168.11.239:80



[1] For simplicity, secondary interface has been implemented with an USB/Ethernet adapter (MOSCHIP 7830/7832/7730 usb-NET adapter) connected to USB port. For a real-world production environment, it is recommended the use of both iMX6UL Ethernet MAC controllers. To do that, a plugin board connected to the one piece connector (J45/J52) can be used. For more details please refer to [mailto:sales@dave.eu sales department].

==Implementation==
To enable routing functionality, the well known [https://www.netfilter.org/ netfilter/iptables packet filtering framework] has been added to the software provided along with [[AXEL_ULite_and_SBC_Lynx_Embedded_Linux_Kit_(XUELK)|XUELK]] by default.

The following steps describe how to set up and configure netfilter to implement the desired routing policy.

First make sure to correctly setup static IP for the two ethernet interfaces on SBC Lynx:
<pre>
root@sbc-lynx:~# ifconfig eth0 192.168.0.209
root@sbc-lynx:~# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:50:C2:B9:CF:82
          inet addr:192.168.0.209  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::250:c2ff:feb9:cf82/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:64117 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19470 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:25075518 (23.9 MiB)  TX bytes:3398088 (3.2 MiB)

root@sbc-lynx:~# ifconfig eth1 192.168.11.209
root@sbc-lynx:~# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:D0:10:03:26:0A
          inet addr:192.168.11.209  Bcast:192.168.11.255  Mask:255.255.255.0
          inet6 addr: fe80::2d0:10ff:fe03:260a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3541 errors:0 dropped:186 overruns:0 frame:0
          TX packets:2023 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1840061 (1.7 MiB)  TX bytes:555670 (542.6 KiB)

</pre>

Before proceeding on port forwarding rules setting, the forwarding capability must be enabled on both <code>eth0</code> and <code>eth1</code> interfaces:
<pre>
root@sbc-lynx:~# sysctl -w net.ipv4.conf.eth0.forwarding=1
net.ipv4.conf.eth0.forwarding = 1
root@sbc-lynx:~# sysctl -w net.ipv4.conf.eth1.forwarding=1
net.ipv4.conf.eth1.forwarding = 1
</pre>

The following <code>iptables</code> commands are used to enable 192.168.0.209:80 <-> 192.168.11.241:80 port forwarding:
<pre>
iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to-destination 192.168.11.241:80
iptables -A FORWARD -p tcp -d 192.168.11.241 --dport 80 -j ACCEPT
iptables -t nat -A POSTROUTING -p tcp --dport 80 -d 192.168.11.241 -o eth1 -j SNAT --to-source 192.168.11.209
</pre>

And the following enables 192.168.0.209:80 <-> 192.168.11.241:80 port forwarding:
<pre>
iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 8080 -j DNAT --to-destination 192.168.11.239:80
iptables -A FORWARD -p tcp -d 192.168.11.239 --dport 80 -j ACCEPT
iptables -t nat -A POSTROUTING -p tcp --dport 80 -d 192.168.11.239 -o eth1 -j SNAT --to-source 192.168.11.209
</pre>

In details the 192.168.0.209:80 <-> 192.168.11.241:80 port forwarding rules are:
* <code>iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 80 -j DNAT --to-destination 192.168.11.241:80</code>
** All TCP packets on port 80 in input from ''eth0'' interface are modified with destination ip adddress ''192.168.11.241'' port 80
* <code>iptables -A FORWARD -p tcp -d 192.168.11.241 --dport 80 -j ACCEPT</code>
** This rule tells forward chain in the filter table to accept TCP packets on port 80 with destination IP address equal to ''192.168.11.241''. This rule is not strictly necessary, because by default filter tables accepts all packets. But it is useful for logging and packet statistic (see [[#Enabling_logging | Enabling logging]])
* <code>iptables -t nat -A POSTROUTING -p tcp --dport 80 -d 192.168.11.241 -o eth1 -j SNAT --to-source 192.168.11.209</code>
** This rule translate the source IP address of all the TCP packets on port 80 in output on ''eth1'' interface with destination IP address equal to ''192.168.11.241''

Here is a dump of the <code>FILTER</code> and <code>NAT</code> tables with the port forwarding rules :
<pre>
root@sbc-lynx:~# iptables -t filter -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  0.0.0.0/0            192.168.11.241       tcp dpt:80
ACCEPT     tcp  --  0.0.0.0/0            192.168.11.239       tcp dpt:80

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
</pre>
<pre>
root@sbc-lynx:~# iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DNAT       tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:192.168.11.241:80
DNAT       tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 to:192.168.11.239:80

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
SNAT       tcp  --  0.0.0.0/0            192.168.11.241       tcp dpt:80 to:192.168.11.209
SNAT       tcp  --  0.0.0.0/0            192.168.11.239       tcp dpt:80 to:192.168.11.209
</pre>

Now on the PC side (192.168.0.28) it is possible to get access to either managed switch and access point web servers by using an web browser.

===Enabling logging===
To enable <code>iptables</code> logging capability some kernel drivers must be added to default configuration provided along with XUELK.

The logging functionality can be useful for troubleshooting the iptables custom configuration. But at the same time if it is not well configured it can be too much verbose and useless, especially if there is lot of traffic on the LAN.

Enabling <code>iptables</code> port forwarding log is a matter of adding rules on the chains that are interested on the port forwarding path. Here is a basic implementation of the port forwarding log:
<pre>
iptables -t nat -I PREROUTING -j LOG --log-prefix "NAT-prerouting: " --log-level 7
iptables -t nat -I POSTROUTING -j LOG --log-prefix "NAT-postrouting: " --log-level 7
iptables -t filter -I FORWARD -j LOG --log-prefix "FORWARD-Filter: " --log-level 7
</pre>

The LOG output is appended on <code>/var/log/messages</code> file. Please note that the size of this log file in XUELK is limited to ''265kB''. When the limit size is reached the log file is backed up on <code>/var/log/messages.0</code> and a new empty log file is started.

There are various logging options. The two used in this example are the most common:
* <code>--log-prefix</code> : it adds a custom string on the beginning of every log entry. This is useful to immediately recognize the rule that is logged.
* <code>--log-level</code> : choose the log level from the standard linux log level. Selecting low log level can prints out all the iptables log also on debug console.

Other common filtering options can be used to reduce <code>iptables</code> log output size: see [https://www.netfilter.org/documentation/HOWTO/packet-filtering-HOWTO-7.html#ss7.3 Filtering Specifications]

Here is a section of the logging output showing port forwarding in case of accessing the access point web server from the PC. Be aware that the nat table is traversed only by the first packet of each connection.
<pre>
Mar  6 03:04:56 sbc-lynx user.debug kernel: NAT-prerouting: IN=eth0 OUT= MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.0.209 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=28609 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=28609 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: NAT-postrouting: IN= OUT=eth1 SRC=192.168.0.28 DST=192.168.11.241 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=28609 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=80 DPT=57227 WINDOW=5840 RES=0x00 ACK SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=28610 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=16425 RES=0x00 ACK URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=520 TOS=0x00 PREC=0x00 TTL=127 ID=28611 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=16425 RES=0x00 ACK PSH URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=63706 DF PROTO=TCP SPT=80 DPT=57227 WINDOW=3456 RES=0x00 ACK URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=89 TOS=0x00 PREC=0x00 TTL=63 ID=63707 DF PROTO=TCP SPT=80 DPT=57227 WINDOW=3456 RES=0x00 ACK PSH URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=839 TOS=0x00 PREC=0x00 TTL=63 ID=63708 DF PROTO=TCP SPT=80 DPT=57227 WINDOW=3456 RES=0x00 ACK PSH FIN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=28612 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=16213 RES=0x00 ACK URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=28613 DF PROTO=TCP SPT=57227 DPT=80 WINDOW=16213 RES=0x00 ACK FIN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=80 DPT=57227 WINDOW=3456 RES=0x00 ACK URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: NAT-prerouting: IN=eth0 OUT= MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.0.209 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=28616 DF PROTO=TCP SPT=57230 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=28616 DF PROTO=TCP SPT=57230 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: NAT-postrouting: IN= OUT=eth1 SRC=192.168.0.28 DST=192.168.11.241 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=28616 DF PROTO=TCP SPT=57230 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: NAT-prerouting: IN=eth0 OUT= MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.0.209 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=28617 DF PROTO=TCP SPT=57231 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=28617 DF PROTO=TCP SPT=57231 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: NAT-postrouting: IN= OUT=eth1 SRC=192.168.0.28 DST=192.168.11.241 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=28617 DF PROTO=TCP SPT=57231 DPT=80 WINDOW=8192 RES=0x00 SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=80 DPT=57230 WINDOW=5840 RES=0x00 ACK SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth1 OUT=eth0 MAC=00:d0:10:03:26:0a:00:1f:1f:b0:52:30:08:00 SRC=192.168.11.241 DST=192.168.0.28 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=80 DPT=57231 WINDOW=5840 RES=0x00 ACK SYN URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=28618 DF PROTO=TCP SPT=57230 DPT=80 WINDOW=16425 RES=0x00 ACK URGP=0
Mar  6 03:04:56 sbc-lynx user.debug kernel: FORWARD-Filter: IN=eth0 OUT=eth1 MAC=00:50:c2:b9:cf:82:90:b1:1c:69:58:80:08:00 SRC=192.168.0.28 DST=192.168.11.241 LEN=484 TOS=0x00 PREC=0x00 TTL=127 ID=28619 DF PROTO=TCP SPT=57230 DPT=80 WINDOW=16425 RES=0x00 ACK PSH URGP=0
</pre>

===Make <code>iptables</code> configuration persistent===
<code>iptables</code> init script is used to make rules persistent in order to load them automatically on boot.

The init script must be saved in the target's root file system as <code>/etc/init.d/iptables</code>. From SBC Lynx the following commands can be used to create and edit the file:
<pre>
root@sbc-lynx:~# touch /etc/init.d/iptables
root@sbc-lynx:~# chmod +x /etc/init.d/iptables
root@sbc-lynx:~# vi /etc/init.d/iptables
</pre>

Here is the content of the script:
<pre>
#! /bin/bash
### BEGIN INIT INFO
# Provides:          iptables
# Required-Start:    mountkernfs $local_fs
# Required-Stop:     mountkernfs $local_fs
# X-Start-Before:    networking
# X-Stop-After:      networking
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Iptables
# Description:       Init script for iptables
### END INIT INFO

function do_start {
	if [ -e "/etc/iptables.rules" ]; then
		echo "Starting iptables service"
		iptables-restore < /etc/iptables.rules
	else
		echo "No rules saved for iptables"
	fi
}

function do_stop {
	echo "Stopping iptables service"
	for chain in INPUT FORWARD OUTPUT
	do
		iptables -P $chain ACCEPT
	done    
	for param in F Z X; do iptables -$param; done
	for table in $(cat /proc/net/ip_tables_names)
	do
		iptables -t $table -F
		iptables -t $table -Z
		iptables -t $table -X
	done
}

function do_save {
	echo "Saving iptables rules"
	iptables-save > /etc/iptables.rules
}

case "$1" in
	start)
		do_start
	;;
	stop)
		do_stop
	;;
	save)
		do_save
	;;
	restart)
		do_stop
		do_start
	;;
	*)
		echo "Usage: /etc/init.d/iptables {start|stop|restart|save}"
		exit 1
	;;
esac

exit 0

</pre>

Install the <code>iptables</code> init script by simply issuing this command:
<pre>
update-rc.d iptables defaults
</pre>

To save the current <code>iptables</code> rules and make them persistent type this command:
<pre>
root@sbc-lynx:~# /etc/init.d/iptables save
Saving iptables rules
</pre>

At the next boot the saved <code>iptables</code> rules will be automatically loaded.


----

Please note that <code>sysctl</code> settings (e.g. the ones used to enable packet forwarding) are not persistent across reboots. To apply sysctl settings at boot time automatically, just add them to [http://linux.die.net/man/5/sysctl.conf <code>/etc/sysctl.conf</code>] as <code>token = value</code>:

<pre>
root@sbc-lynx:~# tail /etc/sysctl.conf
#net.ipv6.conf.all.accept_source_route = 0
#
# Log Martian Packets
#net.ipv4.conf.all.log_martians = 1
#

#kernel.shmmax = 141762560

net.ipv4.conf.eth0.forwarding=1
net.ipv4.conf.eth1.forwarding=1
</pre>

sysctl.conf settings are applied with init script during network configuration (see <code>/etc/init.d/networking</code>)

To check sysctl.conf syntax user can apply those settings also manually with the following command:

<pre>
root@sbc-lynx:~# sysctl -p /etc/sysctl.conf
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.eth1.forwarding = 1
</pre>