Use case: IOT-device connected through AWS cloud
The IOT-device is behind a router that sends all traffic through aws cloud.
The IOT-server can not be configured and thus is not part of the AWS cloud
For configuration, the IOT-device needs to be notfied with an UPD packet to port xxxxx to establish an administion connection. This udp packet can not be sent directly to AWS cloud
Thus, we need a communication server to route the UDP packet:
Routing on the IOT-server can not be configured, thus, the UDP packet needs to be send to zz.zz.zz.zz
The communication server runs debian 10 with strongswan
ipsec.conf:
conn %default
mobike=no
compress=no
authby=secret
keyexchange=ike
ike=aes128-sha1-modp1024!
ikelifetime=8h
esp=aes128-sha1-modp1024!
lifetime=1h
rekeymargin=3m
keyingtries=%forever
installpolicy=yes
dpdaction=restart
type=tunnel
conn dc-aws1
leftsubnet=zz.zz.zz.zz #local subnet
right=vv.vv.vv.vv # AWS Gateway Public IP
rightsubnet=xx.xx.0.0/16 #remoye subnet
auto=start
include /var/lib/strongswan/ipsec.conf.inc
The following parts of the connection work: Standard operation works just fine.
The ipsec connection is running (as expected):
sudo ipsec status
Security Associations (1 up, 0 connecting):
dc-aws1[3]: ESTABLISHED 11 seconds ago, zz.zz.zz.zz[zz.zz.zz.zz]...vv.vv.vv.vv[vv.vv.vv.vv]
dc-aws1{16}: INSTALLED, TUNNEL, reqid 2, ESP in UDP SPIs: cd6dfea5_i 401dc4d5_o
dc-aws1{16}: zz.zz.zz.zz/32 = xx.xx.0.0/16
dc-aws1{17}: INSTALLED, TUNNEL, reqid 2, ESP in UDP SPIs: c2507a98_i 9d083aa4_o
dc-aws1{17}: zz.zz.zz.zz/32 = xx.xx.0.0/16
sudo ip xfrm policy show
src zz.zz.zz.zz/32 dst xx.xx.0.0/16
dir out priority 375423 ptype main
tmpl src zz.zz.zz.zz dst vv.vv.vv.vv
proto esp spi 0x9d083aa4 reqid 2 mode tunnel
src xx.xx.0.0/16 dst zz.zz.zz.zz/32
dir fwd priority 375423 ptype main
tmpl src vv.vv.vv.vv dst zz.zz.zz.zz
proto esp reqid 2 mode tunnel
src xx.xx.0.0/16 dst zz.zz.zz.zz/32
dir in priority 375423 ptype main
tmpl src vv.vv.vv.vv dst zz.zz.zz.zz
proto esp reqid 2 mode tunnel
Ping works between the router and the communication server though the vpn connection.
Traceroute also works if icmp packets are used.
In order to forward the upd packet to the IOT-device, network address translation is used with iptables
iptables -t nat -I PREROUTING -p udp -s yy.yy.yy.yy --dport xxxxx -j DNAT --to xx.xx.xx.xx
xfrm policy does not apply, if the source is yy.yy.yy.yy, thus source network address translation is used as well
iptables -t nat -I POSTROUTING -p udp -s yy.yy.yy.yy --dport xxxxx -j SNAT --to-source zz.zz.zz.zz
also a forwarding rule is needed
iptables -I FORWARD -p udp -d xx.xx.xx.xx --dport xxxxx -j ACCEPT
tcpdump shows, that udp packets arrive and are forwarded (in between, there are alive messages for the vpn connection):
sudo tcpdump -n -i any host vv.vv.vv.vv or port xxxxx
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
08:22:48.520734 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: NONESP-encap: isakmp: child_sa inf2[I]
08:22:48.535700 IP vv.vv.vv.vv.4500 > zz.zz.zz.zz.4500: NONESP-encap: isakmp: child_sa inf2[R]
08:22:56.717778 IP yy.yy.yy.yy.54278 > zz.zz.zz.zz.xxxxx: UDP, length 108
08:22:56.717908 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x81f1d489,seq=0x1), length 180
08:23:06.344622 IP yy.yy.yy.yy.46955 > zz.zz.zz.zz.xxxxx: UDP, length 108
08:23:06.344749 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x81f1d489,seq=0x2), length 180
08:23:10.797048 IP yy.yy.yy.yy.33667 > zz.zz.zz.zz.xxxxx: UDP, length 108
08:23:10.797247 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x81f1d489,seq=0x3), length 180
08:23:18.521104 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: NONESP-encap: isakmp: child_sa inf2[I]
08:23:18.536895 IP vv.vv.vv.vv.4500 > zz.zz.zz.zz.4500: NONESP-encap: isakmp: child_sa inf2[R]
08:23:25.423142 IP yy.yy.yy.yy.40703 > zz.zz.zz.zz.xxxxx: UDP, length 108
08:23:25.423271 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x81f1d489,seq=0x4), length 180
08:23:31.756269 IP yy.yy.yy.yy.58584 > zz.zz.zz.zz.xxxxx: UDP, length 108
08:23:31.756378 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x81f1d489,seq=0x5), length 180
^C
14 packets captured
14 packets received by filter
0 packets dropped by kernel
What does not work:
However the udp pakets seem to get lost. In the logs at aws, no traffic is visible in the tunnel. Also, no packets arrive at the router.
Traceroute with udp and tcp packets does not work.
The problem can be reproduced when running netcat on the communication server in listening mode and connecting to it from behind the router. In the tcp dump, the syn packets are arriving, a response seems to be sent out, but there is no traffic arriving from the communication server in the aws cloud. tcpdump from the communication server for this test:
11:35:06.597736 IP vv.vv.vv.vv.4500 > zz.zz.zz.zz.4500: UDP-encap: ESP(spi=0xcb99370a,seq=0x1), length 100
11:35:06.597736 IP xx.xx.xx.xx.49768 > zz.zz.zz.zz.15952: Flags [S], seq 101710370, win 64240, options [mss 1350,sackOK,TS val 2355221232 ecr 0,nop,wscale 7], length 0
11:35:06.598157 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0xb), length 100
11:35:07.534252 IP vv.vv.vv.vv.4500 > zz.zz.zz.zz.4500: UDP-encap: ESP(spi=0xcb99370a,seq=0x2), length 100
11:35:07.534252 IP xx.xx.xx.xx.49768 > zz.zz.zz.zz.15952: Flags [S], seq 101710370, win 64240, options [mss 1350,sackOK,TS val 2355222233 ecr 0,nop,wscale 7], length 0
11:35:07.534445 IP zz.zz.zz.zz.4500 > vv.vv.vsv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0xc), length 100
11:35:08.561060 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0xd), length 100
11:35:09.559712 IP vv.vv.vv.vv.4500 > zz.zz.zz.zz.4500: UDP-encap: ESP(spi=0xcb99370a,seq=0x3), length 100
11:35:09.559712 IP xx.xx.xx.xx.49768 > zz.zz.zz.zz.15952: Flags [S], seq 101710370, win 64240, options [mss 1350,sackOK,TS val 2355224249 ecr 0,nop,wscale 7], length 0
11:35:09.559908 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0xe), length 100
11:35:11.569079 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0xf), length 100
11:35:13.672232 IP vv.vv.vv.vv.4500 > zz.zz.zz.zz.4500: UDP-encap: ESP(spi=0xcb99370a,seq=0x4), length 100
11:35:13.672232 IP xx.xx.xx.xx.49768 > zz.zz.zz.zz.15952: Flags [S], seq 101710370, win 64240, options [mss 1350,sackOK,TS val 2355228377 ecr 0,nop,wscale 7], length 0
11:35:13.672319 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0x10), length 100
11:35:17.713025 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0x11), length 100
11:35:25.905124 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0x12), length 100
11:35:42.033153 IP zz.zz.zz.zz.4500 > vv.vv.vv.vv.4500: UDP-encap: ESP(spi=0x9a2c6938,seq=0x13), length
It is not clear to me, where the packets could get lost. Any hint on how to narrow down the problem is welcome
** Update **
In the meanwhile I double-checked configuration, but with no success.
Then, I switched to Openswan (2.6.51.5), which is tested by AWS.
With Openswan packets arrive in the cloud as expected.
My conclusion is, that Strongswan is incompatible with AWS VPC.
I'm happy to make further tests if there are ideas on how to test further.