0

I have a CentOS system with libreswan behind a router with static IP, and I have been trying to setup an IPSec tunnel with a server at remote location having juniper firewall. The IPSec VPN settings on the remote server are done via firewall. I have already tried almost every possible combination of settings, but every time, same error of 'malformed payload' is being encountered. Following is the usual log displayed on CentOS shell screen: 

002 "GeojitOMS" #6: initiating Main Mode
104 "GeojitOMS" #6: STATE_MAIN_I1: initiate
003 "GeojitOMS" #6: ignoring unknown Vendor ID payload [2c9d7e81995b9967d23f571ac641f9348122f1cc1200000014060000]
003 "GeojitOMS" #6: received Vendor ID payload [draft-ietf-ipsec-nat-t-ike-02_n]
003 "GeojitOMS" #6: received Vendor ID payload [Dead Peer Detection]
003 "GeojitOMS" #6: ignoring Vendor ID payload [HeartBeat Notify 386b0100]
002 "GeojitOMS" #6: enabling possible NAT-traversal with method draft-ietf-ipsec-nat-t-ike-02/03
002 "GeojitOMS" #6: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
106 "GeojitOMS" #6: STATE_MAIN_I2: sent MI2, expecting MR2
003 "GeojitOMS" #6: NAT-Traversal: Result using draft-ietf-ipsec-nat-t-ike-02/03 sender port 500: I am behind NAT+peer behind NAT
002 "GeojitOMS" #6: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
108 "GeojitOMS" #6: STATE_MAIN_I3: sent MI3, expecting MR3
003 "GeojitOMS" #6: next payload type of ISAKMP Hash Payload has an unknown value: 210 (0xd2)
003 "GeojitOMS" #6: malformed payload in packet
010 "GeojitOMS" #6: STATE_MAIN_I3: retransmission; will wait 500ms for response
010 "GeojitOMS" #6: STATE_MAIN_I3: retransmission; will wait 1000ms for response
010 "GeojitOMS" #6: STATE_MAIN_I3: retransmission; will wait 2000ms for response
010 "GeojitOMS" #6: STATE_MAIN_I3: retransmission; will wait 4000ms for response
003 "GeojitOMS" #6: discarding duplicate packet; already STATE_MAIN_I3

Having tried every combination of settings on libreswan, I am wondering if it has to do with the version on CentOS itself being incompatible with juniper. This is the version of libreswan and kernel:

[root@localhost xyz]# rpm -qa libreswan
libreswan-3.15-5.el7_1.x86_64
[root@localhost xyz]# uname -r
3.10.0-327.4.5.el7.x86_64

Port 500 and 4500 of UDP are open on CentOS iptables. Also all the upstream and downstream traffic on beetel router for the ports is permitted. Here is my final connection setting, which is still not working out:

My Local Subnet :10.0.0.0/24

Remote Subnet: 192.168.11.0/28

The VPN is suppose to connect the local machine with two boxes on remote subnet, namely 192.168.11.11 and 192.168.11.12, which I think is configured via subnet, unless there is a way in libreswan to mention such two specific servers in same connection. The /etc/ipsec.d/connection.conf:

conn Connection
    auto=start
    leftid=1.2.3.4 //Some pre-defined id for local machine
    left=10.0.0.16 //LAN IP of local machine
    #leftnexthop=xxx.yyy.zzz.www // Public static IP on router
    leftsubnet=10.0.0.0/24 //local subnet
    rightid=1.2.3.5 //Some pre-defined id for remote server
    right=www.zzz.yyy.xxx //Public IP of the remote server 
    rightsubnet=192.168.11.0/28 //Remote subnet
    #rightnexthop=192.168.11.11 //Remote server IP in remote LAN? not sure
    authby=secret
    ike=3des-sha1;modp1024
    phase2=esp
    phase2alg=3des-sha1
    #pfs=no
    forceencaps=yes
    compress=yes
    #ikev2=propose
    dpdaction=restart

Secrets file /etc/ipsec.secrets: 1.2.3.4 1.2.3.5: PSK "sharedkey"

Different combinations of variables in 'conn' are tried along with enabling disabling 'nat_traversal'. But no matter what combination is used, I still get the same error. Is there anything missing in these settings or is there a compatibility issue between juniper and libreswan or the particular version of libreswan?

Improve this question

1 Answer 1

Reset to default
1

I found the libreswan documentation helpful here concerning interop with Juniper.

I feel your pain. VPN's are notoriously difficult to get just right. The seemingly smallest thing can render the connection unstable and/or useless. As such, I can't determine exactly which setting you posted above is wrong or missing so I'll comment on a few lines that may be causing trouble.

phase2=esp phase2alg=3des-sha1 #pfs=no

I had no success at all trying to specify these even when they were correct -- the connection never succeeded. When I let these values be auto-negotiated it magically worked.

compress=yes

Compression should not be enabled because it's a security vulnerability.

For reference, I've successfully achieved a libreswan<-->Juniper tunnel with the following (obfuscated) config. In this config the folling is true:

  1. left==local(libreswan) and right==remote(juniper).
  2. OS is CentOS 7.2
  3. libreswan package is libreswan-3.15-5.el7_1.x86_64
  4. The local libreswan is behind NAT. Juniper NAT status is unknown.
  5. The only Juniper side information I was given:
    • A pre-shared key
    • Phase 1 == "pre-gp2-aes256-sha-24h"
    • Phase 2 == "g2-esp-aes128-sha"
  6. nat_traversal=yes in /etc/ipsec.conf

Config file:

conn MyConnection
        ike=aes256-sha1;modp1024
        esp=aes128-sha1
        authby=secret
        keyingtries=0

        left=10.111.111.111
        leftsourceip=10.111.111.111
        leftsubnet=10.111.111.0/24

        right=1.2.3.4
        rightsubnet=10.222.222.0/24
        rightnexthop=%defaultroute

        compress=no
        auto=start
Improve this answer
1
  • We ordered a hardware firewall from sonicwall already and it worked with no issues. Though I guess this is one setting which I may not have tried, i.e. removing the phase 1 and phase 2 specs. Thanks.
    – packet
    Commented May 10, 2016 at 9:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .