Using SSL client certs with Perl’s LWP::UserAgent
by bigpresh on Mar.29, 2013, under Perl, Programming
I recently needed to authenticate to a remote API using an SSL client certificate, and had a bit of trouble getting LWP::UserAgent to work with it.
The examples I found which looked like they should work involved e.g.:
use LWP::UserAgent; my $ua = LWP::UserAgent->new( ssl_opts => { SSL_use_cert => 1, SSL_cert_file => "/path/to/clientcert.crt", SSL_key_file => "/path/to/privatekey.key", }, );
However, that didn’t work; changing the paths to the cert/key to non-existent files didn’t cause any difference, so I suspected that those options were actually being ignored.
After a fair bit of digging, the option I found that actually worked was loading Net::SSL first, to make LWP use Net::SSLeay, and setting env vars to the client cert to use:
use Net::SSL; use LWP::UserAgent; $ENV{HTTPS_CERT_FILE} = "/path/to/clientcert.crt"; $ENV{HTTPS_KEY_FILE} = "/path/to/privatekey.key"; my $ua = LWP::UserAgent->new();
This, to me, is pretty icky – I’d much rather pass config to affect just that single LWP object. However, it gets it working.
March 29th, 2013 on 2:31 pm
This has been seen recently in Mojo::UserAgent too (documented here: http://stackoverflow.com/questions/15534849/mojouseragent-tls-ssl-certificate-authentication). I wonder if there is a real bug in here somewhere?
April 1st, 2013 on 10:18 am
Your approach looks weird. Which version of LWP::UserAgent, LWP::Protocol::https, IO::Socket::SSL is that?
April 1st, 2013 on 1:13 pm
Net::SSL does not use Net::SSLeay, but it uses Crypt::SSLeay. While Crypt::SSLeay does basic SSL stuff, it does not verification of the servers hostname, which makes man in the middle easy.
Current LWP version therefore use IO::Socket::SSL (which is based on Net::SSLeay, not Crypt::SSLeay) and enable strict certificate verification, including checking the hostname of the certificate against the hostname given in the URL.
I’ve checked client-side certificate handling and it works like advertised in LWP::UserAgent. But you will get an error, if the name in the servers certificate does not match the name in the URL. And using Net::SSL in this case only works because the underlying Crypt::SSLeay simply ignores this error. So it’s probably not a fault of LWP, but the fault of a certificate mismatch.
You can switch off checking the hostname with LWP too by setting verify_hostname to false in ssl_opts (as documented).
Unfortunatly all this SSL stuff is not easy to debug :(
April 1st, 2013 on 1:24 pm
Where does one get a certificate/key pair?
April 1st, 2013 on 7:23 pm
It looks like a bug. Did you report it?
April 1st, 2013 on 7:31 pm
@Mark: Not yet – I haven’t had time to verify that it is indeed a bug, rather than me getting it wrong, and if a bug, which part of the toolchain it’s in (LWP::UserAgent, Net::SSL, Net::SSLeay, IO::Socket::SSL…)
April 1st, 2013 on 7:34 pm
@Steffen: Ahhh – that might be the case. I’ll have a look and see if setting verify_hostname solves it. Are you saying that there’ll be an error if the client certificate’s common name doesn’t match the server’s common name, or talking about normal server cert validation? (If it were the latter, I’d expect it to still give the same result, sine I didn’t disable checking, just passed details of the client cert in a different manner.)
April 1st, 2013 on 7:39 pm
@daxim: Versions of modules installed:
April 1st, 2013 on 7:40 pm
@Meir: You can generate your own key and CSR (certificate signing request) via the openssl tool; whoever runs the server would then sign that CSR using the server’s CA cert & key.
April 9th, 2013 on 3:50 pm
I have code that works without use-ing Net::SSL:
my $ua = LWP::UserAgent->new(
keep_alive => undef,
timeout => $self->timeout,
ssl_opts => {
verify_hostname => 0,
SSL_cert_file => $ssl_cert_file,
SSL_key_file => $ssl_key_file,
},
);
I haven’t had to set SSL_use_cert => 1.
I had a similar workaround before ssl_opts => { verify_hostname => 0 } existed:
$ENV{HTTP_PROXY} = ”;
$ENV{HTTPS_PROXY} = ”;
$ENV{http_proxy} = ”;
$ENV{https_proxy} = ”;
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
At this time my deps where:
LWP = 5.834
Crypt::SSLeay = 0.58
Later I swithced Crypt::SSLeay for IO::Socket::SSL.
When LWP::Protocol::https was split into a separate dist I’ve changed my IO::Socket::SSL 1.39 requirement to it.
Hope that helps.
April 12th, 2013 on 12:23 pm
As stated in another comment, LWP can use either
Net::SSL + Crypt::SSLeay (a single CPAN distribution), either IO::Socket::SSL + Net::SSLeay (two distinct CPAN distribution).
However, while https protocol support is indeed transparent, there is absolutly no abstraction over low-level features of those different SSL bindings for server certificat checking, in LWP 5.x, and just minimal ones in LWP 6.x. All you have is pure-perl based regexp checking on certificate subject. If you want fine-grained control, you have to:
– ensure which exact binding is used
– use binding-specific settings
To make the puzzle even more complex, those bindings have different capacities (Net::SSL can’t validate server hostname), LWP 6.x has different default settings, and Redhat backported 6.x features silently into 5.x…
The following piece of code (_setSSLOptions method, line 127) illustrates how to handle the various situations:
https://github.com/fusinv/fusioninventory-agent/blob/master/lib/FusionInventory/Agent/HTTP/Client.pm
Hope this helps.