'Openssl 3.0. Rsa private methods not called
Recently I have migrated my app from OpenSSL 1.0 to OpenSSL 3.0. And now my connection is not working anymore. The methods which perform private encryption and private decryption are not called. This is what I used to have:
RSA *rsaKey = RSA_new();
BN_hex2bn(&rsaKey->n, n.c_str());
BN_hex2bn(&rsaKey->e, e.c_str());
RSA_set_method(rsaKey, &cardMethod);
SSL_CTX_use_RSAPrivateKey(m_sslCtx, rsaKey);
SSL_CTX_set_verify_depth(m_sslCtx, 2);
This what the definition of cardMethod
looked like:
static RSA_METHOD cardMethod = {
0,0,0,
cardPrivEnc,
cardPrivDec,
0,0,0,0,0,0,
0,0,0
};
These functions cardPrivEnc
and cardPrivDec
were later called during connection to server by SSL_connect
inner implementation through RSA_sign
.
Then I had to change my code to:
RSA *rsaKey = RSA_new();
m_cardMethod = RSA_meth_new("cardMethod", 0);
RSA_meth_set_priv_dec(m_cardMethod, cardPrivDec);
RSA_meth_set_priv_enc(m_cardMethod, cardPrivEnc);
BIGNUM* nBignum = NULL;
BIGNUM* eBignum = NULL;
BN_hex2bn(&nBignum, n.c_str());
BN_hex2bn(&eBignum, e.c_str());
RSA_set0_key(rsaKey, nBignum, eBignum, NULL);
RSA_set_method(rsaKey, m_cardMethod);
SSL_CTX_use_RSAPrivateKey(m_sslCtx, rsaKey);
SSL_CTX_set_verify_depth(m_sslCtx, 2);
The inner implementation of SSL_connect
has changed in OpenSSL 3.0 obviously and now cardPrivEnc
and cardPrivDec
are not called. How do I fix it?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|