I am trying to decrypt an XML package that was encrypted by a SHA-256 certificate. So far, any examples have focused on SHA-1.
As directed in this post, I have created a CK_RSA_PKCS_OAEP_PARAMS struct type, and populated it with the SHA-256 algorithms.
Unfortunately, calling the session's DecryptInit() with the CKM_RSA_PKCS_OAEP mechanism and its parameters always returns error code 13 (CKR_MECHANISM_PARAM_INVALID). This occurs whether I use SHA-256 or the SHA-1 sample provided in the post linked above.
Here are the relevant sections of code:
[StructLayout(LayoutKind.Sequential, Pack = 0, CharSet = CharSet.Unicode)]
public struct CK_RSA_PKCS_OAEP_PARAMS
public ulong HashAlg;
public ulong Mgf;
public ulong Source;
public IntPtr SourceData;
public uint SourceDataLen;
Mechanism mech = new Mechanism(Mechanism.CKM_RSA_PKCS_OAEP, new CK_RSA_PKCS_OAEP_PARAMS()
HashAlg = Mechanism.CKM_SHA256,
Mgf = Mechanism.CKG_MGF1_SHA256,
Source = 1,
SourceData = IntPtr.Zero,
SourceDataLen = 0,
_session.DecryptInit(mech, privateKey); // This returns code 113