rsa: pss: Enable use of big hashes with small keys

It should be valid to RSASSA-PSS sign a SHA-512 hash with a 1024-bit or
1032-bit RSA key, but with the salt size being always equal to the hash
size, this isn't possible: the key is too small.

To enable use of hashes that are relatively large compared to the key
size, allow reducing the salt size to no less than the hash size minus 2
bytes. We don't allow salt sizes smaller than the hash size minus 2
bytes because that too significantly changes the security guarantees the
library provides compared to the previous implementation which always
used a salt size equal to the hash size. The new calculated salt size
remains compliant with FIPS 186-4.

We also need to update the "hash too large" test, since we now reduce
the salt size when certain key sizes are used. We used to not support
1024-bit keys with SHA-512, but now we support this by reducing the salt
size to 62. Update the "hash too large" test to use a 1016-bit RSA key
with SHA-512, which still has too large of a hash because we will not
reduce the salt size further than 2 bytes shorter than the hash size.

The RSA private key used for the test was generated using "openssl
genrsa 1016" using OpenSSL 1.1.1-pre8.

    $ openssl genrsa 1016
    Generating RSA private key, 1016 bit long modulus (2 primes)
    ..............++++++
    ....++++++
    e is 65537 (0x010001)
    -----BEGIN RSA PRIVATE KEY-----
    MIICVwIBAAKBgACu54dKTbLxUQBEQF2ynxTfDze7z2H8vMmUo9McqvhYp0zI8qQK
    yanOeqmgaA9iz52NS4JxFFM/2/hvFvyd/ly/hX2GE1UZpGEf/FnLdHOGFhmnjj7D
    FHFegEz/gtbzLp9X3fOQVjYpiDvTT0Do20EyCbFRzul9gXpdZcfaVHNLAgMBAAEC
    gYAAiWht2ksmnP01B2nF8tGV1RQghhUL90Hd4D/AWFJdX1C4O1qc07jRBd1KLDH0
    fH19WocLCImeSZooGCZn+jveTuaEH14w6I0EfnpKDcpWVAoIP6I8eSdAttrnTyTn
    Y7VgPrcobyq4WkCVCD/jLUbn97CneF7EHNspXGMTvorMeQJADjy2hF5SginhnPsk
    YR5oWawc6n01mStuLnloI8Uq/6A0AOQoMPkGl/CESZw+NYfe/BnnSeckM917cMKL
    DIKAtwJADEj55Frjj9tKUUO+N9eaEM1PH5eC7yakhIpESccs/XEsaDUIGHNjhctK
    mrbbWu+OlsVRA5z8yJFYIa7gae1mDQJABjtQ8JOQreTDGkFbZR84MbgCWClCIq89
    5R3DFZUiAw4OdS1o4ja+Shc+8DFxkWDNm6+C63g/Amy5sVuWHX2p9QI/a69Cxmns
    TxHoXm1w9Azublk7N7DgB26yqxlTfWJo+ysOFmLEk47g0ekoCwLPxkwXlYIEoad2
    JqPh418DwYExAkACcqrd9+rfxtrbCbTXHEizW7aHR+fVOr9lpXXDEZTlDJ57sRkS
    SpjXbAmylqQuKLqH8h/72RbiP36kEm5ptmw2
    -----END RSA PRIVATE KEY-----
This commit is contained in:
Jaeden Amero 2018-09-07 19:12:36 +01:00
parent ff215726b4
commit 3725bb2d6d
4 changed files with 33 additions and 6 deletions

View file

@ -894,6 +894,16 @@ int mbedtls_rsa_rsassa_pkcs1_v15_sign( mbedtls_rsa_context *ctx,
* Specifications</em> it is advised to keep both hashes the
* same.
*
* \note This function always uses the maximum possible salt size,
* up to the length of the payload hash. This choice of salt
* size complies with FIPS 186-4 §5.5 (e) and RFC 8017 (PKCS#1
* v2.2) §9.1.1 step 3. Furthermore this function enforces a
* minimum salt size which is the hash size minus 2 bytes. If
* this minimum size is too large given the key size (the salt
* size, plus the hash size, plus 2 bytes must be no more than
* the key size in bytes), this function returns
* #MBEDTLS_ERR_RSA_BAD_INPUT_DATA.
*
* \deprecated It is deprecated and discouraged to call this function
* in #MBEDTLS_RSA_PUBLIC mode. Future versions of the library
* are likely to remove the \p mode argument and have it